2023-12-06T17:57:45.861 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-06T17:57:45.861 DEBUG:teuthology.run:Teuthology command: teuthology --name sseshasa-2023-12-06_17:50:03-rados-wip-sseshasa-testing-2023-12-04-1848-distro-default-smithi --archive /home/teuthworker/archive/sseshasa-2023-12-06_17:50:03-rados-wip-sseshasa-testing-2023-12-04-1848-distro-default-smithi/7480579 --owner scheduled_sseshasa@teuthology --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/mon_recovery validater/valgrind} --verbose -- /home/teuthworker/archive/sseshasa-2023-12-06_17:50:03-rados-wip-sseshasa-testing-2023-12-04-1848-distro-default-smithi/7480579/orig.config.yaml 2023-12-06T17:57:45.899 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T17:57:45.978 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/sseshasa-2023-12-06_17:50:03-rados-wip-sseshasa-testing-2023-12-04-1848-distro-default-smithi/7480579 branch: wip-sseshasa-testing-2023-12-04-1848 description: rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/mon_recovery validater/valgrind} email: null first_in_suite: false job_id: '7480579' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: sseshasa-2023-12-06_17:50:03-rados-wip-sseshasa-testing-2023-12-04-1848-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-sseshasa-testing-2023-12-04-1848 ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr1: false ms bind msgr2: true ms inject socket failures: 5000 ms type: async osd heartbeat grace: 80 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osd crush smoke test: false mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 1 debug bluestore: 1 debug monc: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd fast shutdown: false osd max markdown count: 1000 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd memory target: 1610612736 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(MON_DOWN\) - \(OSDMAP_FLAGS\) - \(SMALLER_PGP_NUM\) - \(POOL_APP_NOT_ENABLED\) - \(SLOW OPS\) - slow request - overall HEALTH_ - \(PG_ - \(MON_DOWN\) - \(SLOW_OPS\) - slow request sha1: 96fa2578c622ca9c169af6f426ca8b1616130e58 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: debuginfo: true flavor: default sha1: 96fa2578c622ca9c169af6f426ca8b1616130e58 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-sseshasa-testing-2023-12-04-1848 sha1: 96fa2578c622ca9c169af6f426ca8b1616130e58 owner: scheduled_sseshasa@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 5341 sha1: 96fa2578c622ca9c169af6f426ca8b1616130e58 sleep_before_teardown: 0 subset: 222/150000 suite: rados suite_branch: wip-sseshasa-testing-2023-12-04-1848 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 96fa2578c622ca9c169af6f426ca8b1616130e58 targets: smithi061.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOOmUHjTe45/NHixBF/csb/zjzk1zdv1ekMBaylRiQCGZ9jpEg1Wr/laHEkskFVFyHo59Ut5qZjChz0i6UlJ+Gw= smithi152.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMB/1Zaef7NTky5wA6AE8OcK28GIDiVUGrOtfy53WeFfxKrMrUh4n/UUWyealTa1cbnzMBnXCmaa32+rVaKVxfM= tasks: - install: null - ceph: null - thrashosds: chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - mon_recovery: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: f2de108beb0a6dde7f3e6c96039071828a1f6269 timestamp: 2023-12-06_17:50:03 tube: smithi user: sseshasa verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-12-06T17:57:45.978 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa; will attempt to use it 2023-12-06T17:57:45.979 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks 2023-12-06T17:57:45.979 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-06T17:57:45.981 INFO:teuthology.task.internal:Checking packages... 2023-12-06T17:57:46.004 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '96fa2578c622ca9c169af6f426ca8b1616130e58' 2023-12-06T17:57:46.004 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-06T17:57:46.004 INFO:teuthology.packaging:ref: None 2023-12-06T17:57:46.004 INFO:teuthology.packaging:tag: None 2023-12-06T17:57:46.004 INFO:teuthology.packaging:branch: wip-sseshasa-testing-2023-12-04-1848 2023-12-06T17:57:46.004 INFO:teuthology.packaging:sha1: 96fa2578c622ca9c169af6f426ca8b1616130e58 2023-12-06T17:57:46.005 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-sseshasa-testing-2023-12-04-1848 2023-12-06T17:57:46.143 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7644.g96fa2578 2023-12-06T17:57:46.143 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-06T17:57:46.150 INFO:teuthology.task.internal:no buildpackages task found 2023-12-06T17:57:46.150 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-06T17:57:46.159 INFO:teuthology.task.internal:Saving configuration 2023-12-06T17:57:46.173 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-06T17:57:46.181 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-06T17:57:46.205 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi061.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sseshasa-2023-12-06_17:50:03-rados-wip-sseshasa-testing-2023-12-04-1848-distro-default-smithi/7480579', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-06 17:52:29.601430', 'locked_by': 'scheduled_sseshasa@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOOmUHjTe45/NHixBF/csb/zjzk1zdv1ekMBaylRiQCGZ9jpEg1Wr/laHEkskFVFyHo59Ut5qZjChz0i6UlJ+Gw='} 2023-12-06T17:57:46.228 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sseshasa-2023-12-06_17:50:03-rados-wip-sseshasa-testing-2023-12-04-1848-distro-default-smithi/7480579', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-06 17:52:29.604306', 'locked_by': 'scheduled_sseshasa@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMB/1Zaef7NTky5wA6AE8OcK28GIDiVUGrOtfy53WeFfxKrMrUh4n/UUWyealTa1cbnzMBnXCmaa32+rVaKVxfM='} 2023-12-06T17:57:46.242 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-06T17:57:46.253 INFO:teuthology.task.internal:roles: ubuntu@smithi061.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-12-06T17:57:46.253 INFO:teuthology.task.internal:roles: ubuntu@smithi152.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-12-06T17:57:46.254 INFO:teuthology.run_tasks:Running task console_log... 2023-12-06T17:57:46.325 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1a37a16ee0>, signals=[15]) 2023-12-06T17:57:46.325 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-06T17:57:46.334 INFO:teuthology.task.internal:Opening connections... 2023-12-06T17:57:46.334 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi061.front.sepia.ceph.com 2023-12-06T17:57:46.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T17:57:46.436 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi152.front.sepia.ceph.com 2023-12-06T17:57:46.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T17:57:46.518 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-06T17:57:46.526 DEBUG:teuthology.orchestra.run.smithi061:> uname -m 2023-12-06T17:57:46.544 INFO:teuthology.orchestra.run.smithi061.stdout:x86_64 2023-12-06T17:57:46.544 DEBUG:teuthology.orchestra.run.smithi061:> cat /etc/os-release 2023-12-06T17:57:46.601 INFO:teuthology.orchestra.run.smithi061.stdout:NAME="CentOS Stream" 2023-12-06T17:57:46.601 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION="9" 2023-12-06T17:57:46.601 INFO:teuthology.orchestra.run.smithi061.stdout:ID="centos" 2023-12-06T17:57:46.601 INFO:teuthology.orchestra.run.smithi061.stdout:ID_LIKE="rhel fedora" 2023-12-06T17:57:46.601 INFO:teuthology.orchestra.run.smithi061.stdout:VERSION_ID="9" 2023-12-06T17:57:46.601 INFO:teuthology.orchestra.run.smithi061.stdout:PLATFORM_ID="platform:el9" 2023-12-06T17:57:46.601 INFO:teuthology.orchestra.run.smithi061.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-06T17:57:46.601 INFO:teuthology.orchestra.run.smithi061.stdout:ANSI_COLOR="0;31" 2023-12-06T17:57:46.601 INFO:teuthology.orchestra.run.smithi061.stdout:LOGO="fedora-logo-icon" 2023-12-06T17:57:46.601 INFO:teuthology.orchestra.run.smithi061.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-06T17:57:46.601 INFO:teuthology.orchestra.run.smithi061.stdout:HOME_URL="https://centos.org/" 2023-12-06T17:57:46.601 INFO:teuthology.orchestra.run.smithi061.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-06T17:57:46.602 INFO:teuthology.orchestra.run.smithi061.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-06T17:57:46.602 INFO:teuthology.orchestra.run.smithi061.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-06T17:57:46.602 INFO:teuthology.lock.ops:Updating smithi061.front.sepia.ceph.com on lock server 2023-12-06T17:57:46.628 DEBUG:teuthology.orchestra.run.smithi152:> uname -m 2023-12-06T17:57:46.646 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64 2023-12-06T17:57:46.646 DEBUG:teuthology.orchestra.run.smithi152:> cat /etc/os-release 2023-12-06T17:57:46.703 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="CentOS Stream" 2023-12-06T17:57:46.703 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="9" 2023-12-06T17:57:46.703 INFO:teuthology.orchestra.run.smithi152.stdout:ID="centos" 2023-12-06T17:57:46.703 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE="rhel fedora" 2023-12-06T17:57:46.703 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="9" 2023-12-06T17:57:46.703 INFO:teuthology.orchestra.run.smithi152.stdout:PLATFORM_ID="platform:el9" 2023-12-06T17:57:46.703 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-06T17:57:46.703 INFO:teuthology.orchestra.run.smithi152.stdout:ANSI_COLOR="0;31" 2023-12-06T17:57:46.703 INFO:teuthology.orchestra.run.smithi152.stdout:LOGO="fedora-logo-icon" 2023-12-06T17:57:46.703 INFO:teuthology.orchestra.run.smithi152.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-06T17:57:46.703 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://centos.org/" 2023-12-06T17:57:46.703 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-06T17:57:46.703 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-06T17:57:46.703 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-06T17:57:46.704 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2023-12-06T17:57:46.728 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-06T17:57:46.738 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-06T17:57:46.746 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-06T17:57:46.746 DEBUG:teuthology.orchestra.run.smithi061:> test '!' -e /home/ubuntu/cephtest 2023-12-06T17:57:46.749 DEBUG:teuthology.orchestra.run.smithi152:> test '!' -e /home/ubuntu/cephtest 2023-12-06T17:57:46.764 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-06T17:57:46.771 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-06T17:57:46.772 DEBUG:teuthology.orchestra.run.smithi061:> test -z $(ls -A /var/lib/ceph) 2023-12-06T17:57:46.806 DEBUG:teuthology.orchestra.run.smithi152:> test -z $(ls -A /var/lib/ceph) 2023-12-06T17:57:46.830 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-06T17:57:46.886 INFO:teuthology.run_tasks:Running task kernel... 2023-12-06T17:57:46.902 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-12-06T17:57:46.902 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-12-06T17:57:46.902 DEBUG:teuthology.orchestra.run.smithi061:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-06T17:57:46.903 DEBUG:teuthology.orchestra.run.smithi152:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-06T17:57:46.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-06T17:57:46.919 DEBUG:teuthology.orchestra.run.smithi152:> uname -r 2023-12-06T17:57:46.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-06T17:57:46.920 DEBUG:teuthology.orchestra.run.smithi061:> uname -r 2023-12-06T17:57:46.975 INFO:teuthology.orchestra.run.smithi152.stdout:5.14.0-361.el9.x86_64 2023-12-06T17:57:46.976 INFO:teuthology.task.kernel:Running kernel on smithi152: 5.14.0-361.el9.x86_64 2023-12-06T17:57:46.976 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum install -y kernel 2023-12-06T17:57:46.977 INFO:teuthology.orchestra.run.smithi061.stdout:5.14.0-361.el9.x86_64 2023-12-06T17:57:46.977 INFO:teuthology.task.kernel:Running kernel on smithi061: 5.14.0-361.el9.x86_64 2023-12-06T17:57:46.977 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum install -y kernel 2023-12-06T17:57:49.372 INFO:teuthology.orchestra.run.smithi061.stdout:CentOS-9 - AppStream 8.1 MB/s | 6.4 MB 00:00 2023-12-06T17:57:49.511 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS-9 - AppStream 7.2 MB/s | 6.4 MB 00:00 2023-12-06T17:57:50.864 INFO:teuthology.orchestra.run.smithi061.stdout:CentOS-9 - BaseOS 5.1 MB/s | 2.0 MB 00:00 2023-12-06T17:57:50.995 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS-9 - BaseOS 4.8 MB/s | 2.0 MB 00:00 2023-12-06T17:57:53.746 INFO:teuthology.orchestra.run.smithi061.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 7.9 MB 00:02 2023-12-06T17:57:54.010 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 7.9 MB 00:02 2023-12-06T17:57:56.864 INFO:teuthology.orchestra.run.smithi061.stdout:CentOS Stream 9 - AppStream 9.2 MB/s | 18 MB 00:01 2023-12-06T17:58:00.907 INFO:teuthology.orchestra.run.smithi061.stdout:CentOS Stream 9 - CRB 11 MB/s | 5.8 MB 00:00 2023-12-06T17:58:02.674 INFO:teuthology.orchestra.run.smithi061.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 15 kB 00:00 2023-12-06T17:58:05.036 INFO:teuthology.orchestra.run.smithi061.stdout:Extra Packages for Enterprise Linux 8.9 MB/s | 20 MB 00:02 2023-12-06T17:58:09.942 INFO:teuthology.orchestra.run.smithi061.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2023-12-06T17:58:12.315 INFO:teuthology.orchestra.run.smithi061.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-12-06T17:58:12.315 INFO:teuthology.orchestra.run.smithi061.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-12-06T17:58:12.419 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T17:58:12.427 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T17:58:12.427 INFO:teuthology.orchestra.run.smithi061.stdout: Package Architecture Version Repository Size 2023-12-06T17:58:12.427 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T17:58:12.427 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T17:58:12.428 INFO:teuthology.orchestra.run.smithi061.stdout: kernel x86_64 5.14.0-390.el9 baseos 5.6 M 2023-12-06T17:58:12.428 INFO:teuthology.orchestra.run.smithi061.stdout:Installing dependencies: 2023-12-06T17:58:12.428 INFO:teuthology.orchestra.run.smithi061.stdout: kernel-core x86_64 5.14.0-390.el9 baseos 20 M 2023-12-06T17:58:12.428 INFO:teuthology.orchestra.run.smithi061.stdout: kernel-modules x86_64 5.14.0-390.el9 baseos 38 M 2023-12-06T17:58:12.428 INFO:teuthology.orchestra.run.smithi061.stdout: kernel-modules-core x86_64 5.14.0-390.el9 baseos 32 M 2023-12-06T17:58:12.428 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T17:58:12.428 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T17:58:12.428 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T17:58:12.428 INFO:teuthology.orchestra.run.smithi061.stdout:Install 4 Packages 2023-12-06T17:58:12.428 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T17:58:12.429 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 96 M 2023-12-06T17:58:12.429 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 121 M 2023-12-06T17:58:12.429 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T17:58:14.743 INFO:teuthology.orchestra.run.smithi061.stdout:(1/4): kernel-5.14.0-390.el9.x86_64.rpm 2.5 MB/s | 5.6 MB 00:02 2023-12-06T17:58:22.692 INFO:teuthology.orchestra.run.smithi061.stdout:(2/4): kernel-core-5.14.0-390.el9.x86_64.rpm 2.0 MB/s | 20 MB 00:10 2023-12-06T17:58:29.734 INFO:teuthology.orchestra.run.smithi061.stdout:(3/4): kernel-modules-core-5.14.0-390.el9.x86_6 2.2 MB/s | 32 MB 00:14 2023-12-06T17:58:33.000 INFO:teuthology.orchestra.run.smithi061.stdout:(4/4): kernel-modules-5.14.0-390.el9.x86_64.rpm 1.8 MB/s | 38 MB 00:20 2023-12-06T17:58:33.000 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T17:58:33.001 INFO:teuthology.orchestra.run.smithi061.stdout:Total 4.7 MB/s | 96 MB 00:20 2023-12-06T17:58:33.898 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T17:58:34.315 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T17:58:34.315 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T17:58:35.677 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T17:58:35.677 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T17:58:38.486 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T17:58:39.155 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : kernel-modules-core-5.14.0-390.el9.x86_64 1/4 2023-12-06T17:58:39.284 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : kernel-core-5.14.0-390.el9.x86_64 2/4 2023-12-06T17:58:42.173 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: kernel-core-5.14.0-390.el9.x86_64 2/4 2023-12-06T17:58:42.353 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : kernel-modules-5.14.0-390.el9.x86_64 3/4 2023-12-06T17:58:47.304 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: kernel-modules-5.14.0-390.el9.x86_64 3/4 2023-12-06T17:58:47.431 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : kernel-5.14.0-390.el9.x86_64 4/4 2023-12-06T17:58:52.421 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: kernel-modules-core-5.14.0-390.el9.x86_64 4/4 2023-12-06T17:59:22.790 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: kernel-core-5.14.0-390.el9.x86_64 4/4 2023-12-06T17:59:22.835 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: kernel-modules-5.14.0-390.el9.x86_64 4/4 2023-12-06T17:59:23.254 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: kernel-5.14.0-390.el9.x86_64 4/4 2023-12-06T17:59:23.254 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : kernel-5.14.0-390.el9.x86_64 1/4 2023-12-06T17:59:23.254 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : kernel-core-5.14.0-390.el9.x86_64 2/4 2023-12-06T17:59:23.254 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : kernel-modules-5.14.0-390.el9.x86_64 3/4 2023-12-06T17:59:23.696 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : kernel-modules-core-5.14.0-390.el9.x86_64 4/4 2023-12-06T17:59:23.697 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T17:59:23.697 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T17:59:23.697 INFO:teuthology.orchestra.run.smithi061.stdout: kernel-5.14.0-390.el9.x86_64 2023-12-06T17:59:23.697 INFO:teuthology.orchestra.run.smithi061.stdout: kernel-core-5.14.0-390.el9.x86_64 2023-12-06T17:59:23.697 INFO:teuthology.orchestra.run.smithi061.stdout: kernel-modules-5.14.0-390.el9.x86_64 2023-12-06T17:59:23.697 INFO:teuthology.orchestra.run.smithi061.stdout: kernel-modules-core-5.14.0-390.el9.x86_64 2023-12-06T17:59:23.697 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T17:59:23.697 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T17:59:24.197 DEBUG:teuthology.orchestra.run.smithi061:> rpm -q kernel | sort -rV | head -n 1 2023-12-06T17:59:24.344 INFO:teuthology.orchestra.run.smithi061.stdout:kernel-5.14.0-390.el9.x86_64 2023-12-06T17:59:24.345 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-390.el9.x86_64 2023-12-06T17:59:24.345 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-390.el9.x86_64 2023-12-06T17:59:24.345 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-12-06T17:59:24.345 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-12-06T17:59:24.345 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-12-06T17:59:24.345 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi061.front.sepia.ceph.com, path=None, version=distro) 2023-12-06T17:59:24.346 DEBUG:teuthology.orchestra.run.smithi061:> rpm -q kernel | sort -rV | head -n 1 2023-12-06T17:59:24.468 INFO:teuthology.orchestra.run.smithi061.stdout:kernel-5.14.0-390.el9.x86_64 2023-12-06T17:59:24.469 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-390.el9.x86_64 2023-12-06T17:59:24.469 DEBUG:teuthology.orchestra.run.smithi061:> sudo rpm -qi grub2-tools 2023-12-06T17:59:24.515 INFO:teuthology.orchestra.run.smithi061.stdout:Name : grub2-tools 2023-12-06T17:59:24.515 INFO:teuthology.orchestra.run.smithi061.stdout:Epoch : 1 2023-12-06T17:59:24.515 INFO:teuthology.orchestra.run.smithi061.stdout:Version : 2.06 2023-12-06T17:59:24.515 INFO:teuthology.orchestra.run.smithi061.stdout:Release : 68.el9 2023-12-06T17:59:24.516 INFO:teuthology.orchestra.run.smithi061.stdout:Architecture: x86_64 2023-12-06T17:59:24.516 INFO:teuthology.orchestra.run.smithi061.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-12-06T17:59:24.516 INFO:teuthology.orchestra.run.smithi061.stdout:Group : Unspecified 2023-12-06T17:59:24.516 INFO:teuthology.orchestra.run.smithi061.stdout:Size : 8264254 2023-12-06T17:59:24.516 INFO:teuthology.orchestra.run.smithi061.stdout:License : GPLv3+ 2023-12-06T17:59:24.516 INFO:teuthology.orchestra.run.smithi061.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-12-06T17:59:24.516 INFO:teuthology.orchestra.run.smithi061.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-12-06T17:59:24.516 INFO:teuthology.orchestra.run.smithi061.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-12-06T17:59:24.516 INFO:teuthology.orchestra.run.smithi061.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-12-06T17:59:24.516 INFO:teuthology.orchestra.run.smithi061.stdout:Packager : builder@centos.org 2023-12-06T17:59:24.516 INFO:teuthology.orchestra.run.smithi061.stdout:Vendor : CentOS 2023-12-06T17:59:24.516 INFO:teuthology.orchestra.run.smithi061.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-06T17:59:24.516 INFO:teuthology.orchestra.run.smithi061.stdout:Summary : Support tools for GRUB. 2023-12-06T17:59:24.516 INFO:teuthology.orchestra.run.smithi061.stdout:Description : 2023-12-06T17:59:24.517 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T17:59:24.517 INFO:teuthology.orchestra.run.smithi061.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-06T17:59:24.517 INFO:teuthology.orchestra.run.smithi061.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-06T17:59:24.517 INFO:teuthology.orchestra.run.smithi061.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-06T17:59:24.517 INFO:teuthology.orchestra.run.smithi061.stdout:hardware devices. 2023-12-06T17:59:24.517 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T17:59:24.517 INFO:teuthology.orchestra.run.smithi061.stdout:This subpackage provides tools for support of all platforms. 2023-12-06T17:59:24.518 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-06T17:59:24.518 INFO:teuthology.task.kernel:Updating grub on smithi061 to boot 5.14.0-390.el9.x86_64 2023-12-06T17:59:24.518 DEBUG:teuthology.orchestra.run.smithi061:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-06T17:59:25.122 INFO:teuthology.orchestra.run.smithi061.stderr:Generating grub configuration file ... 2023-12-06T17:59:26.478 INFO:teuthology.orchestra.run.smithi061.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-12-06T17:59:26.511 INFO:teuthology.orchestra.run.smithi061.stderr:done 2023-12-06T17:59:26.514 DEBUG:teuthology.orchestra.run.smithi061:> mktemp 2023-12-06T17:59:26.529 INFO:teuthology.orchestra.run.smithi061.stdout:/tmp/tmp.U4ZgvYcUOX 2023-12-06T17:59:26.530 DEBUG:teuthology.orchestra.run.smithi061:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.U4ZgvYcUOX 2023-12-06T17:59:26.613 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod 0666 /tmp/tmp.U4ZgvYcUOX 2023-12-06T17:59:26.758 DEBUG:teuthology.orchestra.remote:smithi061:/tmp/tmp.U4ZgvYcUOX is 6KB 2023-12-06T17:59:26.806 DEBUG:teuthology.orchestra.run.smithi061:> rm -fr /tmp/tmp.U4ZgvYcUOX 2023-12-06T17:59:26.820 DEBUG:teuthology.orchestra.run.smithi061:> sudo /bin/ls /boot/loader/entries || true 2023-12-06T17:59:26.889 INFO:teuthology.orchestra.run.smithi061.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-12-06T17:59:26.889 INFO:teuthology.orchestra.run.smithi061.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-12-06T17:59:26.889 INFO:teuthology.orchestra.run.smithi061.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-12-06T17:59:26.889 INFO:teuthology.orchestra.run.smithi061.stdout:15949e3c88704300882482eddfd7baae-5.14.0-390.el9.x86_64.conf 2023-12-06T17:59:26.890 DEBUG:teuthology.orchestra.run.smithi061:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-390.el9.x86_64 2023-12-06T17:59:27.028 DEBUG:teuthology.orchestra.run.smithi061:> sudo shutdown -r now 2023-12-06T17:59:57.031 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-12-06T17:59:57.031 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi061.front.sepia.ceph.com' 2023-12-06T17:59:57.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T18:00:15.458 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.61 2023-12-06T18:00:24.459 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi061.front.sepia.ceph.com' 2023-12-06T18:00:24.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T18:00:27.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.61 2023-12-06T18:00:39.527 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi061.front.sepia.ceph.com' 2023-12-06T18:00:39.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T18:01:11.464 DEBUG:teuthology.orchestra.run.smithi061:> true 2023-12-06T18:01:11.911 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi061.front.sepia.ceph.com' 2023-12-06T18:01:11.911 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-390.el9.x86_64"... 2023-12-06T18:01:11.912 DEBUG:teuthology.orchestra.run.smithi061:> uname -r 2023-12-06T18:01:11.927 INFO:teuthology.orchestra.run.smithi061.stdout:5.14.0-390.el9.x86_64 2023-12-06T18:01:11.927 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-390.el9.x86_64 vs 5.14.0-390.el9.x86_64 2023-12-06T18:01:11.927 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-06T18:01:11.927 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-12-06T18:01:12.928 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-12-06T18:01:12.928 DEBUG:teuthology.orchestra.run.smithi061:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-06T18:01:13.025 INFO:teuthology.orchestra.run.smithi061.stdout:ttyS1 2023-12-06T18:01:13.065 DEBUG:teuthology.parallel:result is None 2023-12-06T18:02:18.260 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - AppStream 71 kB/s | 18 MB 04:23 2023-12-06T18:02:24.799 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 5.8 MB 00:03 2023-12-06T18:02:26.567 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 15 kB 00:00 2023-12-06T18:02:31.538 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 4.1 MB/s | 20 MB 00:04 2023-12-06T18:02:36.251 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2023-12-06T18:02:38.424 INFO:teuthology.orchestra.run.smithi152.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-12-06T18:02:38.424 INFO:teuthology.orchestra.run.smithi152.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-12-06T18:02:38.527 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:02:38.538 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:02:38.538 INFO:teuthology.orchestra.run.smithi152.stdout: Package Architecture Version Repository Size 2023-12-06T18:02:38.538 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:02:38.538 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:02:38.538 INFO:teuthology.orchestra.run.smithi152.stdout: kernel x86_64 5.14.0-390.el9 baseos 5.6 M 2023-12-06T18:02:38.538 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2023-12-06T18:02:38.538 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-core x86_64 5.14.0-390.el9 baseos 20 M 2023-12-06T18:02:38.539 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-modules x86_64 5.14.0-390.el9 baseos 38 M 2023-12-06T18:02:38.539 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-modules-core x86_64 5.14.0-390.el9 baseos 32 M 2023-12-06T18:02:38.539 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:02:38.539 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:02:38.539 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:02:38.539 INFO:teuthology.orchestra.run.smithi152.stdout:Install 4 Packages 2023-12-06T18:02:38.539 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:02:38.539 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 96 M 2023-12-06T18:02:38.540 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 121 M 2023-12-06T18:02:38.540 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:02:39.377 INFO:teuthology.orchestra.run.smithi152.stdout:(1/4): kernel-5.14.0-390.el9.x86_64.rpm 8.0 MB/s | 5.6 MB 00:00 2023-12-06T18:02:40.536 INFO:teuthology.orchestra.run.smithi152.stdout:(2/4): kernel-core-5.14.0-390.el9.x86_64.rpm 11 MB/s | 20 MB 00:01 2023-12-06T18:02:43.394 INFO:teuthology.orchestra.run.smithi152.stdout:(3/4): kernel-modules-5.14.0-390.el9.x86_64.rpm 8.0 MB/s | 38 MB 00:04 2023-12-06T18:02:44.314 INFO:teuthology.orchestra.run.smithi152.stdout:(4/4): kernel-modules-core-5.14.0-390.el9.x86_6 6.6 MB/s | 32 MB 00:04 2023-12-06T18:02:44.314 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:02:44.314 INFO:teuthology.orchestra.run.smithi152.stdout:Total 17 MB/s | 96 MB 00:05 2023-12-06T18:02:45.165 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:02:45.533 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:02:45.533 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:02:47.136 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:02:47.137 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:02:50.127 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:02:50.768 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : kernel-modules-core-5.14.0-390.el9.x86_64 1/4 2023-12-06T18:02:50.898 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : kernel-core-5.14.0-390.el9.x86_64 2/4 2023-12-06T18:02:53.654 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-core-5.14.0-390.el9.x86_64 2/4 2023-12-06T18:02:53.945 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : kernel-modules-5.14.0-390.el9.x86_64 3/4 2023-12-06T18:02:58.581 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-modules-5.14.0-390.el9.x86_64 3/4 2023-12-06T18:02:58.857 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : kernel-5.14.0-390.el9.x86_64 4/4 2023-12-06T18:03:03.464 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-modules-core-5.14.0-390.el9.x86_64 4/4 2023-12-06T18:03:32.178 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-core-5.14.0-390.el9.x86_64 4/4 2023-12-06T18:03:32.221 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-modules-5.14.0-390.el9.x86_64 4/4 2023-12-06T18:03:32.652 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: kernel-5.14.0-390.el9.x86_64 4/4 2023-12-06T18:03:32.652 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : kernel-5.14.0-390.el9.x86_64 1/4 2023-12-06T18:03:32.652 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : kernel-core-5.14.0-390.el9.x86_64 2/4 2023-12-06T18:03:32.652 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : kernel-modules-5.14.0-390.el9.x86_64 3/4 2023-12-06T18:03:33.096 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : kernel-modules-core-5.14.0-390.el9.x86_64 4/4 2023-12-06T18:03:33.096 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:03:33.096 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:03:33.096 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-5.14.0-390.el9.x86_64 2023-12-06T18:03:33.096 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-core-5.14.0-390.el9.x86_64 2023-12-06T18:03:33.096 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-modules-5.14.0-390.el9.x86_64 2023-12-06T18:03:33.097 INFO:teuthology.orchestra.run.smithi152.stdout: kernel-modules-core-5.14.0-390.el9.x86_64 2023-12-06T18:03:33.097 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:03:33.097 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:03:33.608 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q kernel | sort -rV | head -n 1 2023-12-06T18:03:33.749 INFO:teuthology.orchestra.run.smithi152.stdout:kernel-5.14.0-390.el9.x86_64 2023-12-06T18:03:33.749 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-390.el9.x86_64 2023-12-06T18:03:33.749 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-390.el9.x86_64 2023-12-06T18:03:33.749 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-12-06T18:03:33.749 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-12-06T18:03:33.750 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-12-06T18:03:33.750 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi152.front.sepia.ceph.com, path=None, version=distro) 2023-12-06T18:03:33.750 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q kernel | sort -rV | head -n 1 2023-12-06T18:03:33.866 INFO:teuthology.orchestra.run.smithi152.stdout:kernel-5.14.0-390.el9.x86_64 2023-12-06T18:03:33.867 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-390.el9.x86_64 2023-12-06T18:03:33.867 DEBUG:teuthology.orchestra.run.smithi152:> sudo rpm -qi grub2-tools 2023-12-06T18:03:33.905 INFO:teuthology.orchestra.run.smithi152.stdout:Name : grub2-tools 2023-12-06T18:03:33.905 INFO:teuthology.orchestra.run.smithi152.stdout:Epoch : 1 2023-12-06T18:03:33.905 INFO:teuthology.orchestra.run.smithi152.stdout:Version : 2.06 2023-12-06T18:03:33.905 INFO:teuthology.orchestra.run.smithi152.stdout:Release : 68.el9 2023-12-06T18:03:33.905 INFO:teuthology.orchestra.run.smithi152.stdout:Architecture: x86_64 2023-12-06T18:03:33.905 INFO:teuthology.orchestra.run.smithi152.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-12-06T18:03:33.905 INFO:teuthology.orchestra.run.smithi152.stdout:Group : Unspecified 2023-12-06T18:03:33.906 INFO:teuthology.orchestra.run.smithi152.stdout:Size : 8264254 2023-12-06T18:03:33.906 INFO:teuthology.orchestra.run.smithi152.stdout:License : GPLv3+ 2023-12-06T18:03:33.906 INFO:teuthology.orchestra.run.smithi152.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-12-06T18:03:33.906 INFO:teuthology.orchestra.run.smithi152.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-12-06T18:03:33.906 INFO:teuthology.orchestra.run.smithi152.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-12-06T18:03:33.906 INFO:teuthology.orchestra.run.smithi152.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-12-06T18:03:33.906 INFO:teuthology.orchestra.run.smithi152.stdout:Packager : builder@centos.org 2023-12-06T18:03:33.906 INFO:teuthology.orchestra.run.smithi152.stdout:Vendor : CentOS 2023-12-06T18:03:33.906 INFO:teuthology.orchestra.run.smithi152.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-06T18:03:33.906 INFO:teuthology.orchestra.run.smithi152.stdout:Summary : Support tools for GRUB. 2023-12-06T18:03:33.906 INFO:teuthology.orchestra.run.smithi152.stdout:Description : 2023-12-06T18:03:33.907 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:03:33.907 INFO:teuthology.orchestra.run.smithi152.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-06T18:03:33.907 INFO:teuthology.orchestra.run.smithi152.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-06T18:03:33.907 INFO:teuthology.orchestra.run.smithi152.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-06T18:03:33.907 INFO:teuthology.orchestra.run.smithi152.stdout:hardware devices. 2023-12-06T18:03:33.907 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:03:33.907 INFO:teuthology.orchestra.run.smithi152.stdout:This subpackage provides tools for support of all platforms. 2023-12-06T18:03:33.908 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-06T18:03:33.908 INFO:teuthology.task.kernel:Updating grub on smithi152 to boot 5.14.0-390.el9.x86_64 2023-12-06T18:03:33.908 DEBUG:teuthology.orchestra.run.smithi152:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-06T18:03:34.543 INFO:teuthology.orchestra.run.smithi152.stderr:Generating grub configuration file ... 2023-12-06T18:03:35.950 INFO:teuthology.orchestra.run.smithi152.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-12-06T18:03:35.968 INFO:teuthology.orchestra.run.smithi152.stderr:done 2023-12-06T18:03:35.970 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2023-12-06T18:03:35.984 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.6CVyPpVNDY 2023-12-06T18:03:35.985 DEBUG:teuthology.orchestra.run.smithi152:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6CVyPpVNDY 2023-12-06T18:03:36.053 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.6CVyPpVNDY 2023-12-06T18:03:36.187 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.6CVyPpVNDY is 6KB 2023-12-06T18:03:36.236 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.6CVyPpVNDY 2023-12-06T18:03:36.250 DEBUG:teuthology.orchestra.run.smithi152:> sudo /bin/ls /boot/loader/entries || true 2023-12-06T18:03:36.316 INFO:teuthology.orchestra.run.smithi152.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-12-06T18:03:36.316 INFO:teuthology.orchestra.run.smithi152.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-12-06T18:03:36.316 INFO:teuthology.orchestra.run.smithi152.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-12-06T18:03:36.316 INFO:teuthology.orchestra.run.smithi152.stdout:15949e3c88704300882482eddfd7baae-5.14.0-390.el9.x86_64.conf 2023-12-06T18:03:36.317 DEBUG:teuthology.orchestra.run.smithi152:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-390.el9.x86_64 2023-12-06T18:03:36.479 DEBUG:teuthology.orchestra.run.smithi152:> sudo shutdown -r now 2023-12-06T18:04:06.482 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-12-06T18:04:06.482 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2023-12-06T18:04:06.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T18:04:40.930 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.152 2023-12-06T18:04:49.932 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2023-12-06T18:04:49.933 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T18:05:08.322 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.152 2023-12-06T18:05:20.323 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2023-12-06T18:05:20.324 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T18:05:23.394 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.152 2023-12-06T18:05:38.395 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2023-12-06T18:05:38.396 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T18:05:38.721 DEBUG:teuthology.orchestra.run.smithi152:> true 2023-12-06T18:05:39.091 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2023-12-06T18:05:39.091 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-390.el9.x86_64"... 2023-12-06T18:05:39.092 DEBUG:teuthology.orchestra.run.smithi152:> uname -r 2023-12-06T18:05:39.107 INFO:teuthology.orchestra.run.smithi152.stdout:5.14.0-390.el9.x86_64 2023-12-06T18:05:39.107 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-390.el9.x86_64 vs 5.14.0-390.el9.x86_64 2023-12-06T18:05:39.107 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-06T18:05:39.108 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-12-06T18:05:40.109 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-12-06T18:05:40.109 DEBUG:teuthology.orchestra.run.smithi152:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-06T18:05:40.192 INFO:teuthology.orchestra.run.smithi152.stdout:ttyS1 2023-12-06T18:05:40.229 DEBUG:teuthology.parallel:result is None 2023-12-06T18:05:40.230 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-06T18:05:40.237 INFO:teuthology.task.internal:Creating test directory... 2023-12-06T18:05:40.238 DEBUG:teuthology.orchestra.run.smithi061:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-06T18:05:40.241 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-06T18:05:40.258 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-06T18:05:40.267 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-06T18:05:40.275 INFO:teuthology.task.internal:Creating archive directory... 2023-12-06T18:05:40.275 DEBUG:teuthology.orchestra.run.smithi061:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-06T18:05:40.300 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-06T18:05:40.346 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-06T18:05:40.353 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-06T18:05:40.353 DEBUG:teuthology.orchestra.run.smithi061:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-06T18:05:40.365 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-06T18:05:40.403 INFO:teuthology.orchestra.run.smithi061.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-06T18:05:40.418 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-06T18:05:40.421 INFO:teuthology.orchestra.run.smithi061.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-06T18:05:40.432 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-06T18:05:40.434 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-06T18:05:40.442 INFO:teuthology.task.internal:Configuring sudo... 2023-12-06T18:05:40.443 DEBUG:teuthology.orchestra.run.smithi061:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-06T18:05:40.465 DEBUG:teuthology.orchestra.run.smithi152:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-06T18:05:40.504 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-06T18:05:40.515 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-06T18:05:40.515 DEBUG:teuthology.orchestra.run.smithi061:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-06T18:05:40.539 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-06T18:05:40.560 DEBUG:teuthology.orchestra.run.smithi061:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-06T18:05:40.640 DEBUG:teuthology.orchestra.run.smithi061:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-06T18:05:40.689 DEBUG:teuthology.orchestra.run.smithi061:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-06T18:05:40.770 DEBUG:teuthology.orchestra.run.smithi061:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-06T18:05:40.817 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:05:40.817 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-06T18:05:40.887 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-06T18:05:40.929 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-06T18:05:40.981 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-06T18:05:41.063 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-06T18:05:41.108 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:05:41.108 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-06T18:05:41.176 DEBUG:teuthology.orchestra.run.smithi061:> sudo service rsyslog restart 2023-12-06T18:05:41.178 DEBUG:teuthology.orchestra.run.smithi152:> sudo service rsyslog restart 2023-12-06T18:05:41.231 INFO:teuthology.orchestra.run.smithi061.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-06T18:05:41.273 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-06T18:05:41.763 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-06T18:05:41.772 INFO:teuthology.task.internal:Starting timer... 2023-12-06T18:05:41.772 INFO:teuthology.run_tasks:Running task pcp... 2023-12-06T18:05:41.789 INFO:teuthology.run_tasks:Running task selinux... 2023-12-06T18:05:41.803 DEBUG:teuthology.orchestra.run.smithi061:> sudo service auditd rotate 2023-12-06T18:05:41.852 INFO:teuthology.orchestra.run.smithi061.stdout:Rotating logs: 2023-12-06T18:05:41.854 DEBUG:teuthology.orchestra.run.smithi152:> sudo service auditd rotate 2023-12-06T18:05:41.896 INFO:teuthology.orchestra.run.smithi152.stdout:Rotating logs: 2023-12-06T18:05:41.898 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-06T18:05:41.898 DEBUG:teuthology.orchestra.run.smithi061:> /usr/sbin/getenforce 2023-12-06T18:05:41.920 INFO:teuthology.orchestra.run.smithi061.stdout:Permissive 2023-12-06T18:05:41.920 DEBUG:teuthology.orchestra.run.smithi152:> /usr/sbin/getenforce 2023-12-06T18:05:41.968 INFO:teuthology.orchestra.run.smithi152.stdout:Permissive 2023-12-06T18:05:41.968 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi061.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi152.front.sepia.ceph.com': 'permissive'} 2023-12-06T18:05:41.969 DEBUG:teuthology.orchestra.run.smithi061:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-06T18:05:42.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-06T18:05:42.001 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-06T18:05:42.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-06T18:05:42.035 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-06T18:05:42.036 DEBUG:teuthology.orchestra.run.smithi061:> sudo /usr/sbin/setenforce permissive 2023-12-06T18:05:42.080 DEBUG:teuthology.orchestra.run.smithi152:> sudo /usr/sbin/setenforce permissive 2023-12-06T18:05:42.112 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-06T18:05:42.122 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-12-06T18:05:42.122 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-06T18:05:42.139 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-06T18:05:42.140 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi061.front.sepia.ceph.com,smithi152.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-06T18:11:25.063 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi061.front.sepia.ceph.com'), Remote(name='ubuntu@smithi152.front.sepia.ceph.com')] 2023-12-06T18:11:25.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi061.front.sepia.ceph.com' 2023-12-06T18:11:25.065 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi061.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T18:11:25.164 DEBUG:teuthology.orchestra.run.smithi061:> true 2023-12-06T18:11:25.236 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi061.front.sepia.ceph.com' 2023-12-06T18:11:25.236 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2023-12-06T18:11:25.237 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-06T18:11:25.319 DEBUG:teuthology.orchestra.run.smithi152:> true 2023-12-06T18:11:25.389 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2023-12-06T18:11:25.389 INFO:teuthology.run_tasks:Running task clock... 2023-12-06T18:11:25.399 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-06T18:11:25.400 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-06T18:11:25.400 DEBUG:teuthology.orchestra.run.smithi061:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-06T18:11:25.403 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-06T18:11:25.403 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-06T18:11:25.435 INFO:teuthology.orchestra.run.smithi061.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-06T18:11:25.456 INFO:teuthology.orchestra.run.smithi061.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-06T18:11:25.460 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-06T18:11:25.476 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-06T18:11:25.492 INFO:teuthology.orchestra.run.smithi061.stderr:sudo: ntpd: command not found 2023-12-06T18:11:25.505 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: ntpd: command not found 2023-12-06T18:11:25.509 INFO:teuthology.orchestra.run.smithi061.stdout:506 Cannot talk to daemon 2023-12-06T18:11:25.519 INFO:teuthology.orchestra.run.smithi152.stdout:506 Cannot talk to daemon 2023-12-06T18:11:25.529 INFO:teuthology.orchestra.run.smithi061.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-06T18:11:25.535 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-06T18:11:25.549 INFO:teuthology.orchestra.run.smithi061.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-06T18:11:25.552 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-06T18:11:25.595 INFO:teuthology.orchestra.run.smithi152.stderr:bash: line 1: ntpq: command not found 2023-12-06T18:11:25.596 INFO:teuthology.orchestra.run.smithi061.stderr:bash: line 1: ntpq: command not found 2023-12-06T18:11:25.598 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-06T18:11:25.598 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2023-12-06T18:11:25.598 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T18:11:25.598 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T18:11:25.599 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T18:11:25.599 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T18:11:25.600 INFO:teuthology.orchestra.run.smithi061.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-06T18:11:25.600 INFO:teuthology.orchestra.run.smithi061.stdout:=============================================================================== 2023-12-06T18:11:25.600 INFO:teuthology.orchestra.run.smithi061.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T18:11:25.600 INFO:teuthology.orchestra.run.smithi061.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T18:11:25.600 INFO:teuthology.orchestra.run.smithi061.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T18:11:25.600 INFO:teuthology.orchestra.run.smithi061.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T18:11:25.601 INFO:teuthology.run_tasks:Running task install... 2023-12-06T18:11:25.624 DEBUG:teuthology.task.install:project ceph 2023-12-06T18:11:25.625 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': '96fa2578c622ca9c169af6f426ca8b1616130e58'}} 2023-12-06T18:11:25.625 DEBUG:teuthology.task.install:config {'debuginfo': True, 'flavor': 'default', 'sha1': '96fa2578c622ca9c169af6f426ca8b1616130e58'} 2023-12-06T18:11:25.625 INFO:teuthology.task.install:Using flavor: default 2023-12-06T18:11:25.630 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse', 'ceph-common-dbg', 'ceph-fuse-dbg', 'ceph-mds-dbg', 'ceph-mgr-dbg', 'ceph-mon-dbg', 'ceph-osd-dbg', 'ceph-test-dbg', 'libcephfs2-dbg', 'librados2-dbg', 'libradosstriper1-dbg', 'librbd1-dbg', 'librgw2-dbg', 'radosgw-dbg', 'rbd-fuse-dbg', 'rbd-mirror-dbg', 'rbd-nbd-dbg'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo']} 2023-12-06T18:11:25.630 INFO:teuthology.task.install:extra packages: [] 2023-12-06T18:11:25.631 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96fa2578c622ca9c169af6f426ca8b1616130e58', 'tag': None, 'wait_for_package': False} 2023-12-06T18:11:25.631 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96fa2578c622ca9c169af6f426ca8b1616130e58 2023-12-06T18:11:25.633 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96fa2578c622ca9c169af6f426ca8b1616130e58', 'tag': None, 'wait_for_package': False} 2023-12-06T18:11:25.633 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96fa2578c622ca9c169af6f426ca8b1616130e58 2023-12-06T18:11:25.777 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-sseshasa-testing-2023-12-04-1848/96fa2578c622ca9c169af6f426ca8b1616130e58/centos/9/flavors/default/ 2023-12-06T18:11:25.777 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7644.g96fa2578 2023-12-06T18:11:25.884 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-sseshasa-testing-2023-12-04-1848/96fa2578c622ca9c169af6f426ca8b1616130e58/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-sseshasa-testing-2023-12-04-1848/96fa2578c622ca9c169af6f426ca8b1616130e58/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-sseshasa-testing-2023-12-04-1848/96fa2578c622ca9c169af6f426ca8b1616130e58/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-06T18:11:25.885 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:11:25.885 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-06T18:11:25.887 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-sseshasa-testing-2023-12-04-1848/96fa2578c622ca9c169af6f426ca8b1616130e58/centos/9/flavors/default/ 2023-12-06T18:11:25.887 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7644.g96fa2578 2023-12-06T18:11:25.921 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2023-12-06T18:11:25.922 DEBUG:teuthology.orchestra.run.smithi061:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96fa2578c622ca9c169af6f426ca8b1616130e58/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-06T18:11:25.994 DEBUG:teuthology.orchestra.run.smithi061:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-06T18:11:26.054 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-sseshasa-testing-2023-12-04-1848/96fa2578c622ca9c169af6f426ca8b1616130e58/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-sseshasa-testing-2023-12-04-1848/96fa2578c622ca9c169af6f426ca8b1616130e58/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-sseshasa-testing-2023-12-04-1848/96fa2578c622ca9c169af6f426ca8b1616130e58/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-06T18:11:26.054 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:11:26.054 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-06T18:11:26.080 DEBUG:teuthology.orchestra.run.smithi061:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-06T18:11:26.082 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2023-12-06T18:11:26.082 DEBUG:teuthology.orchestra.run.smithi152:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96fa2578c622ca9c169af6f426ca8b1616130e58/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-06T18:11:26.110 INFO:teuthology.orchestra.run.smithi061.stdout:check_obsoletes = 1 2023-12-06T18:11:26.112 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum clean all 2023-12-06T18:11:26.151 DEBUG:teuthology.orchestra.run.smithi152:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-06T18:11:26.230 DEBUG:teuthology.orchestra.run.smithi152:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-06T18:11:26.297 INFO:teuthology.orchestra.run.smithi152.stdout:check_obsoletes = 1 2023-12-06T18:11:26.300 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum clean all 2023-12-06T18:11:26.383 INFO:teuthology.orchestra.run.smithi061.stdout:73 files removed 2023-12-06T18:11:26.410 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-radosgw 2023-12-06T18:11:26.548 INFO:teuthology.orchestra.run.smithi152.stdout:73 files removed 2023-12-06T18:11:26.575 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-radosgw 2023-12-06T18:11:27.290 INFO:teuthology.orchestra.run.smithi061.stdout:ceph packages for x86_64 143 kB/s | 78 kB 00:00 2023-12-06T18:11:27.385 INFO:teuthology.orchestra.run.smithi152.stdout:ceph packages for x86_64 159 kB/s | 78 kB 00:00 2023-12-06T18:11:27.809 INFO:teuthology.orchestra.run.smithi061.stdout:ceph noarch packages 20 kB/s | 9.5 kB 00:00 2023-12-06T18:11:27.844 INFO:teuthology.orchestra.run.smithi152.stdout:ceph noarch packages 23 kB/s | 9.5 kB 00:00 2023-12-06T18:11:29.225 INFO:teuthology.orchestra.run.smithi061.stdout:ceph source packages 1.4 kB/s | 1.9 kB 00:01 2023-12-06T18:11:29.329 INFO:teuthology.orchestra.run.smithi152.stdout:ceph source packages 1.3 kB/s | 1.9 kB 00:01 2023-12-06T18:11:29.980 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS-9 - AppStream 10 MB/s | 6.4 MB 00:00 2023-12-06T18:11:30.043 INFO:teuthology.orchestra.run.smithi061.stdout:CentOS-9 - AppStream 8.2 MB/s | 6.4 MB 00:00 2023-12-06T18:11:31.485 INFO:teuthology.orchestra.run.smithi061.stdout:CentOS-9 - BaseOS 5.7 MB/s | 2.0 MB 00:00 2023-12-06T18:11:31.922 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS-9 - BaseOS 2.2 MB/s | 2.0 MB 00:00 2023-12-06T18:11:32.284 INFO:teuthology.orchestra.run.smithi061.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 7.9 MB 00:00 2023-12-06T18:11:33.121 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - BaseOS 9.2 MB/s | 7.9 MB 00:00 2023-12-06T18:11:35.802 INFO:teuthology.orchestra.run.smithi061.stdout:CentOS Stream 9 - AppStream 7.7 MB/s | 18 MB 00:02 2023-12-06T18:11:36.589 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - AppStream 7.6 MB/s | 18 MB 00:02 2023-12-06T18:11:40.903 INFO:teuthology.orchestra.run.smithi061.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 5.8 MB 00:01 2023-12-06T18:11:41.456 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 5.8 MB 00:01 2023-12-06T18:11:42.354 INFO:teuthology.orchestra.run.smithi061.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 15 kB 00:00 2023-12-06T18:11:42.915 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 15 kB 00:00 2023-12-06T18:11:43.876 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 24 MB/s | 20 MB 00:00 2023-12-06T18:11:46.008 INFO:teuthology.orchestra.run.smithi061.stdout:Extra Packages for Enterprise Linux 5.6 MB/s | 20 MB 00:03 2023-12-06T18:11:48.701 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2023-12-06T18:11:50.830 INFO:teuthology.orchestra.run.smithi061.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2023-12-06T18:11:51.257 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:11:51.283 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:11:51.283 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2023-12-06T18:11:51.283 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:11:51.283 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:11:51.283 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 8.6 M 2023-12-06T18:11:51.283 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrading: 2023-12-06T18:11:51.283 INFO:teuthology.orchestra.run.smithi152.stdout: librados2 x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 3.3 M 2023-12-06T18:11:51.283 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1 x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 3.0 M 2023-12-06T18:11:51.283 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2023-12-06T18:11:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: boost-program-options 2023-12-06T18:11:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-12-06T18:11:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 5.2 M 2023-12-06T18:11:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 19 M 2023-12-06T18:11:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 24 k 2023-12-06T18:11:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-12-06T18:11:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-12-06T18:11:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-12-06T18:11:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2 x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 718 k 2023-12-06T18:11:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-12-06T18:11:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-12-06T18:11:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-12-06T18:11:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1 x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 476 k 2023-12-06T18:11:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-12-06T18:11:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2 x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 4.8 M 2023-12-06T18:11:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-12-06T18:11:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-12-06T18:11:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-12-06T18:11:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse 2023-12-06T18:11:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 45 k 2023-12-06T18:11:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 129 k 2023-12-06T18:11:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 161 k 2023-12-06T18:11:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-12-06T18:11:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 321 k 2023-12-06T18:11:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 297 k 2023-12-06T18:11:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 99 k 2023-12-06T18:11:51.286 INFO:teuthology.orchestra.run.smithi152.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-12-06T18:11:51.286 INFO:teuthology.orchestra.run.smithi152.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-12-06T18:11:51.286 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:11:51.286 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:11:51.286 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:11:51.286 INFO:teuthology.orchestra.run.smithi152.stdout:Install 27 Packages 2023-12-06T18:11:51.286 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrade 2 Packages 2023-12-06T18:11:51.286 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:11:51.287 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 55 M 2023-12-06T18:11:51.287 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:11:52.099 INFO:teuthology.orchestra.run.smithi152.stdout:(1/29): ceph-base-18.0.0-7644.g96fa2578.el9.x86 7.2 MB/s | 5.2 MB 00:00 2023-12-06T18:11:52.158 INFO:teuthology.orchestra.run.smithi152.stdout:(2/29): ceph-selinux-18.0.0-7644.g96fa2578.el9. 412 kB/s | 24 kB 00:00 2023-12-06T18:11:52.317 INFO:teuthology.orchestra.run.smithi152.stdout:(3/29): libcephfs2-18.0.0-7644.g96fa2578.el9.x8 4.4 MB/s | 718 kB 00:00 2023-12-06T18:11:52.459 INFO:teuthology.orchestra.run.smithi152.stdout:(4/29): ceph-radosgw-18.0.0-7644.g96fa2578.el9. 7.9 MB/s | 8.6 MB 00:01 2023-12-06T18:11:52.484 INFO:teuthology.orchestra.run.smithi152.stdout:(5/29): libradosstriper1-18.0.0-7644.g96fa2578. 2.8 MB/s | 476 kB 00:00 2023-12-06T18:11:52.584 INFO:teuthology.orchestra.run.smithi152.stdout:(6/29): python3-ceph-argparse-18.0.0-7644.g96fa 451 kB/s | 45 kB 00:00 2023-12-06T18:11:52.710 INFO:teuthology.orchestra.run.smithi152.stdout:(7/29): python3-ceph-common-18.0.0-7644.g96fa25 1.0 MB/s | 129 kB 00:00 2023-12-06T18:11:52.810 INFO:teuthology.orchestra.run.smithi152.stdout:(8/29): librgw2-18.0.0-7644.g96fa2578.el9.x86_6 14 MB/s | 4.8 MB 00:00 2023-12-06T18:11:52.835 INFO:teuthology.orchestra.run.smithi152.stdout:(9/29): python3-cephfs-18.0.0-7644.g96fa2578.el 1.3 MB/s | 161 kB 00:00 2023-12-06T18:11:52.877 INFO:teuthology.orchestra.run.smithi152.stdout:(10/29): python3-rados-18.0.0-7644.g96fa2578.el 4.7 MB/s | 321 kB 00:00 2023-12-06T18:11:52.903 INFO:teuthology.orchestra.run.smithi152.stdout:(11/29): python3-rbd-18.0.0-7644.g96fa2578.el9. 4.3 MB/s | 297 kB 00:00 2023-12-06T18:11:52.945 INFO:teuthology.orchestra.run.smithi152.stdout:(12/29): python3-rgw-18.0.0-7644.g96fa2578.el9. 1.5 MB/s | 99 kB 00:00 2023-12-06T18:11:53.112 INFO:teuthology.orchestra.run.smithi152.stdout:(13/29): boost-program-options-1.75.0-8.el9.x86 507 kB/s | 106 kB 00:00 2023-12-06T18:11:53.145 INFO:teuthology.orchestra.run.smithi152.stdout:(14/29): libpmemobj-1.12.1-1.el9.x86_64.rpm 902 kB/s | 159 kB 00:00 2023-12-06T18:11:53.162 INFO:teuthology.orchestra.run.smithi152.stdout:(15/29): librabbitmq-0.11.0-7.el9.x86_64.rpm 880 kB/s | 44 kB 00:00 2023-12-06T18:11:53.237 INFO:teuthology.orchestra.run.smithi152.stdout:(16/29): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 661 kB 00:00 2023-12-06T18:11:53.271 INFO:teuthology.orchestra.run.smithi152.stdout:(17/29): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 282 kB 00:00 2023-12-06T18:11:53.288 INFO:teuthology.orchestra.run.smithi152.stdout:(18/29): python3-prettytable-0.7.2-27.el9.noarc 823 kB/s | 41 kB 00:00 2023-12-06T18:11:53.487 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:11:53.497 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:11:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repository Size 2023-12-06T18:11:53.497 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:11:53.497 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:11:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-radosgw x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 8.6 M 2023-12-06T18:11:53.497 INFO:teuthology.orchestra.run.smithi061.stdout:Upgrading: 2023-12-06T18:11:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: librados2 x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 3.3 M 2023-12-06T18:11:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: librbd1 x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 3.0 M 2023-12-06T18:11:53.497 INFO:teuthology.orchestra.run.smithi061.stdout:Installing dependencies: 2023-12-06T18:11:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: boost-program-options 2023-12-06T18:11:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-12-06T18:11:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-base x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 5.2 M 2023-12-06T18:11:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-common x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 19 M 2023-12-06T18:11:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-selinux x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 24 k 2023-12-06T18:11:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-12-06T18:11:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-12-06T18:11:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-12-06T18:11:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: libcephfs2 x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 718 k 2023-12-06T18:11:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-12-06T18:11:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-12-06T18:11:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-12-06T18:11:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: libradosstriper1 x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 476 k 2023-12-06T18:11:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-12-06T18:11:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: librgw2 x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 4.8 M 2023-12-06T18:11:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-12-06T18:11:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-12-06T18:11:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-12-06T18:11:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: python3-ceph-argparse 2023-12-06T18:11:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 45 k 2023-12-06T18:11:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: python3-ceph-common x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 129 k 2023-12-06T18:11:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cephfs x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 161 k 2023-12-06T18:11:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-12-06T18:11:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rados x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 321 k 2023-12-06T18:11:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rbd x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 297 k 2023-12-06T18:11:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rgw x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 99 k 2023-12-06T18:11:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-12-06T18:11:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-12-06T18:11:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:11:53.499 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:11:53.499 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:11:53.499 INFO:teuthology.orchestra.run.smithi061.stdout:Install 27 Packages 2023-12-06T18:11:53.500 INFO:teuthology.orchestra.run.smithi061.stdout:Upgrade 2 Packages 2023-12-06T18:11:53.500 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:11:53.500 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 55 M 2023-12-06T18:11:53.500 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:11:53.547 INFO:teuthology.orchestra.run.smithi152.stdout:(19/29): ceph-common-18.0.0-7644.g96fa2578.el9. 8.7 MB/s | 19 MB 00:02 2023-12-06T18:11:53.647 INFO:teuthology.orchestra.run.smithi152.stdout:(20/29): gperftools-libs-2.9.1-2.el9.x86_64.rpm 823 kB/s | 309 kB 00:00 2023-12-06T18:11:53.672 INFO:teuthology.orchestra.run.smithi152.stdout:(21/29): libarrow-doc-9.0.0-9.el9.noarch.rpm 197 kB/s | 24 kB 00:00 2023-12-06T18:11:53.698 INFO:teuthology.orchestra.run.smithi152.stdout:(22/29): liboath-2.6.7-2.el9.x86_64.rpm 972 kB/s | 49 kB 00:00 2023-12-06T18:11:53.723 INFO:teuthology.orchestra.run.smithi152.stdout:(23/29): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2023-12-06T18:11:53.823 INFO:teuthology.orchestra.run.smithi152.stdout:(24/29): parquet-libs-9.0.0-9.el9.x86_64.rpm 6.5 MB/s | 837 kB 00:00 2023-12-06T18:11:53.856 INFO:teuthology.orchestra.run.smithi152.stdout:(25/29): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2023-12-06T18:11:53.990 INFO:teuthology.orchestra.run.smithi152.stdout:(26/29): libarrow-9.0.0-9.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2023-12-06T18:11:54.049 INFO:teuthology.orchestra.run.smithi152.stdout:(27/29): thrift-0.15.0-2.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2023-12-06T18:11:54.457 INFO:teuthology.orchestra.run.smithi152.stdout:(28/29): librados2-18.0.0-7644.g96fa2578.el9.x8 5.5 MB/s | 3.3 MB 00:00 2023-12-06T18:11:54.641 INFO:teuthology.orchestra.run.smithi152.stdout:(29/29): librbd1-18.0.0-7644.g96fa2578.el9.x86_ 4.7 MB/s | 3.0 MB 00:00 2023-12-06T18:11:54.647 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:11:54.647 INFO:teuthology.orchestra.run.smithi152.stdout:Total 16 MB/s | 55 MB 00:03 2023-12-06T18:11:54.648 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:11:54.682 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:11:54.682 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:11:54.703 INFO:teuthology.orchestra.run.smithi061.stdout:(1/29): ceph-base-18.0.0-7644.g96fa2578.el9.x86 5.0 MB/s | 5.2 MB 00:01 2023-12-06T18:11:54.762 INFO:teuthology.orchestra.run.smithi061.stdout:(2/29): ceph-selinux-18.0.0-7644.g96fa2578.el9. 411 kB/s | 24 kB 00:00 2023-12-06T18:11:54.879 INFO:teuthology.orchestra.run.smithi061.stdout:(3/29): libcephfs2-18.0.0-7644.g96fa2578.el9.x8 6.0 MB/s | 718 kB 00:00 2023-12-06T18:11:54.963 INFO:teuthology.orchestra.run.smithi061.stdout:(4/29): libradosstriper1-18.0.0-7644.g96fa2578. 5.6 MB/s | 476 kB 00:00 2023-12-06T18:11:55.151 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:11:55.152 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:11:55.480 INFO:teuthology.orchestra.run.smithi061.stdout:(5/29): ceph-radosgw-18.0.0-7644.g96fa2578.el9. 4.7 MB/s | 8.6 MB 00:01 2023-12-06T18:11:55.555 INFO:teuthology.orchestra.run.smithi061.stdout:(6/29): librgw2-18.0.0-7644.g96fa2578.el9.x86_6 8.1 MB/s | 4.8 MB 00:00 2023-12-06T18:11:55.572 INFO:teuthology.orchestra.run.smithi061.stdout:(7/29): python3-ceph-argparse-18.0.0-7644.g96fa 492 kB/s | 45 kB 00:00 2023-12-06T18:11:55.597 INFO:teuthology.orchestra.run.smithi061.stdout:(8/29): python3-ceph-common-18.0.0-7644.g96fa25 3.0 MB/s | 129 kB 00:00 2023-12-06T18:11:55.781 INFO:teuthology.orchestra.run.smithi061.stdout:(9/29): ceph-common-18.0.0-7644.g96fa2578.el9.x 9.0 MB/s | 19 MB 00:02 2023-12-06T18:11:55.806 INFO:teuthology.orchestra.run.smithi061.stdout:(10/29): python3-cephfs-18.0.0-7644.g96fa2578.e 690 kB/s | 161 kB 00:00 2023-12-06T18:11:55.831 INFO:teuthology.orchestra.run.smithi061.stdout:(11/29): python3-rados-18.0.0-7644.g96fa2578.el 1.3 MB/s | 321 kB 00:00 2023-12-06T18:11:55.865 INFO:teuthology.orchestra.run.smithi061.stdout:(12/29): python3-rbd-18.0.0-7644.g96fa2578.el9. 3.5 MB/s | 297 kB 00:00 2023-12-06T18:11:55.890 INFO:teuthology.orchestra.run.smithi061.stdout:(13/29): python3-rgw-18.0.0-7644.g96fa2578.el9. 1.2 MB/s | 99 kB 00:00 2023-12-06T18:11:56.032 INFO:teuthology.orchestra.run.smithi061.stdout:(14/29): boost-program-options-1.75.0-8.el9.x86 528 kB/s | 106 kB 00:00 2023-12-06T18:11:56.057 INFO:teuthology.orchestra.run.smithi061.stdout:(15/29): librabbitmq-0.11.0-7.el9.x86_64.rpm 381 kB/s | 44 kB 00:00 2023-12-06T18:11:56.082 INFO:teuthology.orchestra.run.smithi061.stdout:(16/29): libpmemobj-1.12.1-1.el9.x86_64.rpm 733 kB/s | 159 kB 00:00 2023-12-06T18:11:56.141 INFO:teuthology.orchestra.run.smithi061.stdout:(17/29): librdkafka-1.6.1-102.el9.x86_64.rpm 5.9 MB/s | 661 kB 00:00 2023-12-06T18:11:56.166 INFO:teuthology.orchestra.run.smithi061.stdout:(18/29): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 282 kB 00:00 2023-12-06T18:11:56.192 INFO:teuthology.orchestra.run.smithi061.stdout:(19/29): python3-prettytable-0.7.2-27.el9.noarc 379 kB/s | 41 kB 00:00 2023-12-06T18:11:56.434 INFO:teuthology.orchestra.run.smithi061.stdout:(20/29): libarrow-doc-9.0.0-9.el9.noarch.rpm 101 kB/s | 24 kB 00:00 2023-12-06T18:11:56.512 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:11:56.609 INFO:teuthology.orchestra.run.smithi061.stdout:(21/29): liboath-2.6.7-2.el9.x86_64.rpm 277 kB/s | 49 kB 00:00 2023-12-06T18:11:56.616 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/31 2023-12-06T18:11:56.643 INFO:teuthology.orchestra.run.smithi061.stdout:(22/29): gperftools-libs-2.9.1-2.el9.x86_64.rpm 617 kB/s | 309 kB 00:00 2023-12-06T18:11:56.646 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/31 2023-12-06T18:11:56.705 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/31 2023-12-06T18:11:56.718 INFO:teuthology.orchestra.run.smithi061.stdout:(23/29): libunwind-1.6.2-1.el9.x86_64.rpm 619 kB/s | 67 kB 00:00 2023-12-06T18:11:56.902 INFO:teuthology.orchestra.run.smithi061.stdout:(24/29): re2-20211101-3.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2023-12-06T18:11:56.985 INFO:teuthology.orchestra.run.smithi061.stdout:(25/29): libarrow-9.0.0-9.el9.x86_64.rpm 5.6 MB/s | 4.4 MB 00:00 2023-12-06T18:11:57.088 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/31 2023-12-06T18:11:57.121 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librados2-2:18.0.0-7644.g96fa2578.el9.x86_64 5/31 2023-12-06T18:11:57.177 INFO:teuthology.orchestra.run.smithi061.stdout:(26/29): thrift-0.15.0-2.el9.x86_64.rpm 5.7 MB/s | 1.6 MB 00:00 2023-12-06T18:11:57.213 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:18.0.0-7644.g96fa2578.el9.x86_64 5/31 2023-12-06T18:11:57.261 INFO:teuthology.orchestra.run.smithi061.stdout:(27/29): parquet-libs-9.0.0-9.el9.x86_64.rpm 1.3 MB/s | 837 kB 00:00 2023-12-06T18:11:57.306 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rados-2:18.0.0-7644.g96fa2578.el9.x86_64 6/31 2023-12-06T18:11:57.328 INFO:teuthology.orchestra.run.smithi061.stdout:(28/29): librados2-18.0.0-7644.g96fa2578.el9.x8 9.7 MB/s | 3.3 MB 00:00 2023-12-06T18:11:57.334 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs2-2:18.0.0-7644.g96fa2578.el9.x86_64 7/31 2023-12-06T18:11:57.395 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephfs2-2:18.0.0-7644.g96fa2578.el9.x86_64 7/31 2023-12-06T18:11:57.442 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/31 2023-12-06T18:11:57.485 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-argparse-2:18.0.0-7644.g96fa2578.el9. 9/31 2023-12-06T18:11:57.486 INFO:teuthology.orchestra.run.smithi061.stdout:(29/29): librbd1-18.0.0-7644.g96fa2578.el9.x86_ 9.8 MB/s | 3.0 MB 00:00 2023-12-06T18:11:57.491 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:11:57.491 INFO:teuthology.orchestra.run.smithi061.stdout:Total 14 MB/s | 55 MB 00:03 2023-12-06T18:11:57.492 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:11:57.532 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:11:57.533 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:11:57.551 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cephfs-2:18.0.0-7644.g96fa2578.el9.x86_64 10/31 2023-12-06T18:11:57.577 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libradosstriper1-2:18.0.0-7644.g96fa2578.el9.x86_6 11/31 2023-12-06T18:11:57.635 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7644.g96fa2578.el9.x86_6 11/31 2023-12-06T18:11:57.740 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/31 2023-12-06T18:11:57.768 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 13/31 2023-12-06T18:11:57.810 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/31 2023-12-06T18:11:57.850 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/31 2023-12-06T18:11:57.878 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 16/31 2023-12-06T18:11:57.919 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 17/31 2023-12-06T18:11:58.016 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:11:58.016 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:11:58.221 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 18/31 2023-12-06T18:11:58.266 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librbd1-2:18.0.0-7644.g96fa2578.el9.x86_64 19/31 2023-12-06T18:11:58.357 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:18.0.0-7644.g96fa2578.el9.x86_64 19/31 2023-12-06T18:11:58.392 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rbd-2:18.0.0-7644.g96fa2578.el9.x86_64 20/31 2023-12-06T18:11:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 21/31 2023-12-06T18:11:58.856 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 22/31 2023-12-06T18:11:58.885 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librgw2-2:18.0.0-7644.g96fa2578.el9.x86_64 23/31 2023-12-06T18:11:58.956 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librgw2-2:18.0.0-7644.g96fa2578.el9.x86_64 23/31 2023-12-06T18:11:59.023 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rgw-2:18.0.0-7644.g96fa2578.el9.x86_64 24/31 2023-12-06T18:11:59.053 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-common-2:18.0.0-7644.g96fa2578.el9.x8 25/31 2023-12-06T18:11:59.547 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:11:59.631 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/31 2023-12-06T18:11:59.666 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/31 2023-12-06T18:11:59.717 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/31 2023-12-06T18:12:00.119 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/31 2023-12-06T18:12:00.148 INFO:teuthology.orchestra.run.smithi061.stdout: Upgrading : librados2-2:18.0.0-7644.g96fa2578.el9.x86_64 5/31 2023-12-06T18:12:00.249 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librados2-2:18.0.0-7644.g96fa2578.el9.x86_64 5/31 2023-12-06T18:12:00.345 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-rados-2:18.0.0-7644.g96fa2578.el9.x86_64 6/31 2023-12-06T18:12:00.366 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libcephfs2-2:18.0.0-7644.g96fa2578.el9.x86_64 7/31 2023-12-06T18:12:00.429 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: libcephfs2-2:18.0.0-7644.g96fa2578.el9.x86_64 7/31 2023-12-06T18:12:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/31 2023-12-06T18:12:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-ceph-argparse-2:18.0.0-7644.g96fa2578.el9. 9/31 2023-12-06T18:12:00.571 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cephfs-2:18.0.0-7644.g96fa2578.el9.x86_64 10/31 2023-12-06T18:12:00.595 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libradosstriper1-2:18.0.0-7644.g96fa2578.el9.x86_6 11/31 2023-12-06T18:12:00.653 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7644.g96fa2578.el9.x86_6 11/31 2023-12-06T18:12:00.761 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/31 2023-12-06T18:12:00.794 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 13/31 2023-12-06T18:12:00.836 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/31 2023-12-06T18:12:00.868 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/31 2023-12-06T18:12:00.897 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 16/31 2023-12-06T18:12:00.929 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 17/31 2023-12-06T18:12:01.176 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:18.0.0-7644.g96fa2578.el9.x86_64 26/31 2023-12-06T18:12:01.216 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-common-2:18.0.0-7644.g96fa2578.el9.x86_64 26/31 2023-12-06T18:12:01.247 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 18/31 2023-12-06T18:12:01.276 INFO:teuthology.orchestra.run.smithi061.stdout: Upgrading : librbd1-2:18.0.0-7644.g96fa2578.el9.x86_64 19/31 2023-12-06T18:12:01.386 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librbd1-2:18.0.0-7644.g96fa2578.el9.x86_64 19/31 2023-12-06T18:12:01.414 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-rbd-2:18.0.0-7644.g96fa2578.el9.x86_64 20/31 2023-12-06T18:12:01.450 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 21/31 2023-12-06T18:12:01.777 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:18.0.0-7644.g96fa2578.el9.x86_64 26/31 2023-12-06T18:12:01.819 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-base-2:18.0.0-7644.g96fa2578.el9.x86_64 27/31 2023-12-06T18:12:01.899 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 22/31 2023-12-06T18:12:01.902 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-base-2:18.0.0-7644.g96fa2578.el9.x86_64 27/31 2023-12-06T18:12:01.902 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-12-06T18:12:01.902 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-12-06T18:12:01.902 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:01.928 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : librgw2-2:18.0.0-7644.g96fa2578.el9.x86_64 23/31 2023-12-06T18:12:01.937 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-selinux-2:18.0.0-7644.g96fa2578.el9.x86_64 28/31 2023-12-06T18:12:02.000 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librgw2-2:18.0.0-7644.g96fa2578.el9.x86_64 23/31 2023-12-06T18:12:02.078 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-rgw-2:18.0.0-7644.g96fa2578.el9.x86_64 24/31 2023-12-06T18:12:02.105 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-ceph-common-2:18.0.0-7644.g96fa2578.el9.x8 25/31 2023-12-06T18:12:04.237 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-common-2:18.0.0-7644.g96fa2578.el9.x86_64 26/31 2023-12-06T18:12:04.269 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-common-2:18.0.0-7644.g96fa2578.el9.x86_64 26/31 2023-12-06T18:12:04.855 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-common-2:18.0.0-7644.g96fa2578.el9.x86_64 26/31 2023-12-06T18:12:04.889 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-base-2:18.0.0-7644.g96fa2578.el9.x86_64 27/31 2023-12-06T18:12:04.978 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-base-2:18.0.0-7644.g96fa2578.el9.x86_64 27/31 2023-12-06T18:12:04.978 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-12-06T18:12:04.978 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-12-06T18:12:04.978 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:05.015 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-selinux-2:18.0.0-7644.g96fa2578.el9.x86_64 28/31 2023-12-06T18:12:11.441 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7644.g96fa2578.el9.x86_64 28/31 2023-12-06T18:12:11.442 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /sys 2023-12-06T18:12:11.442 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /proc 2023-12-06T18:12:11.442 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /mnt 2023-12-06T18:12:11.442 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /var/tmp 2023-12-06T18:12:11.442 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /home 2023-12-06T18:12:11.442 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /root 2023-12-06T18:12:11.442 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /tmp 2023-12-06T18:12:11.442 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:12.234 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-radosgw-2:18.0.0-7644.g96fa2578.el9.x86_64 29/31 2023-12-06T18:12:12.258 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7644.g96fa2578.el9.x86_64 29/31 2023-12-06T18:12:12.258 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-06T18:12:12.258 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-12-06T18:12:12.258 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-06T18:12:12.258 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-06T18:12:12.258 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:12.260 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 30/31 2023-12-06T18:12:12.327 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 30/31 2023-12-06T18:12:12.327 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 31/31 2023-12-06T18:12:13.423 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 31/31 2023-12-06T18:12:13.424 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-base-2:18.0.0-7644.g96fa2578.el9.x86_64 1/31 2023-12-06T18:12:13.424 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-common-2:18.0.0-7644.g96fa2578.el9.x86_64 2/31 2023-12-06T18:12:13.424 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-radosgw-2:18.0.0-7644.g96fa2578.el9.x86_64 3/31 2023-12-06T18:12:13.424 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-selinux-2:18.0.0-7644.g96fa2578.el9.x86_64 4/31 2023-12-06T18:12:13.424 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs2-2:18.0.0-7644.g96fa2578.el9.x86_64 5/31 2023-12-06T18:12:13.424 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libradosstriper1-2:18.0.0-7644.g96fa2578.el9.x86_6 6/31 2023-12-06T18:12:13.424 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librgw2-2:18.0.0-7644.g96fa2578.el9.x86_64 7/31 2023-12-06T18:12:13.424 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7644.g96fa2578.el9. 8/31 2023-12-06T18:12:13.424 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-common-2:18.0.0-7644.g96fa2578.el9.x8 9/31 2023-12-06T18:12:13.424 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cephfs-2:18.0.0-7644.g96fa2578.el9.x86_64 10/31 2023-12-06T18:12:13.424 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rados-2:18.0.0-7644.g96fa2578.el9.x86_64 11/31 2023-12-06T18:12:13.424 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rbd-2:18.0.0-7644.g96fa2578.el9.x86_64 12/31 2023-12-06T18:12:13.425 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rgw-2:18.0.0-7644.g96fa2578.el9.x86_64 13/31 2023-12-06T18:12:13.425 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/31 2023-12-06T18:12:13.425 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/31 2023-12-06T18:12:13.425 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/31 2023-12-06T18:12:13.425 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/31 2023-12-06T18:12:13.425 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/31 2023-12-06T18:12:13.425 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/31 2023-12-06T18:12:13.425 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 20/31 2023-12-06T18:12:13.425 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 21/31 2023-12-06T18:12:13.425 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 22/31 2023-12-06T18:12:13.426 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 23/31 2023-12-06T18:12:13.426 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 24/31 2023-12-06T18:12:13.426 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 25/31 2023-12-06T18:12:13.426 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : re2-1:20211101-3.el9.x86_64 26/31 2023-12-06T18:12:13.426 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 27/31 2023-12-06T18:12:13.426 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:18.0.0-7644.g96fa2578.el9.x86_64 28/31 2023-12-06T18:12:13.426 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 29/31 2023-12-06T18:12:13.427 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:18.0.0-7644.g96fa2578.el9.x86_64 30/31 2023-12-06T18:12:14.448 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 31/31 2023-12-06T18:12:14.448 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:14.448 INFO:teuthology.orchestra.run.smithi152.stdout:Upgraded: 2023-12-06T18:12:14.448 INFO:teuthology.orchestra.run.smithi152.stdout: librados2-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:14.448 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:14.448 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:12:14.448 INFO:teuthology.orchestra.run.smithi152.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-12-06T18:12:14.448 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:14.448 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:14.448 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:14.448 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:14.448 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-12-06T18:12:14.449 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-9.0.0-9.el9.x86_64 2023-12-06T18:12:14.449 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-12-06T18:12:14.449 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:14.449 INFO:teuthology.orchestra.run.smithi152.stdout: liboath-2.6.7-2.el9.x86_64 2023-12-06T18:12:14.449 INFO:teuthology.orchestra.run.smithi152.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-12-06T18:12:14.449 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-12-06T18:12:14.449 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:14.449 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-12-06T18:12:14.449 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:14.449 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind-1.6.2-1.el9.x86_64 2023-12-06T18:12:14.449 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-12-06T18:12:14.449 INFO:teuthology.orchestra.run.smithi152.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-12-06T18:12:14.449 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:14.449 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:14.449 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:14.449 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-12-06T18:12:14.450 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:14.450 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:14.450 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:14.450 INFO:teuthology.orchestra.run.smithi152.stdout: re2-1:20211101-3.el9.x86_64 2023-12-06T18:12:14.450 INFO:teuthology.orchestra.run.smithi152.stdout: thrift-0.15.0-2.el9.x86_64 2023-12-06T18:12:14.450 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:14.450 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:12:14.785 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-test 2023-12-06T18:12:15.168 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7644.g96fa2578.el9.x86_64 28/31 2023-12-06T18:12:15.168 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /sys 2023-12-06T18:12:15.169 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /proc 2023-12-06T18:12:15.169 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /mnt 2023-12-06T18:12:15.169 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /var/tmp 2023-12-06T18:12:15.169 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /home 2023-12-06T18:12:15.169 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /root 2023-12-06T18:12:15.169 INFO:teuthology.orchestra.run.smithi061.stdout:skipping the directory /tmp 2023-12-06T18:12:15.169 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:15.373 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:27 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:12:15.517 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:12:15.518 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:12:15.518 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2023-12-06T18:12:15.518 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:12:15.518 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:12:15.518 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 41 M 2023-12-06T18:12:15.519 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2023-12-06T18:12:15.519 INFO:teuthology.orchestra.run.smithi152.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-12-06T18:12:15.519 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 164 k 2023-12-06T18:12:15.519 INFO:teuthology.orchestra.run.smithi152.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-12-06T18:12:15.519 INFO:teuthology.orchestra.run.smithi152.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-12-06T18:12:15.519 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-12-06T18:12:15.519 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:15.519 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:12:15.519 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:12:15.519 INFO:teuthology.orchestra.run.smithi152.stdout:Install 6 Packages 2023-12-06T18:12:15.519 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:15.520 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 42 M 2023-12-06T18:12:15.520 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 173 M 2023-12-06T18:12:15.520 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:12:15.730 INFO:teuthology.orchestra.run.smithi152.stdout:(1/6): jq-1.6-16.el9.x86_64.rpm 891 kB/s | 186 kB 00:00 2023-12-06T18:12:15.797 INFO:teuthology.orchestra.run.smithi152.stdout:(2/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2023-12-06T18:12:15.864 INFO:teuthology.orchestra.run.smithi152.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 4.4 MB/s | 300 kB 00:00 2023-12-06T18:12:15.923 INFO:teuthology.orchestra.run.smithi152.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.0 MB/s | 63 kB 00:00 2023-12-06T18:12:16.006 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-radosgw-2:18.0.0-7644.g96fa2578.el9.x86_64 29/31 2023-12-06T18:12:16.033 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7644.g96fa2578.el9.x86_64 29/31 2023-12-06T18:12:16.033 INFO:teuthology.orchestra.run.smithi061.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-06T18:12:16.033 INFO:teuthology.orchestra.run.smithi061.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-12-06T18:12:16.033 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-06T18:12:16.033 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-06T18:12:16.033 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:16.035 INFO:teuthology.orchestra.run.smithi061.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 30/31 2023-12-06T18:12:16.073 INFO:teuthology.orchestra.run.smithi152.stdout:(5/6): libcephsqlite-18.0.0-7644.g96fa2578.el9. 296 kB/s | 164 kB 00:00 2023-12-06T18:12:16.117 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 30/31 2023-12-06T18:12:16.117 INFO:teuthology.orchestra.run.smithi061.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 31/31 2023-12-06T18:12:17.261 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 31/31 2023-12-06T18:12:17.261 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-base-2:18.0.0-7644.g96fa2578.el9.x86_64 1/31 2023-12-06T18:12:17.261 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-common-2:18.0.0-7644.g96fa2578.el9.x86_64 2/31 2023-12-06T18:12:17.262 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-radosgw-2:18.0.0-7644.g96fa2578.el9.x86_64 3/31 2023-12-06T18:12:17.262 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-selinux-2:18.0.0-7644.g96fa2578.el9.x86_64 4/31 2023-12-06T18:12:17.262 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libcephfs2-2:18.0.0-7644.g96fa2578.el9.x86_64 5/31 2023-12-06T18:12:17.262 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libradosstriper1-2:18.0.0-7644.g96fa2578.el9.x86_6 6/31 2023-12-06T18:12:17.262 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librgw2-2:18.0.0-7644.g96fa2578.el9.x86_64 7/31 2023-12-06T18:12:17.262 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7644.g96fa2578.el9. 8/31 2023-12-06T18:12:17.262 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-ceph-common-2:18.0.0-7644.g96fa2578.el9.x8 9/31 2023-12-06T18:12:17.262 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cephfs-2:18.0.0-7644.g96fa2578.el9.x86_64 10/31 2023-12-06T18:12:17.262 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-rados-2:18.0.0-7644.g96fa2578.el9.x86_64 11/31 2023-12-06T18:12:17.262 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-rbd-2:18.0.0-7644.g96fa2578.el9.x86_64 12/31 2023-12-06T18:12:17.262 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-rgw-2:18.0.0-7644.g96fa2578.el9.x86_64 13/31 2023-12-06T18:12:17.262 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/31 2023-12-06T18:12:17.262 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/31 2023-12-06T18:12:17.263 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/31 2023-12-06T18:12:17.263 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/31 2023-12-06T18:12:17.263 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/31 2023-12-06T18:12:17.263 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/31 2023-12-06T18:12:17.263 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 20/31 2023-12-06T18:12:17.263 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 21/31 2023-12-06T18:12:17.263 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 22/31 2023-12-06T18:12:17.263 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 23/31 2023-12-06T18:12:17.263 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 24/31 2023-12-06T18:12:17.263 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 25/31 2023-12-06T18:12:17.263 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : re2-1:20211101-3.el9.x86_64 26/31 2023-12-06T18:12:17.263 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 27/31 2023-12-06T18:12:17.264 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librados2-2:18.0.0-7644.g96fa2578.el9.x86_64 28/31 2023-12-06T18:12:17.264 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 29/31 2023-12-06T18:12:17.264 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librbd1-2:18.0.0-7644.g96fa2578.el9.x86_64 30/31 2023-12-06T18:12:18.267 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 31/31 2023-12-06T18:12:18.267 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:18.267 INFO:teuthology.orchestra.run.smithi061.stdout:Upgraded: 2023-12-06T18:12:18.267 INFO:teuthology.orchestra.run.smithi061.stdout: librados2-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:18.267 INFO:teuthology.orchestra.run.smithi061.stdout: librbd1-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:18.267 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:12:18.268 INFO:teuthology.orchestra.run.smithi061.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-12-06T18:12:18.268 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-base-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:18.268 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-common-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:18.268 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-radosgw-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:18.268 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-selinux-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:18.268 INFO:teuthology.orchestra.run.smithi061.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-12-06T18:12:18.268 INFO:teuthology.orchestra.run.smithi061.stdout: libarrow-9.0.0-9.el9.x86_64 2023-12-06T18:12:18.268 INFO:teuthology.orchestra.run.smithi061.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-12-06T18:12:18.268 INFO:teuthology.orchestra.run.smithi061.stdout: libcephfs2-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:18.268 INFO:teuthology.orchestra.run.smithi061.stdout: liboath-2.6.7-2.el9.x86_64 2023-12-06T18:12:18.268 INFO:teuthology.orchestra.run.smithi061.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-12-06T18:12:18.268 INFO:teuthology.orchestra.run.smithi061.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-12-06T18:12:18.268 INFO:teuthology.orchestra.run.smithi061.stdout: libradosstriper1-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:18.269 INFO:teuthology.orchestra.run.smithi061.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-12-06T18:12:18.269 INFO:teuthology.orchestra.run.smithi061.stdout: librgw2-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:18.269 INFO:teuthology.orchestra.run.smithi061.stdout: libunwind-1.6.2-1.el9.x86_64 2023-12-06T18:12:18.269 INFO:teuthology.orchestra.run.smithi061.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-12-06T18:12:18.269 INFO:teuthology.orchestra.run.smithi061.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-12-06T18:12:18.269 INFO:teuthology.orchestra.run.smithi061.stdout: python3-ceph-argparse-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:18.269 INFO:teuthology.orchestra.run.smithi061.stdout: python3-ceph-common-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:18.269 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cephfs-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:18.269 INFO:teuthology.orchestra.run.smithi061.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-12-06T18:12:18.269 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rados-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:18.269 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rbd-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:18.269 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rgw-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:18.269 INFO:teuthology.orchestra.run.smithi061.stdout: re2-1:20211101-3.el9.x86_64 2023-12-06T18:12:18.269 INFO:teuthology.orchestra.run.smithi061.stdout: thrift-0.15.0-2.el9.x86_64 2023-12-06T18:12:18.270 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:18.270 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:12:18.650 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-test 2023-12-06T18:12:19.148 INFO:teuthology.orchestra.run.smithi152.stdout:(6/6): ceph-test-18.0.0-7644.g96fa2578.el9.x86_ 11 MB/s | 41 MB 00:03 2023-12-06T18:12:19.150 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:12:19.151 INFO:teuthology.orchestra.run.smithi152.stdout:Total 12 MB/s | 42 MB 00:03 2023-12-06T18:12:19.151 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:12:19.171 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:12:19.171 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:12:19.256 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:00:29 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:12:19.407 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:12:19.408 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:12:19.408 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repository Size 2023-12-06T18:12:19.408 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:12:19.408 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:12:19.408 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-test x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 41 M 2023-12-06T18:12:19.408 INFO:teuthology.orchestra.run.smithi061.stdout:Installing dependencies: 2023-12-06T18:12:19.408 INFO:teuthology.orchestra.run.smithi061.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-12-06T18:12:19.408 INFO:teuthology.orchestra.run.smithi061.stdout: libcephsqlite x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 164 k 2023-12-06T18:12:19.408 INFO:teuthology.orchestra.run.smithi061.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-12-06T18:12:19.408 INFO:teuthology.orchestra.run.smithi061.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-12-06T18:12:19.408 INFO:teuthology.orchestra.run.smithi061.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-12-06T18:12:19.409 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:19.409 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:12:19.409 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:12:19.409 INFO:teuthology.orchestra.run.smithi061.stdout:Install 6 Packages 2023-12-06T18:12:19.409 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:19.409 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 42 M 2023-12-06T18:12:19.409 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 173 M 2023-12-06T18:12:19.409 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:12:19.526 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:12:19.526 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:12:19.664 INFO:teuthology.orchestra.run.smithi061.stdout:(1/6): libcephsqlite-18.0.0-7644.g96fa2578.el9. 647 kB/s | 164 kB 00:00 2023-12-06T18:12:19.714 INFO:teuthology.orchestra.run.smithi061.stdout:(2/6): jq-1.6-16.el9.x86_64.rpm 614 kB/s | 186 kB 00:00 2023-12-06T18:12:19.764 INFO:teuthology.orchestra.run.smithi061.stdout:(3/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.1 MB/s | 217 kB 00:00 2023-12-06T18:12:19.798 INFO:teuthology.orchestra.run.smithi061.stdout:(4/6): socat-1.7.4.1-5.el9.x86_64.rpm 3.5 MB/s | 300 kB 00:00 2023-12-06T18:12:19.823 INFO:teuthology.orchestra.run.smithi061.stdout:(5/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2023-12-06T18:12:20.067 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:12:20.150 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2023-12-06T18:12:20.197 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2023-12-06T18:12:20.230 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2023-12-06T18:12:20.273 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2023-12-06T18:12:20.301 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephsqlite-2:18.0.0-7644.g96fa2578.el9.x86_64 5/6 2023-12-06T18:12:21.432 INFO:teuthology.orchestra.run.smithi061.stdout:(6/6): ceph-test-18.0.0-7644.g96fa2578.el9.x86_ 20 MB/s | 41 MB 00:02 2023-12-06T18:12:21.434 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:12:21.435 INFO:teuthology.orchestra.run.smithi061.stdout:Total 21 MB/s | 42 MB 00:02 2023-12-06T18:12:21.435 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:12:21.457 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:12:21.457 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:12:21.838 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:12:21.838 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:12:22.404 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:12:22.502 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2023-12-06T18:12:22.555 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2023-12-06T18:12:22.589 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2023-12-06T18:12:22.633 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2023-12-06T18:12:22.660 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libcephsqlite-2:18.0.0-7644.g96fa2578.el9.x86_64 5/6 2023-12-06T18:12:24.161 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7644.g96fa2578.el9.x86_64 5/6 2023-12-06T18:12:24.232 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-test-2:18.0.0-7644.g96fa2578.el9.x86_64 6/6 2023-12-06T18:12:26.307 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-test-2:18.0.0-7644.g96fa2578.el9.x86_64 6/6 2023-12-06T18:12:26.308 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-test-2:18.0.0-7644.g96fa2578.el9.x86_64 1/6 2023-12-06T18:12:26.308 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephsqlite-2:18.0.0-7644.g96fa2578.el9.x86_64 2/6 2023-12-06T18:12:26.308 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : jq-1.6-16.el9.x86_64 3/6 2023-12-06T18:12:26.308 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 4/6 2023-12-06T18:12:26.308 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/6 2023-12-06T18:12:26.700 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/6 2023-12-06T18:12:26.700 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:26.700 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:12:26.700 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:26.701 INFO:teuthology.orchestra.run.smithi152.stdout: jq-1.6-16.el9.x86_64 2023-12-06T18:12:26.701 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:26.701 INFO:teuthology.orchestra.run.smithi152.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-12-06T18:12:26.701 INFO:teuthology.orchestra.run.smithi152.stdout: socat-1.7.4.1-5.el9.x86_64 2023-12-06T18:12:26.701 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-12-06T18:12:26.701 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:26.701 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:12:26.737 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7644.g96fa2578.el9.x86_64 5/6 2023-12-06T18:12:26.828 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-test-2:18.0.0-7644.g96fa2578.el9.x86_64 6/6 2023-12-06T18:12:26.975 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph 2023-12-06T18:12:27.562 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:39 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:12:27.726 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:12:27.726 INFO:teuthology.orchestra.run.smithi152.stdout:========================================================================================= 2023-12-06T18:12:27.726 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2023-12-06T18:12:27.726 INFO:teuthology.orchestra.run.smithi152.stdout:========================================================================================= 2023-12-06T18:12:27.726 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:12:27.726 INFO:teuthology.orchestra.run.smithi152.stdout: ceph x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 6.6 k 2023-12-06T18:12:27.726 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2023-12-06T18:12:27.726 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 2.1 M 2023-12-06T18:12:27.726 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 1.5 M 2023-12-06T18:12:27.726 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7644.g96fa2578.el9 ceph-noarch 243 k 2023-12-06T18:12:27.726 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 4.4 M 2023-12-06T18:12:27.727 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 18 M 2023-12-06T18:12:27.727 INFO:teuthology.orchestra.run.smithi152.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2023-12-06T18:12:27.727 INFO:teuthology.orchestra.run.smithi152.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2023-12-06T18:12:27.727 INFO:teuthology.orchestra.run.smithi152.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2023-12-06T18:12:27.727 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-12-06T18:12:27.727 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-12-06T18:12:27.727 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-12-06T18:12:27.727 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-12-06T18:12:27.727 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-12-06T18:12:27.727 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-12-06T18:12:27.727 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-12-06T18:12:27.727 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-12-06T18:12:27.727 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-12-06T18:12:27.727 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-12-06T18:12:27.728 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-12-06T18:12:27.728 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-12-06T18:12:27.728 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-12-06T18:12:27.728 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-12-06T18:12:27.728 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-12-06T18:12:27.728 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-12-06T18:12:27.728 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-12-06T18:12:27.728 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-12-06T18:12:27.728 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-12-06T18:12:27.728 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-12-06T18:12:27.728 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-12-06T18:12:27.728 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-12-06T18:12:27.728 INFO:teuthology.orchestra.run.smithi152.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-12-06T18:12:27.728 INFO:teuthology.orchestra.run.smithi152.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-12-06T18:12:27.728 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-12-06T18:12:27.729 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-12-06T18:12:27.729 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-12-06T18:12:27.729 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:27.729 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:12:27.729 INFO:teuthology.orchestra.run.smithi152.stdout:========================================================================================= 2023-12-06T18:12:27.729 INFO:teuthology.orchestra.run.smithi152.stdout:Install 36 Packages 2023-12-06T18:12:27.729 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:27.729 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 30 M 2023-12-06T18:12:27.730 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 111 M 2023-12-06T18:12:27.730 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:12:28.265 INFO:teuthology.orchestra.run.smithi152.stdout:(1/36): ceph-18.0.0-7644.g96fa2578.el9.x86_64.r 37 kB/s | 6.6 kB 00:00 2023-12-06T18:12:28.757 INFO:teuthology.orchestra.run.smithi152.stdout:(2/36): ceph-mon-18.0.0-7644.g96fa2578.el9.x86_ 8.9 MB/s | 4.4 MB 00:00 2023-12-06T18:12:28.807 INFO:teuthology.orchestra.run.smithi152.stdout:(3/36): ceph-mds-18.0.0-7644.g96fa2578.el9.x86_ 2.9 MB/s | 2.1 MB 00:00 2023-12-06T18:12:28.849 INFO:teuthology.orchestra.run.smithi152.stdout:(4/36): ceph-mgr-18.0.0-7644.g96fa2578.el9.x86_ 1.9 MB/s | 1.5 MB 00:00 2023-12-06T18:12:28.900 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-test-2:18.0.0-7644.g96fa2578.el9.x86_64 6/6 2023-12-06T18:12:28.900 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-test-2:18.0.0-7644.g96fa2578.el9.x86_64 1/6 2023-12-06T18:12:28.900 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libcephsqlite-2:18.0.0-7644.g96fa2578.el9.x86_64 2/6 2023-12-06T18:12:28.901 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : jq-1.6-16.el9.x86_64 3/6 2023-12-06T18:12:28.901 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 4/6 2023-12-06T18:12:28.901 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 5/6 2023-12-06T18:12:28.916 INFO:teuthology.orchestra.run.smithi152.stdout:(5/36): ceph-mgr-modules-core-18.0.0-7644.g96fa 2.2 MB/s | 243 kB 00:00 2023-12-06T18:12:29.016 INFO:teuthology.orchestra.run.smithi152.stdout:(6/36): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2023-12-06T18:12:29.041 INFO:teuthology.orchestra.run.smithi152.stdout:(7/36): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 144 kB 00:00 2023-12-06T18:12:29.058 INFO:teuthology.orchestra.run.smithi152.stdout:(8/36): python3-markupsafe-1.1.1-12.el9.x86_64. 760 kB/s | 32 kB 00:00 2023-12-06T18:12:29.081 INFO:teuthology.orchestra.run.smithi152.stdout:(9/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 47 kB 00:00 2023-12-06T18:12:29.098 INFO:teuthology.orchestra.run.smithi152.stdout:(10/36): python3-toml-0.10.2-6.el9.noarch.rpm 991 kB/s | 39 kB 00:00 2023-12-06T18:12:29.131 INFO:teuthology.orchestra.run.smithi152.stdout:(11/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2023-12-06T18:12:29.190 INFO:teuthology.orchestra.run.smithi152.stdout:(12/36): python3-cryptography-36.0.1-4.el9.x86_ 12 MB/s | 1.1 MB 00:00 2023-12-06T18:12:29.215 INFO:teuthology.orchestra.run.smithi152.stdout:(13/36): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2023-12-06T18:12:29.240 INFO:teuthology.orchestra.run.smithi152.stdout:(14/36): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2023-12-06T18:12:29.266 INFO:teuthology.orchestra.run.smithi152.stdout:(15/36): python3-requests-2.25.1-7.el9.noarch.r 2.2 MB/s | 113 kB 00:00 2023-12-06T18:12:29.291 INFO:teuthology.orchestra.run.smithi152.stdout:(16/36): python3-urllib3-1.26.5-4.el9.noarch.rp 3.6 MB/s | 187 kB 00:00 2023-12-06T18:12:29.296 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 6/6 2023-12-06T18:12:29.296 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:29.296 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:12:29.296 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-test-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:29.296 INFO:teuthology.orchestra.run.smithi061.stdout: jq-1.6-16.el9.x86_64 2023-12-06T18:12:29.296 INFO:teuthology.orchestra.run.smithi061.stdout: libcephsqlite-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:29.296 INFO:teuthology.orchestra.run.smithi061.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-12-06T18:12:29.296 INFO:teuthology.orchestra.run.smithi061.stdout: socat-1.7.4.1-5.el9.x86_64 2023-12-06T18:12:29.297 INFO:teuthology.orchestra.run.smithi061.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-12-06T18:12:29.297 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:29.297 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:12:29.450 INFO:teuthology.orchestra.run.smithi152.stdout:(17/36): luarocks-3.9.2-1.el9.noarch.rpm 954 kB/s | 151 kB 00:00 2023-12-06T18:12:29.467 INFO:teuthology.orchestra.run.smithi152.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 111 kB/s | 22 kB 00:00 2023-12-06T18:12:29.492 INFO:teuthology.orchestra.run.smithi152.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.0 MB/s | 43 kB 00:00 2023-12-06T18:12:29.558 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph 2023-12-06T18:12:29.684 INFO:teuthology.orchestra.run.smithi152.stdout:(20/36): ceph-osd-18.0.0-7644.g96fa2578.el9.x86 19 MB/s | 18 MB 00:00 2023-12-06T18:12:29.709 INFO:teuthology.orchestra.run.smithi152.stdout:(21/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.6 MB/s | 358 kB 00:00 2023-12-06T18:12:29.742 INFO:teuthology.orchestra.run.smithi152.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 534 kB/s | 18 kB 00:00 2023-12-06T18:12:29.759 INFO:teuthology.orchestra.run.smithi152.stdout:(23/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 142 kB/s | 11 kB 00:00 2023-12-06T18:12:29.785 INFO:teuthology.orchestra.run.smithi152.stdout:(24/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 540 kB/s | 172 kB 00:00 2023-12-06T18:12:29.802 INFO:teuthology.orchestra.run.smithi152.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 393 kB/s | 23 kB 00:00 2023-12-06T18:12:29.818 INFO:teuthology.orchestra.run.smithi152.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 331 kB/s | 19 kB 00:00 2023-12-06T18:12:29.835 INFO:teuthology.orchestra.run.smithi152.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2023-12-06T18:12:29.852 INFO:teuthology.orchestra.run.smithi152.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 1.4 MB/s | 46 kB 00:00 2023-12-06T18:12:29.886 INFO:teuthology.orchestra.run.smithi152.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 1.1 MB/s | 79 kB 00:00 2023-12-06T18:12:29.911 INFO:teuthology.orchestra.run.smithi152.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 280 kB/s | 16 kB 00:00 2023-12-06T18:12:29.945 INFO:teuthology.orchestra.run.smithi152.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2023-12-06T18:12:29.970 INFO:teuthology.orchestra.run.smithi152.stdout:(32/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2023-12-06T18:12:29.995 INFO:teuthology.orchestra.run.smithi152.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 427 kB/s | 36 kB 00:00 2023-12-06T18:12:30.037 INFO:teuthology.orchestra.run.smithi152.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2023-12-06T18:12:30.062 INFO:teuthology.orchestra.run.smithi152.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 299 kB/s | 20 kB 00:00 2023-12-06T18:12:30.096 INFO:teuthology.orchestra.run.smithi152.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2023-12-06T18:12:30.099 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:12:30.099 INFO:teuthology.orchestra.run.smithi152.stdout:Total 13 MB/s | 30 MB 00:02 2023-12-06T18:12:30.111 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:12:30.153 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:12:30.153 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:12:30.163 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:00:40 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:12:30.318 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:12:30.320 INFO:teuthology.orchestra.run.smithi061.stdout:========================================================================================= 2023-12-06T18:12:30.320 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repository Size 2023-12-06T18:12:30.320 INFO:teuthology.orchestra.run.smithi061.stdout:========================================================================================= 2023-12-06T18:12:30.320 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:12:30.320 INFO:teuthology.orchestra.run.smithi061.stdout: ceph x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 6.6 k 2023-12-06T18:12:30.320 INFO:teuthology.orchestra.run.smithi061.stdout:Installing dependencies: 2023-12-06T18:12:30.321 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mds x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 2.1 M 2023-12-06T18:12:30.321 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 1.5 M 2023-12-06T18:12:30.321 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7644.g96fa2578.el9 ceph-noarch 243 k 2023-12-06T18:12:30.321 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mon x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 4.4 M 2023-12-06T18:12:30.321 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-osd x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 18 M 2023-12-06T18:12:30.321 INFO:teuthology.orchestra.run.smithi061.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2023-12-06T18:12:30.321 INFO:teuthology.orchestra.run.smithi061.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2023-12-06T18:12:30.321 INFO:teuthology.orchestra.run.smithi061.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2023-12-06T18:12:30.321 INFO:teuthology.orchestra.run.smithi061.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-12-06T18:12:30.321 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-12-06T18:12:30.321 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-12-06T18:12:30.321 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-12-06T18:12:30.321 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-12-06T18:12:30.321 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-12-06T18:12:30.321 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-12-06T18:12:30.322 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-12-06T18:12:30.322 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-12-06T18:12:30.322 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-12-06T18:12:30.322 INFO:teuthology.orchestra.run.smithi061.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-12-06T18:12:30.322 INFO:teuthology.orchestra.run.smithi061.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-12-06T18:12:30.322 INFO:teuthology.orchestra.run.smithi061.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-12-06T18:12:30.322 INFO:teuthology.orchestra.run.smithi061.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-12-06T18:12:30.322 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-12-06T18:12:30.322 INFO:teuthology.orchestra.run.smithi061.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-12-06T18:12:30.322 INFO:teuthology.orchestra.run.smithi061.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-12-06T18:12:30.322 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-12-06T18:12:30.322 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-12-06T18:12:30.322 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-12-06T18:12:30.322 INFO:teuthology.orchestra.run.smithi061.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-12-06T18:12:30.323 INFO:teuthology.orchestra.run.smithi061.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-12-06T18:12:30.323 INFO:teuthology.orchestra.run.smithi061.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-12-06T18:12:30.323 INFO:teuthology.orchestra.run.smithi061.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-12-06T18:12:30.323 INFO:teuthology.orchestra.run.smithi061.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-12-06T18:12:30.323 INFO:teuthology.orchestra.run.smithi061.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-12-06T18:12:30.323 INFO:teuthology.orchestra.run.smithi061.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-12-06T18:12:30.323 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:30.323 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:12:30.323 INFO:teuthology.orchestra.run.smithi061.stdout:========================================================================================= 2023-12-06T18:12:30.323 INFO:teuthology.orchestra.run.smithi061.stdout:Install 36 Packages 2023-12-06T18:12:30.323 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:30.324 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 30 M 2023-12-06T18:12:30.324 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 111 M 2023-12-06T18:12:30.324 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:12:30.562 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:12:30.562 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:12:30.975 INFO:teuthology.orchestra.run.smithi061.stdout:(1/36): ceph-18.0.0-7644.g96fa2578.el9.x86_64.r 38 kB/s | 6.6 kB 00:00 2023-12-06T18:12:31.308 INFO:teuthology.orchestra.run.smithi061.stdout:(2/36): ceph-mon-18.0.0-7644.g96fa2578.el9.x86_ 13 MB/s | 4.4 MB 00:00 2023-12-06T18:12:31.861 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:12:31.924 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2023-12-06T18:12:31.957 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2023-12-06T18:12:31.984 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2023-12-06T18:12:32.053 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2023-12-06T18:12:32.091 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2023-12-06T18:12:32.124 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2023-12-06T18:12:32.142 INFO:teuthology.orchestra.run.smithi061.stdout:(3/36): ceph-osd-18.0.0-7644.g96fa2578.el9.x86_ 21 MB/s | 18 MB 00:00 2023-12-06T18:12:32.200 INFO:teuthology.orchestra.run.smithi061.stdout:(4/36): ceph-mgr-modules-core-18.0.0-7644.g96fa 4.1 MB/s | 243 kB 00:00 2023-12-06T18:12:32.231 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2023-12-06T18:12:32.301 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2023-12-06T18:12:32.343 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2023-12-06T18:12:32.373 INFO:teuthology.orchestra.run.smithi061.stdout:(5/36): ceph-mgr-18.0.0-7644.g96fa2578.el9.x86_ 953 kB/s | 1.5 MB 00:01 2023-12-06T18:12:32.398 INFO:teuthology.orchestra.run.smithi061.stdout:(6/36): lua-5.4.4-4.el9.x86_64.rpm 943 kB/s | 187 kB 00:00 2023-12-06T18:12:32.414 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2023-12-06T18:12:32.432 INFO:teuthology.orchestra.run.smithi061.stdout:(7/36): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2023-12-06T18:12:32.457 INFO:teuthology.orchestra.run.smithi061.stdout:(8/36): python3-markupsafe-1.1.1-12.el9.x86_64. 538 kB/s | 32 kB 00:00 2023-12-06T18:12:32.469 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 11/36 2023-12-06T18:12:32.482 INFO:teuthology.orchestra.run.smithi061.stdout:(9/36): python3-pytz-2021.1-5.el9.noarch.rpm 944 kB/s | 47 kB 00:00 2023-12-06T18:12:32.506 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 12/36 2023-12-06T18:12:32.508 INFO:teuthology.orchestra.run.smithi061.stdout:(10/36): python3-toml-0.10.2-6.el9.noarch.rpm 773 kB/s | 39 kB 00:00 2023-12-06T18:12:32.550 INFO:teuthology.orchestra.run.smithi061.stdout:(11/36): ceph-mds-18.0.0-7644.g96fa2578.el9.x86 1.2 MB/s | 2.1 MB 00:01 2023-12-06T18:12:32.556 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2023-12-06T18:12:32.575 INFO:teuthology.orchestra.run.smithi061.stdout:(12/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2023-12-06T18:12:32.611 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2023-12-06T18:12:32.684 INFO:teuthology.orchestra.run.smithi061.stdout:(13/36): python3-cryptography-36.0.1-4.el9.x86_ 6.5 MB/s | 1.1 MB 00:00 2023-12-06T18:12:32.709 INFO:teuthology.orchestra.run.smithi061.stdout:(14/36): python3-ply-3.11-14.el9.noarch.rpm 650 kB/s | 103 kB 00:00 2023-12-06T18:12:32.734 INFO:teuthology.orchestra.run.smithi061.stdout:(15/36): python3-pycparser-2.20-6.el9.noarch.rp 782 kB/s | 124 kB 00:00 2023-12-06T18:12:32.759 INFO:teuthology.orchestra.run.smithi061.stdout:(16/36): python3-requests-2.25.1-7.el9.noarch.r 1.5 MB/s | 113 kB 00:00 2023-12-06T18:12:32.785 INFO:teuthology.orchestra.run.smithi061.stdout:(17/36): python3-urllib3-1.26.5-4.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2023-12-06T18:12:32.798 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2023-12-06T18:12:32.844 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2023-12-06T18:12:32.883 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2023-12-06T18:12:32.902 INFO:teuthology.orchestra.run.smithi061.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2023-12-06T18:12:32.917 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2023-12-06T18:12:32.947 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2023-12-06T18:12:33.022 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2023-12-06T18:12:33.058 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2023-12-06T18:12:33.090 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2023-12-06T18:12:33.131 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2023-12-06T18:12:33.166 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2023-12-06T18:12:33.192 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2023-12-06T18:12:33.202 INFO:teuthology.orchestra.run.smithi061.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 104 kB/s | 43 kB 00:00 2023-12-06T18:12:33.280 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2023-12-06T18:12:33.339 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2023-12-06T18:12:33.352 INFO:teuthology.orchestra.run.smithi061.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 255 kB/s | 151 kB 00:00 2023-12-06T18:12:33.411 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2023-12-06T18:12:33.427 INFO:teuthology.orchestra.run.smithi061.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 327 kB/s | 172 kB 00:00 2023-12-06T18:12:33.469 INFO:teuthology.orchestra.run.smithi061.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 91 kB/s | 11 kB 00:00 2023-12-06T18:12:33.545 INFO:teuthology.orchestra.run.smithi061.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 151 kB/s | 18 kB 00:00 2023-12-06T18:12:33.547 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2023-12-06T18:12:33.587 INFO:teuthology.orchestra.run.smithi061.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 198 kB/s | 23 kB 00:00 2023-12-06T18:12:33.670 INFO:teuthology.orchestra.run.smithi061.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 155 kB/s | 19 kB 00:00 2023-12-06T18:12:33.672 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2023-12-06T18:12:33.712 INFO:teuthology.orchestra.run.smithi061.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 157 kB/s | 20 kB 00:00 2023-12-06T18:12:33.779 INFO:teuthology.orchestra.run.smithi061.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 426 kB/s | 46 kB 00:00 2023-12-06T18:12:33.851 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7644.g96fa2578.el9. 31/36 2023-12-06T18:12:33.878 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-2:18.0.0-7644.g96fa2578.el9.x86_64 32/36 2023-12-06T18:12:33.888 INFO:teuthology.orchestra.run.smithi061.stdout:(28/36): python3-cherrypy-18.6.1-2.el9.noarch.r 523 kB/s | 358 kB 00:00 2023-12-06T18:12:33.902 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7644.g96fa2578.el9.x86_64 32/36 2023-12-06T18:12:33.902 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-06T18:12:33.903 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-12-06T18:12:33.903 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-06T18:12:33.903 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-06T18:12:33.903 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:33.947 INFO:teuthology.orchestra.run.smithi061.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 337 kB/s | 79 kB 00:00 2023-12-06T18:12:33.980 INFO:teuthology.orchestra.run.smithi061.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 179 kB/s | 16 kB 00:00 2023-12-06T18:12:34.013 INFO:teuthology.orchestra.run.smithi061.stdout:(31/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.1 MB/s | 272 kB 00:00 2023-12-06T18:12:34.097 INFO:teuthology.orchestra.run.smithi061.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 306 kB/s | 36 kB 00:00 2023-12-06T18:12:34.189 INFO:teuthology.orchestra.run.smithi061.stdout:(33/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2023-12-06T18:12:34.214 INFO:teuthology.orchestra.run.smithi061.stdout:(34/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 336 kB/s | 90 kB 00:00 2023-12-06T18:12:34.281 INFO:teuthology.orchestra.run.smithi061.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 218 kB/s | 20 kB 00:00 2023-12-06T18:12:34.581 INFO:teuthology.orchestra.run.smithi061.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 883 kB/s | 427 kB 00:00 2023-12-06T18:12:34.585 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:12:34.585 INFO:teuthology.orchestra.run.smithi061.stdout:Total 7.1 MB/s | 30 MB 00:04 2023-12-06T18:12:34.598 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:12:34.645 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:12:34.645 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:12:35.081 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:12:35.081 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:12:35.437 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-osd-2:18.0.0-7644.g96fa2578.el9.x86_64 33/36 2023-12-06T18:12:35.464 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-osd-2:18.0.0-7644.g96fa2578.el9.x86_64 33/36 2023-12-06T18:12:35.464 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-06T18:12:35.464 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-12-06T18:12:35.465 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-06T18:12:35.465 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-06T18:12:35.465 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:35.930 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mon-2:18.0.0-7644.g96fa2578.el9.x86_64 34/36 2023-12-06T18:12:35.955 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mon-2:18.0.0-7644.g96fa2578.el9.x86_64 34/36 2023-12-06T18:12:35.955 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-06T18:12:35.955 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-12-06T18:12:35.955 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-06T18:12:35.955 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-06T18:12:35.956 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:36.164 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mds-2:18.0.0-7644.g96fa2578.el9.x86_64 35/36 2023-12-06T18:12:36.188 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mds-2:18.0.0-7644.g96fa2578.el9.x86_64 35/36 2023-12-06T18:12:36.188 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-06T18:12:36.188 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-12-06T18:12:36.188 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-06T18:12:36.189 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-06T18:12:36.189 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:36.255 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-2:18.0.0-7644.g96fa2578.el9.x86_64 36/36 2023-12-06T18:12:36.440 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:12:36.509 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2023-12-06T18:12:36.551 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2023-12-06T18:12:36.578 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2023-12-06T18:12:36.652 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2023-12-06T18:12:36.702 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2023-12-06T18:12:36.735 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2023-12-06T18:12:36.831 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2023-12-06T18:12:36.923 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2023-12-06T18:12:36.972 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2023-12-06T18:12:37.045 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2023-12-06T18:12:37.106 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 11/36 2023-12-06T18:12:37.143 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 12/36 2023-12-06T18:12:37.194 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2023-12-06T18:12:37.258 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2023-12-06T18:12:37.453 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2023-12-06T18:12:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2023-12-06T18:12:37.553 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2023-12-06T18:12:37.587 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2023-12-06T18:12:37.617 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2023-12-06T18:12:37.697 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2023-12-06T18:12:37.727 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2023-12-06T18:12:37.761 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2023-12-06T18:12:37.801 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2023-12-06T18:12:37.845 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2023-12-06T18:12:37.870 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2023-12-06T18:12:37.964 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2023-12-06T18:12:38.001 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2023-12-06T18:12:38.077 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2023-12-06T18:12:38.115 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-2:18.0.0-7644.g96fa2578.el9.x86_64 36/36 2023-12-06T18:12:38.115 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-2:18.0.0-7644.g96fa2578.el9.x86_64 1/36 2023-12-06T18:12:38.115 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mds-2:18.0.0-7644.g96fa2578.el9.x86_64 2/36 2023-12-06T18:12:38.116 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-2:18.0.0-7644.g96fa2578.el9.x86_64 3/36 2023-12-06T18:12:38.116 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mon-2:18.0.0-7644.g96fa2578.el9.x86_64 4/36 2023-12-06T18:12:38.116 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-osd-2:18.0.0-7644.g96fa2578.el9.x86_64 5/36 2023-12-06T18:12:38.116 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7644.g96fa2578.el9. 6/36 2023-12-06T18:12:38.116 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2023-12-06T18:12:38.116 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2023-12-06T18:12:38.116 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2023-12-06T18:12:38.116 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2023-12-06T18:12:38.116 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2023-12-06T18:12:38.116 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2023-12-06T18:12:38.116 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2023-12-06T18:12:38.116 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2023-12-06T18:12:38.116 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2023-12-06T18:12:38.116 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 16/36 2023-12-06T18:12:38.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 17/36 2023-12-06T18:12:38.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2023-12-06T18:12:38.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2023-12-06T18:12:38.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2023-12-06T18:12:38.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2023-12-06T18:12:38.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2023-12-06T18:12:38.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2023-12-06T18:12:38.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2023-12-06T18:12:38.117 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2023-12-06T18:12:38.118 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2023-12-06T18:12:38.118 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2023-12-06T18:12:38.118 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2023-12-06T18:12:38.118 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2023-12-06T18:12:38.119 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2023-12-06T18:12:38.119 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2023-12-06T18:12:38.119 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2023-12-06T18:12:38.119 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2023-12-06T18:12:38.119 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2023-12-06T18:12:38.119 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2023-12-06T18:12:38.217 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2023-12-06T18:12:38.338 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2023-12-06T18:12:38.513 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7644.g96fa2578.el9. 31/36 2023-12-06T18:12:38.539 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-2:18.0.0-7644.g96fa2578.el9.x86_64 32/36 2023-12-06T18:12:38.565 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7644.g96fa2578.el9.x86_64 32/36 2023-12-06T18:12:38.566 INFO:teuthology.orchestra.run.smithi061.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-06T18:12:38.566 INFO:teuthology.orchestra.run.smithi061.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-12-06T18:12:38.566 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-06T18:12:38.566 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-06T18:12:38.566 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:39.112 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2023-12-06T18:12:39.112 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:39.112 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:12:39.112 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:39.112 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:39.112 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:39.112 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core-2:18.0.0-7644.g96fa2578.el9.noarch 2023-12-06T18:12:39.112 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:39.112 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:39.112 INFO:teuthology.orchestra.run.smithi152.stdout: lua-5.4.4-4.el9.x86_64 2023-12-06T18:12:39.113 INFO:teuthology.orchestra.run.smithi152.stdout: lua-devel-5.4.4-4.el9.x86_64 2023-12-06T18:12:39.113 INFO:teuthology.orchestra.run.smithi152.stdout: luarocks-3.9.2-1.el9.noarch 2023-12-06T18:12:39.113 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-12-06T18:12:39.113 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-12-06T18:12:39.113 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-12-06T18:12:39.113 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-12-06T18:12:39.113 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-12-06T18:12:39.113 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-12-06T18:12:39.113 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-12-06T18:12:39.113 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-12-06T18:12:39.113 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-12-06T18:12:39.113 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-12-06T18:12:39.113 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-12-06T18:12:39.113 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako-1.1.4-6.el9.noarch 2023-12-06T18:12:39.114 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-12-06T18:12:39.114 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-12-06T18:12:39.114 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-12-06T18:12:39.114 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ply-3.11-14.el9.noarch 2023-12-06T18:12:39.114 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend-3.1.0-2.el9.noarch 2023-12-06T18:12:39.114 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-12-06T18:12:39.114 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser-2.20-6.el9.noarch 2023-12-06T18:12:39.114 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz-2021.1-5.el9.noarch 2023-12-06T18:12:39.114 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-2.25.1-7.el9.noarch 2023-12-06T18:12:39.114 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-12-06T18:12:39.114 INFO:teuthology.orchestra.run.smithi152.stdout: python3-toml-0.10.2-6.el9.noarch 2023-12-06T18:12:39.114 INFO:teuthology.orchestra.run.smithi152.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-12-06T18:12:39.114 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob-1.8.7-6.el9.noarch 2023-12-06T18:12:39.114 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-12-06T18:12:39.114 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-12-06T18:12:39.115 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:39.115 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:12:39.461 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-base 2023-12-06T18:12:40.046 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:52 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:12:40.139 INFO:teuthology.orchestra.run.smithi152.stdout:Package ceph-base-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:12:40.188 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:12:40.189 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2023-12-06T18:12:40.189 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:12:40.190 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-osd-2:18.0.0-7644.g96fa2578.el9.x86_64 33/36 2023-12-06T18:12:40.219 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-osd-2:18.0.0-7644.g96fa2578.el9.x86_64 33/36 2023-12-06T18:12:40.219 INFO:teuthology.orchestra.run.smithi061.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-06T18:12:40.219 INFO:teuthology.orchestra.run.smithi061.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-12-06T18:12:40.219 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-06T18:12:40.219 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-06T18:12:40.219 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:40.243 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install cephadm 2023-12-06T18:12:40.691 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mon-2:18.0.0-7644.g96fa2578.el9.x86_64 34/36 2023-12-06T18:12:40.716 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mon-2:18.0.0-7644.g96fa2578.el9.x86_64 34/36 2023-12-06T18:12:40.716 INFO:teuthology.orchestra.run.smithi061.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-06T18:12:40.716 INFO:teuthology.orchestra.run.smithi061.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-12-06T18:12:40.716 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-06T18:12:40.716 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-06T18:12:40.716 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:40.828 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:52 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:12:40.925 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mds-2:18.0.0-7644.g96fa2578.el9.x86_64 35/36 2023-12-06T18:12:40.949 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mds-2:18.0.0-7644.g96fa2578.el9.x86_64 35/36 2023-12-06T18:12:40.950 INFO:teuthology.orchestra.run.smithi061.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-06T18:12:40.950 INFO:teuthology.orchestra.run.smithi061.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-12-06T18:12:40.950 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-06T18:12:40.950 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-06T18:12:40.950 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:40.973 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:12:40.973 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:12:40.973 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2023-12-06T18:12:40.973 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:12:40.974 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:12:40.974 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm noarch 2:18.0.0-7644.g96fa2578.el9 ceph-noarch 778 k 2023-12-06T18:12:40.974 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:40.974 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:12:40.974 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:12:40.974 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:12:40.974 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:40.974 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 778 k 2023-12-06T18:12:40.975 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 790 k 2023-12-06T18:12:40.975 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:12:41.017 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-2:18.0.0-7644.g96fa2578.el9.x86_64 36/36 2023-12-06T18:12:41.414 INFO:teuthology.orchestra.run.smithi152.stdout:cephadm-18.0.0-7644.g96fa2578.el9.noarch.rpm 1.7 MB/s | 778 kB 00:00 2023-12-06T18:12:41.414 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:12:41.415 INFO:teuthology.orchestra.run.smithi152.stdout:Total 1.7 MB/s | 778 kB 00:00 2023-12-06T18:12:41.415 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:12:41.420 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:12:41.420 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:12:41.426 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:12:41.426 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:12:41.504 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:12:41.976 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephadm-2:18.0.0-7644.g96fa2578.el9.noarch 1/1 2023-12-06T18:12:42.068 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephadm-2:18.0.0-7644.g96fa2578.el9.noarch 1/1 2023-12-06T18:12:42.491 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephadm-2:18.0.0-7644.g96fa2578.el9.noarch 1/1 2023-12-06T18:12:42.778 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephadm-2:18.0.0-7644.g96fa2578.el9.noarch 1/1 2023-12-06T18:12:42.778 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:42.778 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:12:42.779 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm-2:18.0.0-7644.g96fa2578.el9.noarch 2023-12-06T18:12:42.779 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:42.779 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:12:42.796 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-2:18.0.0-7644.g96fa2578.el9.x86_64 36/36 2023-12-06T18:12:42.796 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-2:18.0.0-7644.g96fa2578.el9.x86_64 1/36 2023-12-06T18:12:42.796 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mds-2:18.0.0-7644.g96fa2578.el9.x86_64 2/36 2023-12-06T18:12:42.796 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-2:18.0.0-7644.g96fa2578.el9.x86_64 3/36 2023-12-06T18:12:42.796 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mon-2:18.0.0-7644.g96fa2578.el9.x86_64 4/36 2023-12-06T18:12:42.796 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-osd-2:18.0.0-7644.g96fa2578.el9.x86_64 5/36 2023-12-06T18:12:42.796 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7644.g96fa2578.el9. 6/36 2023-12-06T18:12:42.796 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2023-12-06T18:12:42.797 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2023-12-06T18:12:42.797 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2023-12-06T18:12:42.797 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2023-12-06T18:12:42.797 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2023-12-06T18:12:42.797 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2023-12-06T18:12:42.797 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2023-12-06T18:12:42.797 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2023-12-06T18:12:42.797 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2023-12-06T18:12:42.797 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 16/36 2023-12-06T18:12:42.797 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 17/36 2023-12-06T18:12:42.797 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2023-12-06T18:12:42.797 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2023-12-06T18:12:42.797 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2023-12-06T18:12:42.797 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2023-12-06T18:12:42.797 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2023-12-06T18:12:42.798 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2023-12-06T18:12:42.798 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2023-12-06T18:12:42.798 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2023-12-06T18:12:42.798 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2023-12-06T18:12:42.798 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2023-12-06T18:12:42.798 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2023-12-06T18:12:42.798 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2023-12-06T18:12:42.798 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2023-12-06T18:12:42.799 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2023-12-06T18:12:42.799 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2023-12-06T18:12:42.799 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2023-12-06T18:12:42.799 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2023-12-06T18:12:42.799 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2023-12-06T18:12:42.993 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-immutable-object-cache 2023-12-06T18:12:43.572 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:55 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:12:43.719 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:12:43.720 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:12:43.720 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:12:43.720 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:12:43.720 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:12:43.720 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 141 k 2023-12-06T18:12:43.720 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:43.720 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:12:43.720 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:12:43.720 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:12:43.720 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:43.721 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 141 k 2023-12-06T18:12:43.721 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 400 k 2023-12-06T18:12:43.721 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:12:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2023-12-06T18:12:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:43.815 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:12:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mds-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-modules-core-2:18.0.0-7644.g96fa2578.el9.noarch 2023-12-06T18:12:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mon-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-osd-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: lua-5.4.4-4.el9.x86_64 2023-12-06T18:12:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: lua-devel-5.4.4-4.el9.x86_64 2023-12-06T18:12:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: luarocks-3.9.2-1.el9.noarch 2023-12-06T18:12:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-12-06T18:12:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-12-06T18:12:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-12-06T18:12:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-12-06T18:12:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-12-06T18:12:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-12-06T18:12:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-12-06T18:12:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-12-06T18:12:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-12-06T18:12:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-12-06T18:12:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-12-06T18:12:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: python3-mako-1.1.4-6.el9.noarch 2023-12-06T18:12:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-12-06T18:12:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-12-06T18:12:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-12-06T18:12:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: python3-ply-3.11-14.el9.noarch 2023-12-06T18:12:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: python3-portend-3.1.0-2.el9.noarch 2023-12-06T18:12:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-12-06T18:12:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pycparser-2.20-6.el9.noarch 2023-12-06T18:12:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pytz-2021.1-5.el9.noarch 2023-12-06T18:12:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: python3-requests-2.25.1-7.el9.noarch 2023-12-06T18:12:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-12-06T18:12:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: python3-toml-0.10.2-6.el9.noarch 2023-12-06T18:12:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-12-06T18:12:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: python3-webob-1.8.7-6.el9.noarch 2023-12-06T18:12:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-12-06T18:12:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-12-06T18:12:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:43.817 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:12:44.094 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-immutable-object-cache-18.0.0-7644.g96fa25 379 kB/s | 141 kB 00:00 2023-12-06T18:12:44.094 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:12:44.095 INFO:teuthology.orchestra.run.smithi152.stdout:Total 378 kB/s | 141 kB 00:00 2023-12-06T18:12:44.095 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:12:44.102 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:12:44.102 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:12:44.158 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:12:44.159 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:12:44.165 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-base 2023-12-06T18:12:44.319 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:12:44.378 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7644.g96fa2578. 1/1 2023-12-06T18:12:44.402 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7644.g96fa2578. 1/1 2023-12-06T18:12:44.402 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-06T18:12:44.403 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-12-06T18:12:44.403 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:44.775 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:00:54 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:12:44.871 INFO:teuthology.orchestra.run.smithi061.stdout:Package ceph-base-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:12:44.922 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:12:44.923 INFO:teuthology.orchestra.run.smithi061.stdout:Nothing to do. 2023-12-06T18:12:44.923 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:12:44.985 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install cephadm 2023-12-06T18:12:45.424 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7644.g96fa2578. 1/1 2023-12-06T18:12:45.424 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:45.424 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:12:45.424 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:45.424 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:45.424 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:12:45.588 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:00:55 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:12:45.647 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-mgr 2023-12-06T18:12:45.735 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:12:45.736 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:12:45.736 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repository Size 2023-12-06T18:12:45.736 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:12:45.737 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:12:45.737 INFO:teuthology.orchestra.run.smithi061.stdout: cephadm noarch 2:18.0.0-7644.g96fa2578.el9 ceph-noarch 778 k 2023-12-06T18:12:45.737 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:45.737 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:12:45.737 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:12:45.737 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:12:45.737 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:45.737 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 778 k 2023-12-06T18:12:45.737 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 790 k 2023-12-06T18:12:45.737 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:12:46.198 INFO:teuthology.orchestra.run.smithi061.stdout:cephadm-18.0.0-7644.g96fa2578.el9.noarch.rpm 1.6 MB/s | 778 kB 00:00 2023-12-06T18:12:46.199 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:12:46.199 INFO:teuthology.orchestra.run.smithi061.stdout:Total 1.6 MB/s | 778 kB 00:00 2023-12-06T18:12:46.199 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:12:46.204 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:12:46.204 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:12:46.211 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:12:46.211 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:12:46.244 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:58 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:12:46.290 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:12:46.338 INFO:teuthology.orchestra.run.smithi152.stdout:Package ceph-mgr-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:12:46.389 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:12:46.390 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2023-12-06T18:12:46.390 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:12:46.447 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-mgr-dashboard 2023-12-06T18:12:46.652 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: cephadm-2:18.0.0-7644.g96fa2578.el9.noarch 1/1 2023-12-06T18:12:46.746 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : cephadm-2:18.0.0-7644.g96fa2578.el9.noarch 1/1 2023-12-06T18:12:47.044 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:00:59 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:12:47.116 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: cephadm-2:18.0.0-7644.g96fa2578.el9.noarch 1/1 2023-12-06T18:12:47.192 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:12:47.193 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:12:47.193 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2023-12-06T18:12:47.193 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:12:47.193 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:12:47.193 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7644.g96fa2578.el9 ceph-noarch 1.8 M 2023-12-06T18:12:47.193 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2023-12-06T18:12:47.194 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7644.g96fa2578.el9 ceph-noarch 28 k 2023-12-06T18:12:47.194 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7644.g96fa2578.el9 ceph-noarch 15 k 2023-12-06T18:12:47.194 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-12-06T18:12:47.194 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-12-06T18:12:47.194 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-12-06T18:12:47.194 INFO:teuthology.orchestra.run.smithi152.stdout:Installing weak dependencies: 2023-12-06T18:12:47.194 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-12-06T18:12:47.194 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:47.194 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:12:47.194 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:12:47.194 INFO:teuthology.orchestra.run.smithi152.stdout:Install 7 Packages 2023-12-06T18:12:47.194 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:47.195 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 2.1 M 2023-12-06T18:12:47.195 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 8.7 M 2023-12-06T18:12:47.195 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:12:47.439 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : cephadm-2:18.0.0-7644.g96fa2578.el9.noarch 1/1 2023-12-06T18:12:47.439 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:47.439 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:12:47.439 INFO:teuthology.orchestra.run.smithi061.stdout: cephadm-2:18.0.0-7644.g96fa2578.el9.noarch 2023-12-06T18:12:47.439 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:47.439 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:12:47.455 INFO:teuthology.orchestra.run.smithi152.stdout:(1/7): ceph-prometheus-alerts-18.0.0-7644.g96fa 73 kB/s | 15 kB 00:00 2023-12-06T18:12:47.522 INFO:teuthology.orchestra.run.smithi152.stdout:(2/7): ceph-grafana-dashboards-18.0.0-7644.g96f 103 kB/s | 28 kB 00:00 2023-12-06T18:12:47.636 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-immutable-object-cache 2023-12-06T18:12:47.681 INFO:teuthology.orchestra.run.smithi152.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 40 kB/s | 9.0 kB 00:00 2023-12-06T18:12:47.772 INFO:teuthology.orchestra.run.smithi152.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 336 kB/s | 31 kB 00:00 2023-12-06T18:12:47.823 INFO:teuthology.orchestra.run.smithi152.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 136 kB/s | 41 kB 00:00 2023-12-06T18:12:47.898 INFO:teuthology.orchestra.run.smithi152.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.5 MB/s | 188 kB 00:00 2023-12-06T18:12:48.031 INFO:teuthology.orchestra.run.smithi152.stdout:(7/7): ceph-mgr-dashboard-18.0.0-7644.g96fa2578 2.2 MB/s | 1.8 MB 00:00 2023-12-06T18:12:48.035 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:12:48.035 INFO:teuthology.orchestra.run.smithi152.stdout:Total 2.5 MB/s | 2.1 MB 00:00 2023-12-06T18:12:48.035 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:12:48.043 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:12:48.043 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:12:48.111 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:12:48.111 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:12:48.242 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:00:58 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:12:48.393 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:12:48.394 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:12:48.394 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:12:48.394 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:12:48.394 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:12:48.394 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 141 k 2023-12-06T18:12:48.394 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:48.394 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:12:48.394 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:12:48.394 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:12:48.394 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:48.395 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 141 k 2023-12-06T18:12:48.395 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 400 k 2023-12-06T18:12:48.395 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:12:48.429 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:12:48.483 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2023-12-06T18:12:48.517 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2023-12-06T18:12:48.563 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2023-12-06T18:12:48.597 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2023-12-06T18:12:48.611 INFO:teuthology.orchestra.run.smithi061.stdout:ceph-immutable-object-cache-18.0.0-7644.g96fa25 655 kB/s | 141 kB 00:00 2023-12-06T18:12:48.612 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:12:48.612 INFO:teuthology.orchestra.run.smithi061.stdout:Total 650 kB/s | 141 kB 00:00 2023-12-06T18:12:48.612 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:12:48.620 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:12:48.620 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:12:48.625 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7644.g96fa2578.el9.n 5/7 2023-12-06T18:12:48.679 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:12:48.679 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:12:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:12:48.865 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7644.g96fa2578.el9. 6/7 2023-12-06T18:12:48.887 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7644.g96fa2578. 1/1 2023-12-06T18:12:48.893 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7644.g96fa2578.el9.noarc 7/7 2023-12-06T18:12:48.913 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7644.g96fa2578. 1/1 2023-12-06T18:12:48.913 INFO:teuthology.orchestra.run.smithi061.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-06T18:12:48.913 INFO:teuthology.orchestra.run.smithi061.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-12-06T18:12:48.913 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:49.455 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7644.g96fa2578.el9.noarc 7/7 2023-12-06T18:12:49.455 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7644.g96fa2578.el9. 1/7 2023-12-06T18:12:49.455 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7644.g96fa2578.el9.noarc 2/7 2023-12-06T18:12:49.455 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7644.g96fa2578.el9.n 3/7 2023-12-06T18:12:49.455 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2023-12-06T18:12:49.455 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2023-12-06T18:12:49.455 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2023-12-06T18:12:49.793 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2023-12-06T18:12:49.793 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:49.793 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:12:49.793 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards-2:18.0.0-7644.g96fa2578.el9.noarch 2023-12-06T18:12:49.793 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard-2:18.0.0-7644.g96fa2578.el9.noarch 2023-12-06T18:12:49.793 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts-2:18.0.0-7644.g96fa2578.el9.noarch 2023-12-06T18:12:49.794 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-12-06T18:12:49.794 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-12-06T18:12:49.794 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-12-06T18:12:49.794 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes-2.5.1-5.el9.noarch 2023-12-06T18:12:49.794 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:49.794 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:12:49.934 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7644.g96fa2578. 1/1 2023-12-06T18:12:49.934 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:49.934 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:12:49.934 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-immutable-object-cache-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:12:49.934 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:49.934 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:12:50.036 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-mgr-diskprediction-local 2023-12-06T18:12:50.164 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-mgr 2023-12-06T18:12:50.613 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:02 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:12:50.761 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:12:50.762 INFO:teuthology.orchestra.run.smithi152.stdout:============================================================================================ 2023-12-06T18:12:50.762 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2023-12-06T18:12:50.762 INFO:teuthology.orchestra.run.smithi152.stdout:============================================================================================ 2023-12-06T18:12:50.762 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:12:50.762 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7644.g96fa2578.el9 ceph-noarch 7.4 M 2023-12-06T18:12:50.762 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2023-12-06T18:12:50.762 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-12-06T18:12:50.762 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-12-06T18:12:50.762 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-12-06T18:12:50.762 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-12-06T18:12:50.763 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-12-06T18:12:50.763 INFO:teuthology.orchestra.run.smithi152.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-12-06T18:12:50.763 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-12-06T18:12:50.763 INFO:teuthology.orchestra.run.smithi152.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-12-06T18:12:50.763 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-12-06T18:12:50.763 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-12-06T18:12:50.763 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-12-06T18:12:50.763 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:50.763 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:12:50.763 INFO:teuthology.orchestra.run.smithi152.stdout:============================================================================================ 2023-12-06T18:12:50.763 INFO:teuthology.orchestra.run.smithi152.stdout:Install 12 Packages 2023-12-06T18:12:50.763 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:12:50.764 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 37 M 2023-12-06T18:12:50.764 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 209 M 2023-12-06T18:12:50.764 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:12:50.776 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:00 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:12:50.873 INFO:teuthology.orchestra.run.smithi061.stdout:Package ceph-mgr-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:12:50.924 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:12:50.925 INFO:teuthology.orchestra.run.smithi061.stdout:Nothing to do. 2023-12-06T18:12:50.925 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:12:50.988 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-mgr-dashboard 2023-12-06T18:12:51.013 INFO:teuthology.orchestra.run.smithi152.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 153 kB/s | 32 kB 00:00 2023-12-06T18:12:51.063 INFO:teuthology.orchestra.run.smithi152.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 342 kB/s | 17 kB 00:00 2023-12-06T18:12:51.113 INFO:teuthology.orchestra.run.smithi152.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 685 kB/s | 34 kB 00:00 2023-12-06T18:12:51.238 INFO:teuthology.orchestra.run.smithi152.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 6.9 MB/s | 3.0 MB 00:00 2023-12-06T18:12:51.397 INFO:teuthology.orchestra.run.smithi152.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-12-06T18:12:51.506 INFO:teuthology.orchestra.run.smithi152.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 4.8 MB 00:00 2023-12-06T18:12:51.531 INFO:teuthology.orchestra.run.smithi152.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.6 MB/s | 217 kB 00:00 2023-12-06T18:12:51.600 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:01 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:12:51.725 INFO:teuthology.orchestra.run.smithi152.stdout:(8/12): libgfortran-11.4.1-2.3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2023-12-06T18:12:51.753 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:12:51.753 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:12:51.753 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repository Size 2023-12-06T18:12:51.754 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:12:51.754 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:12:51.754 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7644.g96fa2578.el9 ceph-noarch 1.8 M 2023-12-06T18:12:51.754 INFO:teuthology.orchestra.run.smithi061.stdout:Installing dependencies: 2023-12-06T18:12:51.754 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7644.g96fa2578.el9 ceph-noarch 28 k 2023-12-06T18:12:51.754 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7644.g96fa2578.el9 ceph-noarch 15 k 2023-12-06T18:12:51.754 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-12-06T18:12:51.754 INFO:teuthology.orchestra.run.smithi061.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-12-06T18:12:51.754 INFO:teuthology.orchestra.run.smithi061.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-12-06T18:12:51.754 INFO:teuthology.orchestra.run.smithi061.stdout:Installing weak dependencies: 2023-12-06T18:12:51.754 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-12-06T18:12:51.754 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:51.754 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:12:51.754 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:12:51.755 INFO:teuthology.orchestra.run.smithi061.stdout:Install 7 Packages 2023-12-06T18:12:51.755 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:51.755 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 2.1 M 2023-12-06T18:12:51.755 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 8.7 M 2023-12-06T18:12:51.755 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:12:51.907 INFO:teuthology.orchestra.run.smithi152.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2023-12-06T18:12:51.933 INFO:teuthology.orchestra.run.smithi152.stdout:(10/12): libquadmath-11.4.1-2.3.el9.x86_64.rpm 947 kB/s | 195 kB 00:00 2023-12-06T18:12:52.074 INFO:teuthology.orchestra.run.smithi152.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2023-12-06T18:12:52.114 INFO:teuthology.orchestra.run.smithi061.stdout:(1/7): ceph-prometheus-alerts-18.0.0-7644.g96fa 78 kB/s | 15 kB 00:00 2023-12-06T18:12:52.214 INFO:teuthology.orchestra.run.smithi061.stdout:(2/7): ceph-grafana-dashboards-18.0.0-7644.g96f 95 kB/s | 28 kB 00:00 2023-12-06T18:12:52.257 INFO:teuthology.orchestra.run.smithi152.stdout:(12/12): ceph-mgr-diskprediction-local-18.0.0-7 5.1 MB/s | 7.4 MB 00:01 2023-12-06T18:12:52.260 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:12:52.260 INFO:teuthology.orchestra.run.smithi152.stdout:Total 25 MB/s | 37 MB 00:01 2023-12-06T18:12:52.275 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:12:52.299 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:12:52.300 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:12:52.565 INFO:teuthology.orchestra.run.smithi061.stdout:(3/7): ceph-mgr-dashboard-18.0.0-7644.g96fa2578 2.7 MB/s | 1.8 MB 00:00 2023-12-06T18:12:52.691 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:12:52.692 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:12:52.815 INFO:teuthology.orchestra.run.smithi061.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 13 kB/s | 9.0 kB 00:00 2023-12-06T18:12:53.099 INFO:teuthology.orchestra.run.smithi061.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 58 kB/s | 31 kB 00:00 2023-12-06T18:12:53.215 INFO:teuthology.orchestra.run.smithi061.stdout:(6/7): python3-jwt-2.4.0-1.el9.noarch.rpm 41 kB/s | 41 kB 00:01 2023-12-06T18:12:53.392 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:12:53.475 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 1/12 2023-12-06T18:12:53.503 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 2/12 2023-12-06T18:12:53.599 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2023-12-06T18:12:53.639 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2023-12-06T18:12:53.657 INFO:teuthology.orchestra.run.smithi061.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 224 kB/s | 188 kB 00:00 2023-12-06T18:12:53.661 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:12:53.661 INFO:teuthology.orchestra.run.smithi061.stdout:Total 1.1 MB/s | 2.1 MB 00:01 2023-12-06T18:12:53.662 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:12:53.671 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:12:53.672 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:12:53.749 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:12:53.750 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:12:53.818 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2023-12-06T18:12:53.849 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-12-06T18:12:53.932 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2023-12-06T18:12:54.061 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:12:54.108 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2023-12-06T18:12:54.143 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2023-12-06T18:12:54.190 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2023-12-06T18:12:54.221 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2023-12-06T18:12:54.251 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7644.g96fa2578.el9.n 5/7 2023-12-06T18:12:54.500 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7644.g96fa2578.el9. 6/7 2023-12-06T18:12:54.527 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7644.g96fa2578.el9.noarc 7/7 2023-12-06T18:12:54.596 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2023-12-06T18:12:54.712 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2023-12-06T18:12:55.038 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7644.g96fa2578.el9.noarc 7/7 2023-12-06T18:12:55.038 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7644.g96fa2578.el9. 1/7 2023-12-06T18:12:55.038 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7644.g96fa2578.el9.noarc 2/7 2023-12-06T18:12:55.038 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7644.g96fa2578.el9.n 3/7 2023-12-06T18:12:55.039 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2023-12-06T18:12:55.039 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2023-12-06T18:12:55.039 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2023-12-06T18:12:55.406 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2023-12-06T18:12:55.406 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:55.407 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:12:55.407 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-grafana-dashboards-2:18.0.0-7644.g96fa2578.el9.noarch 2023-12-06T18:12:55.407 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-dashboard-2:18.0.0-7644.g96fa2578.el9.noarch 2023-12-06T18:12:55.407 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-prometheus-alerts-2:18.0.0-7644.g96fa2578.el9.noarch 2023-12-06T18:12:55.407 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-12-06T18:12:55.407 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-12-06T18:12:55.407 INFO:teuthology.orchestra.run.smithi061.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-12-06T18:12:55.407 INFO:teuthology.orchestra.run.smithi061.stdout: python3-routes-2.5.1-5.el9.noarch 2023-12-06T18:12:55.407 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:55.407 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:12:55.635 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-mgr-diskprediction-local 2023-12-06T18:12:55.997 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2023-12-06T18:12:56.243 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:06 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:12:56.399 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:12:56.400 INFO:teuthology.orchestra.run.smithi061.stdout:============================================================================================ 2023-12-06T18:12:56.400 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repository Size 2023-12-06T18:12:56.400 INFO:teuthology.orchestra.run.smithi061.stdout:============================================================================================ 2023-12-06T18:12:56.400 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:12:56.400 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7644.g96fa2578.el9 ceph-noarch 7.4 M 2023-12-06T18:12:56.400 INFO:teuthology.orchestra.run.smithi061.stdout:Installing dependencies: 2023-12-06T18:12:56.400 INFO:teuthology.orchestra.run.smithi061.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-12-06T18:12:56.400 INFO:teuthology.orchestra.run.smithi061.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-12-06T18:12:56.401 INFO:teuthology.orchestra.run.smithi061.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-12-06T18:12:56.401 INFO:teuthology.orchestra.run.smithi061.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-12-06T18:12:56.401 INFO:teuthology.orchestra.run.smithi061.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-12-06T18:12:56.401 INFO:teuthology.orchestra.run.smithi061.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-12-06T18:12:56.401 INFO:teuthology.orchestra.run.smithi061.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-12-06T18:12:56.401 INFO:teuthology.orchestra.run.smithi061.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-12-06T18:12:56.401 INFO:teuthology.orchestra.run.smithi061.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-12-06T18:12:56.401 INFO:teuthology.orchestra.run.smithi061.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-12-06T18:12:56.401 INFO:teuthology.orchestra.run.smithi061.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-12-06T18:12:56.401 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:56.401 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:12:56.401 INFO:teuthology.orchestra.run.smithi061.stdout:============================================================================================ 2023-12-06T18:12:56.401 INFO:teuthology.orchestra.run.smithi061.stdout:Install 12 Packages 2023-12-06T18:12:56.401 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:12:56.402 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 37 M 2023-12-06T18:12:56.402 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 209 M 2023-12-06T18:12:56.402 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:12:56.631 INFO:teuthology.orchestra.run.smithi061.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 195 kB/s | 32 kB 00:00 2023-12-06T18:12:56.672 INFO:teuthology.orchestra.run.smithi061.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 413 kB/s | 17 kB 00:00 2023-12-06T18:12:56.714 INFO:teuthology.orchestra.run.smithi061.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 831 kB/s | 34 kB 00:00 2023-12-06T18:12:56.773 INFO:teuthology.orchestra.run.smithi061.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.8 MB/s | 3.0 MB 00:00 2023-12-06T18:12:57.015 INFO:teuthology.orchestra.run.smithi061.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-12-06T18:12:57.031 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2023-12-06T18:12:57.064 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7644.g96fa2 12/12 2023-12-06T18:12:57.082 INFO:teuthology.orchestra.run.smithi061.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 4.8 MB 00:00 2023-12-06T18:12:57.107 INFO:teuthology.orchestra.run.smithi061.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 217 kB 00:00 2023-12-06T18:12:57.207 INFO:teuthology.orchestra.run.smithi061.stdout:(8/12): ceph-mgr-diskprediction-local-18.0.0-76 10 MB/s | 7.4 MB 00:00 2023-12-06T18:12:57.349 INFO:teuthology.orchestra.run.smithi061.stdout:(9/12): libgfortran-11.4.1-2.3.el9.x86_64.rpm 3.3 MB/s | 803 kB 00:00 2023-12-06T18:12:57.525 INFO:teuthology.orchestra.run.smithi061.stdout:(10/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2023-12-06T18:12:57.550 INFO:teuthology.orchestra.run.smithi061.stdout:(11/12): libquadmath-11.4.1-2.3.el9.x86_64.rpm 574 kB/s | 195 kB 00:00 2023-12-06T18:12:57.633 INFO:teuthology.orchestra.run.smithi061.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 866 kB/s | 245 kB 00:00 2023-12-06T18:12:57.636 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:12:57.636 INFO:teuthology.orchestra.run.smithi061.stdout:Total 30 MB/s | 37 MB 00:01 2023-12-06T18:12:57.649 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:12:57.674 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:12:57.674 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:12:58.077 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:12:58.077 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:12:58.809 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:12:58.887 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 1/12 2023-12-06T18:12:58.921 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 2/12 2023-12-06T18:12:59.020 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2023-12-06T18:12:59.057 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2023-12-06T18:12:59.241 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2023-12-06T18:12:59.273 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-12-06T18:12:59.354 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2023-12-06T18:12:59.624 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7644.g96fa2 12/12 2023-12-06T18:12:59.624 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7644.g96fa2 1/12 2023-12-06T18:12:59.625 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2023-12-06T18:12:59.625 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2023-12-06T18:12:59.625 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2023-12-06T18:12:59.625 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2023-12-06T18:12:59.625 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-12-06T18:12:59.625 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2023-12-06T18:12:59.625 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2023-12-06T18:12:59.625 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2023-12-06T18:12:59.625 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 10/12 2023-12-06T18:12:59.625 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 11/12 2023-12-06T18:13:00.034 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2023-12-06T18:13:00.095 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2023-12-06T18:13:00.095 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:00.095 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:13:00.095 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7644.g96fa2578.el9.noarch 2023-12-06T18:13:00.095 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-12-06T18:13:00.095 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-12-06T18:13:00.095 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-12-06T18:13:00.095 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-12-06T18:13:00.095 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-12-06T18:13:00.096 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-0.3.21-2.el9.x86_64 2023-12-06T18:13:00.096 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-12-06T18:13:00.096 INFO:teuthology.orchestra.run.smithi152.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-12-06T18:13:00.096 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-12-06T18:13:00.096 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-12-06T18:13:00.096 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-12-06T18:13:00.096 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:00.096 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:13:00.138 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2023-12-06T18:13:00.482 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-mgr-rook 2023-12-06T18:13:01.070 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:13 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:13:01.217 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:13:01.218 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================== 2023-12-06T18:13:01.218 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2023-12-06T18:13:01.218 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================== 2023-12-06T18:13:01.218 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:13:01.218 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook noarch 2:18.0.0-7644.g96fa2578.el9 ceph-noarch 50 k 2023-12-06T18:13:01.218 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2023-12-06T18:13:01.218 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-12-06T18:13:01.218 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-12-06T18:13:01.218 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-12-06T18:13:01.219 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-12-06T18:13:01.219 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-12-06T18:13:01.219 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-12-06T18:13:01.219 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-12-06T18:13:01.219 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-12-06T18:13:01.219 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-12-06T18:13:01.219 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-12-06T18:13:01.219 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-12-06T18:13:01.219 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-12-06T18:13:01.219 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:01.219 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:13:01.219 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================== 2023-12-06T18:13:01.219 INFO:teuthology.orchestra.run.smithi152.stdout:Install 13 Packages 2023-12-06T18:13:01.219 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:01.220 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 2.0 M 2023-12-06T18:13:01.220 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 26 M 2023-12-06T18:13:01.220 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:13:01.434 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2023-12-06T18:13:01.446 INFO:teuthology.orchestra.run.smithi152.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 129 kB/s | 18 kB 00:00 2023-12-06T18:13:01.471 INFO:teuthology.orchestra.run.smithi152.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 147 kB/s | 24 kB 00:00 2023-12-06T18:13:01.538 INFO:teuthology.orchestra.run.smithi152.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 174 kB 00:00 2023-12-06T18:13:01.563 INFO:teuthology.orchestra.run.smithi152.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 132 kB 00:00 2023-12-06T18:13:01.588 INFO:teuthology.orchestra.run.smithi152.stdout:(5/13): ceph-mgr-rook-18.0.0-7644.g96fa2578.el9 177 kB/s | 50 kB 00:00 2023-12-06T18:13:01.614 INFO:teuthology.orchestra.run.smithi152.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 211 kB 00:00 2023-12-06T18:13:01.639 INFO:teuthology.orchestra.run.smithi152.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 569 kB/s | 43 kB 00:00 2023-12-06T18:13:02.014 INFO:teuthology.orchestra.run.smithi152.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 40 kB/s | 15 kB 00:00 2023-12-06T18:13:02.039 INFO:teuthology.orchestra.run.smithi152.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 71 kB/s | 32 kB 00:00 2023-12-06T18:13:02.273 INFO:teuthology.orchestra.run.smithi152.stdout:(10/13): python3-google-auth-2.23.4-1.el9.noarc 312 kB/s | 197 kB 00:00 2023-12-06T18:13:02.381 INFO:teuthology.orchestra.run.smithi152.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 173 kB/s | 59 kB 00:00 2023-12-06T18:13:02.498 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2023-12-06T18:13:02.515 INFO:teuthology.orchestra.run.smithi152.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 2.0 MB/s | 1.0 MB 00:00 2023-12-06T18:13:02.521 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7644.g96fa2 12/12 2023-12-06T18:13:02.540 INFO:teuthology.orchestra.run.smithi152.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 336 kB/s | 90 kB 00:00 2023-12-06T18:13:02.544 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:02.545 INFO:teuthology.orchestra.run.smithi152.stdout:Total 1.5 MB/s | 2.0 MB 00:01 2023-12-06T18:13:02.545 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:13:02.558 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:13:02.558 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:13:02.658 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:13:02.658 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:13:03.185 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:13:03.363 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2023-12-06T18:13:03.432 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2023-12-06T18:13:03.496 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2023-12-06T18:13:03.534 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2023-12-06T18:13:03.575 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2023-12-06T18:13:03.668 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2023-12-06T18:13:03.777 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 7/13 2023-12-06T18:13:03.827 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2023-12-06T18:13:04.544 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2023-12-06T18:13:04.598 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2023-12-06T18:13:04.632 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2023-12-06T18:13:04.679 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2023-12-06T18:13:04.701 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-rook-2:18.0.0-7644.g96fa2578.el9.noarch 13/13 2023-12-06T18:13:05.040 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7644.g96fa2 12/12 2023-12-06T18:13:05.040 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7644.g96fa2 1/12 2023-12-06T18:13:05.040 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2023-12-06T18:13:05.040 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2023-12-06T18:13:05.040 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2023-12-06T18:13:05.040 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2023-12-06T18:13:05.040 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2023-12-06T18:13:05.040 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2023-12-06T18:13:05.040 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2023-12-06T18:13:05.041 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2023-12-06T18:13:05.041 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 10/12 2023-12-06T18:13:05.041 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 11/12 2023-12-06T18:13:05.437 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2023-12-06T18:13:05.437 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:05.437 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:13:05.437 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7644.g96fa2578.el9.noarch 2023-12-06T18:13:05.437 INFO:teuthology.orchestra.run.smithi061.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-12-06T18:13:05.437 INFO:teuthology.orchestra.run.smithi061.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-12-06T18:13:05.437 INFO:teuthology.orchestra.run.smithi061.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-12-06T18:13:05.437 INFO:teuthology.orchestra.run.smithi061.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-12-06T18:13:05.437 INFO:teuthology.orchestra.run.smithi061.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-12-06T18:13:05.437 INFO:teuthology.orchestra.run.smithi061.stdout: openblas-0.3.21-2.el9.x86_64 2023-12-06T18:13:05.437 INFO:teuthology.orchestra.run.smithi061.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-12-06T18:13:05.437 INFO:teuthology.orchestra.run.smithi061.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-12-06T18:13:05.438 INFO:teuthology.orchestra.run.smithi061.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-12-06T18:13:05.438 INFO:teuthology.orchestra.run.smithi061.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-12-06T18:13:05.438 INFO:teuthology.orchestra.run.smithi061.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-12-06T18:13:05.438 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:05.438 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:13:05.583 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7644.g96fa2578.el9.noarch 13/13 2023-12-06T18:13:05.583 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7644.g96fa2578.el9.noarch 1/13 2023-12-06T18:13:05.584 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2023-12-06T18:13:05.584 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2023-12-06T18:13:05.584 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2023-12-06T18:13:05.584 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2023-12-06T18:13:05.584 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2023-12-06T18:13:05.584 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2023-12-06T18:13:05.584 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2023-12-06T18:13:05.584 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2023-12-06T18:13:05.584 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 10/13 2023-12-06T18:13:05.584 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2023-12-06T18:13:05.584 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2023-12-06T18:13:05.797 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-mgr-rook 2023-12-06T18:13:06.001 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2023-12-06T18:13:06.001 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:06.001 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:13:06.001 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook-2:18.0.0-7644.g96fa2578.el9.noarch 2023-12-06T18:13:06.001 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-12-06T18:13:06.001 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-12-06T18:13:06.001 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-12-06T18:13:06.001 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-12-06T18:13:06.001 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-12-06T18:13:06.001 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-12-06T18:13:06.002 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-12-06T18:13:06.002 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-12-06T18:13:06.002 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-12-06T18:13:06.002 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-12-06T18:13:06.002 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa-4.9-2.el9.noarch 2023-12-06T18:13:06.002 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-12-06T18:13:06.002 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:06.002 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:13:06.251 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-mgr-cephadm 2023-12-06T18:13:06.414 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:16 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:13:06.567 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:13:06.569 INFO:teuthology.orchestra.run.smithi061.stdout:======================================================================================== 2023-12-06T18:13:06.569 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repository Size 2023-12-06T18:13:06.569 INFO:teuthology.orchestra.run.smithi061.stdout:======================================================================================== 2023-12-06T18:13:06.569 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:13:06.569 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-rook noarch 2:18.0.0-7644.g96fa2578.el9 ceph-noarch 50 k 2023-12-06T18:13:06.569 INFO:teuthology.orchestra.run.smithi061.stdout:Installing dependencies: 2023-12-06T18:13:06.569 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-12-06T18:13:06.569 INFO:teuthology.orchestra.run.smithi061.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-12-06T18:13:06.569 INFO:teuthology.orchestra.run.smithi061.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-12-06T18:13:06.569 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-12-06T18:13:06.569 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-12-06T18:13:06.570 INFO:teuthology.orchestra.run.smithi061.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-12-06T18:13:06.570 INFO:teuthology.orchestra.run.smithi061.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-12-06T18:13:06.570 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-12-06T18:13:06.570 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-12-06T18:13:06.570 INFO:teuthology.orchestra.run.smithi061.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-12-06T18:13:06.570 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-12-06T18:13:06.570 INFO:teuthology.orchestra.run.smithi061.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-12-06T18:13:06.570 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:06.570 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:13:06.570 INFO:teuthology.orchestra.run.smithi061.stdout:======================================================================================== 2023-12-06T18:13:06.570 INFO:teuthology.orchestra.run.smithi061.stdout:Install 13 Packages 2023-12-06T18:13:06.570 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:06.571 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 2.0 M 2023-12-06T18:13:06.571 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 26 M 2023-12-06T18:13:06.571 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:13:06.842 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:18 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:13:06.894 INFO:teuthology.orchestra.run.smithi061.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 117 kB/s | 18 kB 00:00 2023-12-06T18:13:06.920 INFO:teuthology.orchestra.run.smithi061.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 136 kB/s | 24 kB 00:00 2023-12-06T18:13:06.954 INFO:teuthology.orchestra.run.smithi061.stdout:(3/13): ceph-mgr-rook-18.0.0-7644.g96fa2578.el9 235 kB/s | 50 kB 00:00 2023-12-06T18:13:06.987 INFO:teuthology.orchestra.run.smithi061.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.8 MB/s | 174 kB 00:00 2023-12-06T18:13:06.990 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:13:06.994 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================== 2023-12-06T18:13:06.994 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2023-12-06T18:13:06.994 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================== 2023-12-06T18:13:06.994 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:13:06.994 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7644.g96fa2578.el9 ceph-noarch 133 k 2023-12-06T18:13:06.994 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2023-12-06T18:13:06.994 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-12-06T18:13:06.994 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-12-06T18:13:06.994 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-12-06T18:13:06.994 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-12-06T18:13:06.994 INFO:teuthology.orchestra.run.smithi152.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-12-06T18:13:06.994 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:06.995 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:13:06.995 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================== 2023-12-06T18:13:06.995 INFO:teuthology.orchestra.run.smithi152.stdout:Install 6 Packages 2023-12-06T18:13:06.995 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:06.995 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 6.8 M 2023-12-06T18:13:06.995 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 33 M 2023-12-06T18:13:06.995 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:13:07.021 INFO:teuthology.orchestra.run.smithi061.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 132 kB 00:00 2023-12-06T18:13:07.046 INFO:teuthology.orchestra.run.smithi061.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.2 MB/s | 211 kB 00:00 2023-12-06T18:13:07.063 INFO:teuthology.orchestra.run.smithi061.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 570 kB/s | 43 kB 00:00 2023-12-06T18:13:07.238 INFO:teuthology.orchestra.run.smithi061.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 148 kB/s | 32 kB 00:00 2023-12-06T18:13:07.274 INFO:teuthology.orchestra.run.smithi152.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.1 MB/s | 228 kB 00:00 2023-12-06T18:13:07.280 INFO:teuthology.orchestra.run.smithi061.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 69 kB/s | 15 kB 00:00 2023-12-06T18:13:07.364 INFO:teuthology.orchestra.run.smithi061.stdout:(10/13): python3-google-auth-2.23.4-1.el9.noarc 657 kB/s | 197 kB 00:00 2023-12-06T18:13:07.408 INFO:teuthology.orchestra.run.smithi152.stdout:(2/6): ceph-mgr-cephadm-18.0.0-7644.g96fa2578.e 397 kB/s | 133 kB 00:00 2023-12-06T18:13:07.517 INFO:teuthology.orchestra.run.smithi152.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 5.8 MB 00:00 2023-12-06T18:13:07.539 INFO:teuthology.orchestra.run.smithi061.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 3.4 MB/s | 1.0 MB 00:00 2023-12-06T18:13:07.556 INFO:teuthology.orchestra.run.smithi061.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 214 kB/s | 59 kB 00:00 2023-12-06T18:13:07.581 INFO:teuthology.orchestra.run.smithi061.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 413 kB/s | 90 kB 00:00 2023-12-06T18:13:07.586 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:07.586 INFO:teuthology.orchestra.run.smithi061.stdout:Total 2.0 MB/s | 2.0 MB 00:01 2023-12-06T18:13:07.586 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:13:07.600 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:13:07.600 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:13:07.703 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:13:07.704 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:13:07.900 INFO:teuthology.orchestra.run.smithi152.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 142 kB/s | 51 kB 00:00 2023-12-06T18:13:07.925 INFO:teuthology.orchestra.run.smithi152.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 112 kB/s | 58 kB 00:00 2023-12-06T18:13:08.059 INFO:teuthology.orchestra.run.smithi152.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 699 kB/s | 548 kB 00:00 2023-12-06T18:13:08.062 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:08.062 INFO:teuthology.orchestra.run.smithi152.stdout:Total 6.3 MB/s | 6.8 MB 00:01 2023-12-06T18:13:08.063 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:13:08.071 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:13:08.071 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:13:08.160 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:13:08.161 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:13:08.227 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:13:08.393 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2023-12-06T18:13:08.455 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:13:08.458 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2023-12-06T18:13:08.522 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2023-12-06T18:13:08.558 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2023-12-06T18:13:08.600 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2023-12-06T18:13:08.622 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2023-12-06T18:13:08.676 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2023-12-06T18:13:08.695 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2023-12-06T18:13:08.805 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 7/13 2023-12-06T18:13:08.860 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2023-12-06T18:13:09.187 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2023-12-06T18:13:09.270 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2023-12-06T18:13:09.340 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2023-12-06T18:13:09.364 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7644.g96fa2578.el9.noarch 6/6 2023-12-06T18:13:09.622 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2023-12-06T18:13:09.664 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2023-12-06T18:13:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2023-12-06T18:13:09.771 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2023-12-06T18:13:09.792 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-rook-2:18.0.0-7644.g96fa2578.el9.noarch 13/13 2023-12-06T18:13:10.178 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7644.g96fa2578.el9.noarch 6/6 2023-12-06T18:13:10.178 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7644.g96fa2578.el9.noarch 1/6 2023-12-06T18:13:10.179 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2023-12-06T18:13:10.179 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2023-12-06T18:13:10.179 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2023-12-06T18:13:10.179 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2023-12-06T18:13:10.511 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2023-12-06T18:13:10.512 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:10.512 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:13:10.512 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm-2:18.0.0-7644.g96fa2578.el9.noarch 2023-12-06T18:13:10.512 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-12-06T18:13:10.512 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel-2.9.1-2.el9.noarch 2023-12-06T18:13:10.512 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-12-06T18:13:10.512 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-12-06T18:13:10.512 INFO:teuthology.orchestra.run.smithi152.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-12-06T18:13:10.512 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:10.512 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:13:10.607 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7644.g96fa2578.el9.noarch 13/13 2023-12-06T18:13:10.607 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7644.g96fa2578.el9.noarch 1/13 2023-12-06T18:13:10.607 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2023-12-06T18:13:10.608 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2023-12-06T18:13:10.608 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2023-12-06T18:13:10.608 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2023-12-06T18:13:10.608 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2023-12-06T18:13:10.608 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2023-12-06T18:13:10.608 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2023-12-06T18:13:10.608 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2023-12-06T18:13:10.608 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 10/13 2023-12-06T18:13:10.608 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2023-12-06T18:13:10.608 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2023-12-06T18:13:10.750 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-fuse 2023-12-06T18:13:10.997 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2023-12-06T18:13:10.997 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:10.997 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:13:10.997 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-rook-2:18.0.0-7644.g96fa2578.el9.noarch 2023-12-06T18:13:10.997 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-12-06T18:13:10.997 INFO:teuthology.orchestra.run.smithi061.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-12-06T18:13:10.997 INFO:teuthology.orchestra.run.smithi061.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-12-06T18:13:10.997 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-12-06T18:13:10.997 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-12-06T18:13:10.997 INFO:teuthology.orchestra.run.smithi061.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-12-06T18:13:10.998 INFO:teuthology.orchestra.run.smithi061.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-12-06T18:13:10.998 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-12-06T18:13:10.998 INFO:teuthology.orchestra.run.smithi061.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-12-06T18:13:10.998 INFO:teuthology.orchestra.run.smithi061.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-12-06T18:13:10.998 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rsa-4.9-2.el9.noarch 2023-12-06T18:13:10.998 INFO:teuthology.orchestra.run.smithi061.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-12-06T18:13:10.998 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:10.998 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:13:11.240 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-mgr-cephadm 2023-12-06T18:13:11.341 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:23 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:13:11.488 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:13:11.488 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:11.488 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2023-12-06T18:13:11.488 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:11.488 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:13:11.488 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 860 k 2023-12-06T18:13:11.488 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2023-12-06T18:13:11.489 INFO:teuthology.orchestra.run.smithi152.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-12-06T18:13:11.489 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:11.489 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:13:11.489 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:11.489 INFO:teuthology.orchestra.run.smithi152.stdout:Install 2 Packages 2023-12-06T18:13:11.489 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:11.489 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 939 k 2023-12-06T18:13:11.489 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 2.7 M 2023-12-06T18:13:11.489 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:13:11.651 INFO:teuthology.orchestra.run.smithi152.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 486 kB/s | 79 kB 00:00 2023-12-06T18:13:11.857 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:21 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:13:11.952 INFO:teuthology.orchestra.run.smithi152.stdout:(2/2): ceph-fuse-18.0.0-7644.g96fa2578.el9.x86_ 1.8 MB/s | 860 kB 00:00 2023-12-06T18:13:11.953 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:11.953 INFO:teuthology.orchestra.run.smithi152.stdout:Total 2.0 MB/s | 939 kB 00:00 2023-12-06T18:13:11.953 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:13:11.973 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:13:11.974 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:13:12.011 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:13:12.012 INFO:teuthology.orchestra.run.smithi061.stdout:======================================================================================== 2023-12-06T18:13:12.012 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repository Size 2023-12-06T18:13:12.012 INFO:teuthology.orchestra.run.smithi061.stdout:======================================================================================== 2023-12-06T18:13:12.012 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:13:12.012 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7644.g96fa2578.el9 ceph-noarch 133 k 2023-12-06T18:13:12.012 INFO:teuthology.orchestra.run.smithi061.stdout:Installing dependencies: 2023-12-06T18:13:12.012 INFO:teuthology.orchestra.run.smithi061.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-12-06T18:13:12.013 INFO:teuthology.orchestra.run.smithi061.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-12-06T18:13:12.013 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-12-06T18:13:12.013 INFO:teuthology.orchestra.run.smithi061.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-12-06T18:13:12.013 INFO:teuthology.orchestra.run.smithi061.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-12-06T18:13:12.013 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:12.013 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:13:12.013 INFO:teuthology.orchestra.run.smithi061.stdout:======================================================================================== 2023-12-06T18:13:12.013 INFO:teuthology.orchestra.run.smithi061.stdout:Install 6 Packages 2023-12-06T18:13:12.013 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:12.014 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 6.8 M 2023-12-06T18:13:12.014 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 33 M 2023-12-06T18:13:12.014 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:13:12.049 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:13:12.049 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:13:12.239 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:13:12.382 INFO:teuthology.orchestra.run.smithi061.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.1 MB/s | 228 kB 00:00 2023-12-06T18:13:12.383 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2023-12-06T18:13:12.468 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-fuse-2:18.0.0-7644.g96fa2578.el9.x86_64 2/2 2023-12-06T18:13:12.541 INFO:teuthology.orchestra.run.smithi061.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 5.8 MB 00:00 2023-12-06T18:13:12.567 INFO:teuthology.orchestra.run.smithi061.stdout:(3/6): ceph-mgr-cephadm-18.0.0-7644.g96fa2578.e 338 kB/s | 133 kB 00:00 2023-12-06T18:13:12.592 INFO:teuthology.orchestra.run.smithi061.stdout:(4/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 2.6 MB/s | 548 kB 00:00 2023-12-06T18:13:12.642 INFO:teuthology.orchestra.run.smithi061.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 574 kB/s | 58 kB 00:00 2023-12-06T18:13:12.667 INFO:teuthology.orchestra.run.smithi061.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 509 kB/s | 51 kB 00:00 2023-12-06T18:13:12.669 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:12.669 INFO:teuthology.orchestra.run.smithi061.stdout:Total 10 MB/s | 6.8 MB 00:00 2023-12-06T18:13:12.669 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:13:12.679 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:13:12.679 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:13:12.774 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:13:12.774 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:13:13.074 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:13:13.199 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-fuse-2:18.0.0-7644.g96fa2578.el9.x86_64 2/2 2023-12-06T18:13:13.199 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-fuse-2:18.0.0-7644.g96fa2578.el9.x86_64 1/2 2023-12-06T18:13:13.257 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2023-12-06T18:13:13.328 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2023-12-06T18:13:13.545 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2023-12-06T18:13:13.545 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:13.545 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:13:13.545 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse-2:18.0.0-7644.g96fa2578.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2023-12-06T18:13:13.545 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:13.545 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:13:13.809 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-volume 2023-12-06T18:13:13.879 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2023-12-06T18:13:13.976 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2023-12-06T18:13:14.061 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2023-12-06T18:13:14.091 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7644.g96fa2578.el9.noarch 6/6 2023-12-06T18:13:14.413 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:26 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:13:14.560 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:13:14.561 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:14.561 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2023-12-06T18:13:14.561 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:14.561 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:13:14.561 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume noarch 2:18.0.0-7644.g96fa2578.el9 ceph-noarch 262 k 2023-12-06T18:13:14.561 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:14.561 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:13:14.561 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:14.561 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:13:14.562 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:14.562 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 262 k 2023-12-06T18:13:14.562 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 1.2 M 2023-12-06T18:13:14.562 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:13:14.885 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7644.g96fa2578.el9.noarch 6/6 2023-12-06T18:13:14.885 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7644.g96fa2578.el9.noarch 1/6 2023-12-06T18:13:14.885 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2023-12-06T18:13:14.885 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2023-12-06T18:13:14.885 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2023-12-06T18:13:14.885 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2023-12-06T18:13:15.096 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-volume-18.0.0-7644.g96fa2578.el9.noarch.rp 491 kB/s | 262 kB 00:00 2023-12-06T18:13:15.097 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:15.097 INFO:teuthology.orchestra.run.smithi152.stdout:Total 490 kB/s | 262 kB 00:00 2023-12-06T18:13:15.097 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:13:15.105 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:13:15.106 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:13:15.140 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:13:15.140 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:13:15.194 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2023-12-06T18:13:15.194 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:15.194 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:13:15.195 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-cephadm-2:18.0.0-7644.g96fa2578.el9.noarch 2023-12-06T18:13:15.195 INFO:teuthology.orchestra.run.smithi061.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-12-06T18:13:15.195 INFO:teuthology.orchestra.run.smithi061.stdout: python3-babel-2.9.1-2.el9.noarch 2023-12-06T18:13:15.195 INFO:teuthology.orchestra.run.smithi061.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-12-06T18:13:15.195 INFO:teuthology.orchestra.run.smithi061.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-12-06T18:13:15.195 INFO:teuthology.orchestra.run.smithi061.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-12-06T18:13:15.195 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:15.195 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:13:15.356 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:13:15.429 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-fuse 2023-12-06T18:13:15.431 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-volume-2:18.0.0-7644.g96fa2578.el9.noarch 1/1 2023-12-06T18:13:15.446 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-volume-2:18.0.0-7644.g96fa2578.el9.noarch 1/1 2023-12-06T18:13:15.446 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-06T18:13:15.446 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-12-06T18:13:15.447 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:16.041 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:26 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:13:16.193 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:13:16.193 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:16.194 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repository Size 2023-12-06T18:13:16.194 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:16.194 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:13:16.194 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-fuse x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 860 k 2023-12-06T18:13:16.194 INFO:teuthology.orchestra.run.smithi061.stdout:Installing dependencies: 2023-12-06T18:13:16.194 INFO:teuthology.orchestra.run.smithi061.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-12-06T18:13:16.194 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:16.195 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:13:16.195 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:16.195 INFO:teuthology.orchestra.run.smithi061.stdout:Install 2 Packages 2023-12-06T18:13:16.195 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:16.195 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 939 k 2023-12-06T18:13:16.195 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 2.7 M 2023-12-06T18:13:16.195 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:13:16.345 INFO:teuthology.orchestra.run.smithi061.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 524 kB/s | 79 kB 00:00 2023-12-06T18:13:16.534 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-volume-2:18.0.0-7644.g96fa2578.el9.noarch 1/1 2023-12-06T18:13:16.535 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:16.535 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:13:16.535 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume-2:18.0.0-7644.g96fa2578.el9.noarch 2023-12-06T18:13:16.535 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:16.535 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:13:16.678 INFO:teuthology.orchestra.run.smithi061.stdout:(2/2): ceph-fuse-18.0.0-7644.g96fa2578.el9.x86_ 1.7 MB/s | 860 kB 00:00 2023-12-06T18:13:16.680 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:16.680 INFO:teuthology.orchestra.run.smithi061.stdout:Total 1.9 MB/s | 939 kB 00:00 2023-12-06T18:13:16.680 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:13:16.702 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:13:16.702 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:13:16.782 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:13:16.782 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:13:16.831 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install librados-devel 2023-12-06T18:13:16.972 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:13:17.115 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2023-12-06T18:13:17.182 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-fuse-2:18.0.0-7644.g96fa2578.el9.x86_64 2/2 2023-12-06T18:13:17.443 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:29 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:13:17.589 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:13:17.590 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:17.590 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:13:17.590 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:17.590 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:13:17.590 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 126 k 2023-12-06T18:13:17.590 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:17.590 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:13:17.590 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:17.590 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:13:17.590 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:17.590 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 126 k 2023-12-06T18:13:17.590 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 456 k 2023-12-06T18:13:17.590 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:13:17.800 INFO:teuthology.orchestra.run.smithi152.stdout:librados-devel-18.0.0-7644.g96fa2578.el9.x86_64 604 kB/s | 126 kB 00:00 2023-12-06T18:13:17.801 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:17.801 INFO:teuthology.orchestra.run.smithi152.stdout:Total 601 kB/s | 126 kB 00:00 2023-12-06T18:13:17.801 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:13:17.806 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:13:17.806 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:13:17.862 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:13:17.862 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:13:17.946 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-fuse-2:18.0.0-7644.g96fa2578.el9.x86_64 2/2 2023-12-06T18:13:17.946 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-fuse-2:18.0.0-7644.g96fa2578.el9.x86_64 1/2 2023-12-06T18:13:18.010 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:13:18.106 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librados-devel-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:18.300 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2023-12-06T18:13:18.300 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:18.300 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:13:18.300 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-fuse-2:18.0.0-7644.g96fa2578.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2023-12-06T18:13:18.300 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:18.300 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:13:18.554 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados-devel-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:18.576 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-volume 2023-12-06T18:13:18.864 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados-devel-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:18.864 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:18.864 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:13:18.865 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:13:18.865 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:18.865 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:13:19.147 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install libcephfs2 2023-12-06T18:13:19.195 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:29 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:13:19.345 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:13:19.346 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:19.346 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repository Size 2023-12-06T18:13:19.346 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:19.346 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:13:19.346 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-volume noarch 2:18.0.0-7644.g96fa2578.el9 ceph-noarch 262 k 2023-12-06T18:13:19.346 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:19.346 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:13:19.347 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:19.347 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:13:19.347 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:19.347 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 262 k 2023-12-06T18:13:19.347 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 1.2 M 2023-12-06T18:13:19.347 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:13:19.665 INFO:teuthology.orchestra.run.smithi061.stdout:ceph-volume-18.0.0-7644.g96fa2578.el9.noarch.rp 824 kB/s | 262 kB 00:00 2023-12-06T18:13:19.666 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:19.666 INFO:teuthology.orchestra.run.smithi061.stdout:Total 820 kB/s | 262 kB 00:00 2023-12-06T18:13:19.666 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:13:19.675 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:13:19.675 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:13:19.717 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:13:19.717 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:13:19.747 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:31 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:13:19.845 INFO:teuthology.orchestra.run.smithi152.stdout:Package libcephfs2-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:13:19.896 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:13:19.897 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2023-12-06T18:13:19.898 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:13:19.945 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:13:19.952 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install libcephfs-devel 2023-12-06T18:13:20.017 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-volume-2:18.0.0-7644.g96fa2578.el9.noarch 1/1 2023-12-06T18:13:20.034 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-volume-2:18.0.0-7644.g96fa2578.el9.noarch 1/1 2023-12-06T18:13:20.034 INFO:teuthology.orchestra.run.smithi061.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-06T18:13:20.034 INFO:teuthology.orchestra.run.smithi061.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-12-06T18:13:20.034 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:20.539 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:32 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:13:20.683 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:13:20.684 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:20.684 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:13:20.684 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:20.684 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:13:20.684 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 31 k 2023-12-06T18:13:20.684 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:20.684 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:13:20.685 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:20.685 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:13:20.685 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:20.685 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 31 k 2023-12-06T18:13:20.685 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 139 k 2023-12-06T18:13:20.685 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:13:20.953 INFO:teuthology.orchestra.run.smithi152.stdout:libcephfs-devel-18.0.0-7644.g96fa2578.el9.x86_6 116 kB/s | 31 kB 00:00 2023-12-06T18:13:20.953 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:20.954 INFO:teuthology.orchestra.run.smithi152.stdout:Total 116 kB/s | 31 kB 00:00 2023-12-06T18:13:20.954 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:13:20.957 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:13:20.957 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:13:20.987 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:13:20.988 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:13:21.082 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:13:21.092 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-volume-2:18.0.0-7644.g96fa2578.el9.noarch 1/1 2023-12-06T18:13:21.092 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:21.092 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:13:21.092 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-volume-2:18.0.0-7644.g96fa2578.el9.noarch 2023-12-06T18:13:21.092 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:21.093 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:13:21.208 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs-devel-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:21.400 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install librados-devel 2023-12-06T18:13:21.615 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:21.909 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs-devel-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:21.909 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:21.909 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:13:21.909 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:13:21.909 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:21.909 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:13:22.024 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:32 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:13:22.092 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install librados2 2023-12-06T18:13:22.175 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:13:22.175 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:13:22.176 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:22.176 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:13:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: librados-devel x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 126 k 2023-12-06T18:13:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:22.176 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:13:22.176 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:22.176 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:13:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:22.176 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 126 k 2023-12-06T18:13:22.176 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 456 k 2023-12-06T18:13:22.176 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:13:22.462 INFO:teuthology.orchestra.run.smithi061.stdout:librados-devel-18.0.0-7644.g96fa2578.el9.x86_64 444 kB/s | 126 kB 00:00 2023-12-06T18:13:22.462 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:22.462 INFO:teuthology.orchestra.run.smithi061.stdout:Total 442 kB/s | 126 kB 00:00 2023-12-06T18:13:22.462 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:13:22.468 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:13:22.468 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:13:22.527 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:13:22.527 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:13:22.668 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:34 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:13:22.684 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:13:22.763 INFO:teuthology.orchestra.run.smithi152.stdout:Package librados2-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:13:22.780 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : librados-devel-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:22.813 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:13:22.814 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2023-12-06T18:13:22.814 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:13:22.870 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install librbd1 2023-12-06T18:13:23.273 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librados-devel-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:23.465 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:35 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:13:23.561 INFO:teuthology.orchestra.run.smithi152.stdout:Package librbd1-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:13:23.591 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librados-devel-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:23.591 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:23.591 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:13:23.591 INFO:teuthology.orchestra.run.smithi061.stdout: librados-devel-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:13:23.591 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:23.591 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:13:23.611 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:13:23.612 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2023-12-06T18:13:23.612 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:13:23.665 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install python3-rados 2023-12-06T18:13:23.851 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install libcephfs2 2023-12-06T18:13:24.248 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:36 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:13:24.343 INFO:teuthology.orchestra.run.smithi152.stdout:Package python3-rados-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:13:24.393 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:13:24.395 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2023-12-06T18:13:24.395 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:13:24.457 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install python3-rgw 2023-12-06T18:13:24.469 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:34 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:13:24.567 INFO:teuthology.orchestra.run.smithi061.stdout:Package libcephfs2-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:13:24.621 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:13:24.622 INFO:teuthology.orchestra.run.smithi061.stdout:Nothing to do. 2023-12-06T18:13:24.622 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:13:24.687 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install libcephfs-devel 2023-12-06T18:13:25.040 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:37 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:13:25.136 INFO:teuthology.orchestra.run.smithi152.stdout:Package python3-rgw-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:13:25.186 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:13:25.187 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2023-12-06T18:13:25.187 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:13:25.239 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install python3-cephfs 2023-12-06T18:13:25.306 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:35 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:13:25.457 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:13:25.458 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:25.458 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:13:25.458 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:25.458 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:13:25.458 INFO:teuthology.orchestra.run.smithi061.stdout: libcephfs-devel x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 31 k 2023-12-06T18:13:25.458 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:25.458 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:13:25.458 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:25.458 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:13:25.458 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:25.459 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 31 k 2023-12-06T18:13:25.459 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 139 k 2023-12-06T18:13:25.459 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:13:25.723 INFO:teuthology.orchestra.run.smithi061.stdout:libcephfs-devel-18.0.0-7644.g96fa2578.el9.x86_6 118 kB/s | 31 kB 00:00 2023-12-06T18:13:25.723 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:25.724 INFO:teuthology.orchestra.run.smithi061.stdout:Total 117 kB/s | 31 kB 00:00 2023-12-06T18:13:25.724 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:13:25.727 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:13:25.727 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:13:25.756 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:13:25.756 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:13:25.833 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:37 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:13:25.846 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:13:25.927 INFO:teuthology.orchestra.run.smithi152.stdout:Package python3-cephfs-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:13:25.979 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:13:25.980 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2023-12-06T18:13:25.980 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:13:25.982 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libcephfs-devel-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:26.037 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install python3-rbd 2023-12-06T18:13:26.360 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:26.618 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:38 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:13:26.642 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libcephfs-devel-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:26.642 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:26.643 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:13:26.643 INFO:teuthology.orchestra.run.smithi061.stdout: libcephfs-devel-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:13:26.643 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:26.643 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:13:26.714 INFO:teuthology.orchestra.run.smithi152.stdout:Package python3-rbd-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:13:26.764 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:13:26.765 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2023-12-06T18:13:26.765 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:13:26.798 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install librados2 2023-12-06T18:13:26.819 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install rbd-fuse 2023-12-06T18:13:27.406 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:39 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:13:27.408 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:37 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:13:27.507 INFO:teuthology.orchestra.run.smithi061.stdout:Package librados2-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:13:27.555 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:13:27.557 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:27.557 INFO:teuthology.orchestra.run.smithi152.stdout: Package Architecture Version Repository Size 2023-12-06T18:13:27.557 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:27.557 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:13:27.557 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 85 k 2023-12-06T18:13:27.557 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:27.558 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:13:27.558 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:27.558 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:13:27.558 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:27.558 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 85 k 2023-12-06T18:13:27.558 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 231 k 2023-12-06T18:13:27.558 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:13:27.561 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:13:27.562 INFO:teuthology.orchestra.run.smithi061.stdout:Nothing to do. 2023-12-06T18:13:27.563 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:13:27.626 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install librbd1 2023-12-06T18:13:28.114 INFO:teuthology.orchestra.run.smithi152.stdout:rbd-fuse-18.0.0-7644.g96fa2578.el9.x86_64.rpm 153 kB/s | 85 kB 00:00 2023-12-06T18:13:28.115 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:28.115 INFO:teuthology.orchestra.run.smithi152.stdout:Total 153 kB/s | 85 kB 00:00 2023-12-06T18:13:28.115 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:13:28.122 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:13:28.122 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:13:28.178 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:13:28.178 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:13:28.240 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:38 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:13:28.336 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:13:28.339 INFO:teuthology.orchestra.run.smithi061.stdout:Package librbd1-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:13:28.394 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:13:28.395 INFO:teuthology.orchestra.run.smithi061.stdout:Nothing to do. 2023-12-06T18:13:28.395 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:13:28.438 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-fuse-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:28.458 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install python3-rados 2023-12-06T18:13:28.832 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-fuse-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:29.071 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:39 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:13:29.163 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-fuse-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:29.163 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:29.164 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:13:29.164 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:13:29.164 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:29.164 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:13:29.169 INFO:teuthology.orchestra.run.smithi061.stdout:Package python3-rados-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:13:29.223 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:13:29.224 INFO:teuthology.orchestra.run.smithi061.stdout:Nothing to do. 2023-12-06T18:13:29.224 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:13:29.284 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install python3-rgw 2023-12-06T18:13:29.389 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install rbd-mirror 2023-12-06T18:13:29.900 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:39 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:13:29.980 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:41 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:13:29.997 INFO:teuthology.orchestra.run.smithi061.stdout:Package python3-rgw-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:13:30.050 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:13:30.051 INFO:teuthology.orchestra.run.smithi061.stdout:Nothing to do. 2023-12-06T18:13:30.051 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:13:30.119 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install python3-cephfs 2023-12-06T18:13:30.138 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:13:30.139 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:30.139 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:13:30.139 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:30.139 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:13:30.139 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 3.0 M 2023-12-06T18:13:30.139 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:30.139 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:13:30.139 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:30.139 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:13:30.139 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:30.139 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 3.0 M 2023-12-06T18:13:30.139 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 12 M 2023-12-06T18:13:30.139 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:13:30.740 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:40 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:13:30.837 INFO:teuthology.orchestra.run.smithi061.stdout:Package python3-cephfs-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:13:30.893 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:13:30.894 INFO:teuthology.orchestra.run.smithi061.stdout:Nothing to do. 2023-12-06T18:13:30.894 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:13:30.899 INFO:teuthology.orchestra.run.smithi152.stdout:rbd-mirror-18.0.0-7644.g96fa2578.el9.x86_64.rpm 4.0 MB/s | 3.0 MB 00:00 2023-12-06T18:13:30.900 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:30.900 INFO:teuthology.orchestra.run.smithi152.stdout:Total 3.9 MB/s | 3.0 MB 00:00 2023-12-06T18:13:30.900 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:13:30.910 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:13:30.910 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:13:30.955 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install python3-rbd 2023-12-06T18:13:30.983 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:13:30.983 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:13:31.392 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:13:31.458 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-mirror-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:31.482 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:31.482 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-06T18:13:31.482 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-12-06T18:13:31.482 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-06T18:13:31.482 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-06T18:13:31.482 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:31.571 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:41 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:13:31.668 INFO:teuthology.orchestra.run.smithi061.stdout:Package python3-rbd-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:13:31.721 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:13:31.722 INFO:teuthology.orchestra.run.smithi061.stdout:Nothing to do. 2023-12-06T18:13:31.722 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:13:31.784 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install rbd-fuse 2023-12-06T18:13:32.399 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:42 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:13:32.552 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:13:32.552 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:32.552 INFO:teuthology.orchestra.run.smithi061.stdout: Package Architecture Version Repository Size 2023-12-06T18:13:32.552 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:32.553 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:13:32.553 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-fuse x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 85 k 2023-12-06T18:13:32.553 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:32.553 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:13:32.553 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:32.553 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:13:32.553 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:32.553 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 85 k 2023-12-06T18:13:32.553 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 231 k 2023-12-06T18:13:32.553 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:13:32.648 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-mirror-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:32.648 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:32.648 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:13:32.648 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:13:32.648 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:32.648 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:13:32.884 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install rbd-nbd 2023-12-06T18:13:32.892 INFO:teuthology.orchestra.run.smithi061.stdout:rbd-fuse-18.0.0-7644.g96fa2578.el9.x86_64.rpm 252 kB/s | 85 kB 00:00 2023-12-06T18:13:32.892 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:32.893 INFO:teuthology.orchestra.run.smithi061.stdout:Total 251 kB/s | 85 kB 00:00 2023-12-06T18:13:32.893 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:13:32.900 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:13:32.900 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:13:32.958 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:13:32.959 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:13:33.107 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:13:33.201 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : rbd-fuse-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:33.485 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:45 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:13:33.583 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: rbd-fuse-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:33.632 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:13:33.632 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:33.632 INFO:teuthology.orchestra.run.smithi152.stdout: Package Architecture Version Repository Size 2023-12-06T18:13:33.632 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:33.633 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:13:33.633 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 171 k 2023-12-06T18:13:33.633 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:33.633 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:13:33.633 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:33.633 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:13:33.633 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:33.633 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 171 k 2023-12-06T18:13:33.634 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 494 k 2023-12-06T18:13:33.634 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:13:33.885 INFO:teuthology.orchestra.run.smithi152.stdout:rbd-nbd-18.0.0-7644.g96fa2578.el9.x86_64.rpm 679 kB/s | 171 kB 00:00 2023-12-06T18:13:33.886 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:33.886 INFO:teuthology.orchestra.run.smithi152.stdout:Total 675 kB/s | 171 kB 00:00 2023-12-06T18:13:33.886 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:13:33.894 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:13:33.894 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:13:33.949 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:13:33.950 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:13:33.967 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : rbd-fuse-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:33.967 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:33.967 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:13:33.967 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-fuse-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:13:33.967 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:33.967 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:13:34.095 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:13:34.186 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install rbd-mirror 2023-12-06T18:13:34.208 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-nbd-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:34.648 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-nbd-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:34.798 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:44 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:13:34.949 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:13:34.950 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:34.950 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:13:34.950 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:34.950 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:13:34.950 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-mirror x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 3.0 M 2023-12-06T18:13:34.950 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:34.950 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:13:34.950 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:34.950 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:13:34.950 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:34.950 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 3.0 M 2023-12-06T18:13:34.951 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 12 M 2023-12-06T18:13:34.951 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:13:34.992 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-nbd-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:34.992 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:34.993 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:13:34.993 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:13:34.993 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:34.993 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:13:35.195 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-base-debuginfo 2023-12-06T18:13:35.344 INFO:teuthology.orchestra.run.smithi061.stdout:rbd-mirror-18.0.0-7644.g96fa2578.el9.x86_64.rpm 7.6 MB/s | 3.0 MB 00:00 2023-12-06T18:13:35.345 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:35.345 INFO:teuthology.orchestra.run.smithi061.stdout:Total 7.6 MB/s | 3.0 MB 00:00 2023-12-06T18:13:35.345 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:13:35.355 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:13:35.355 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:13:35.431 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:13:35.431 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:13:35.807 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:01:47 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:13:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:13:35.968 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:13:35.969 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:35.969 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:13:35.969 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:35.969 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:13:35.969 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 73 M 2023-12-06T18:13:35.970 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2023-12-06T18:13:35.970 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 21 M 2023-12-06T18:13:35.970 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:35.970 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:13:35.970 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:13:35.970 INFO:teuthology.orchestra.run.smithi152.stdout:Install 2 Packages 2023-12-06T18:13:35.970 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:35.970 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 94 M 2023-12-06T18:13:35.970 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 508 M 2023-12-06T18:13:35.970 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:13:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : rbd-mirror-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:36.054 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:36.054 INFO:teuthology.orchestra.run.smithi061.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-06T18:13:36.055 INFO:teuthology.orchestra.run.smithi061.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-12-06T18:13:36.055 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-06T18:13:36.055 INFO:teuthology.orchestra.run.smithi061.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-06T18:13:36.055 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:37.188 INFO:teuthology.orchestra.run.smithi152.stdout:(1/2): ceph-debuginfo-18.0.0-7644.g96fa2578.el9 17 MB/s | 21 MB 00:01 2023-12-06T18:13:37.214 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : rbd-mirror-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:37.214 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:37.214 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:13:37.214 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-mirror-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:13:37.214 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:37.214 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:13:37.413 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install rbd-nbd 2023-12-06T18:13:38.036 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:48 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:13:38.189 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:13:38.189 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:38.189 INFO:teuthology.orchestra.run.smithi061.stdout: Package Architecture Version Repository Size 2023-12-06T18:13:38.190 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:38.190 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:13:38.190 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-nbd x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 171 k 2023-12-06T18:13:38.190 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:38.190 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:13:38.190 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:38.190 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:13:38.190 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:38.190 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 171 k 2023-12-06T18:13:38.190 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 494 k 2023-12-06T18:13:38.190 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:13:38.539 INFO:teuthology.orchestra.run.smithi061.stdout:rbd-nbd-18.0.0-7644.g96fa2578.el9.x86_64.rpm 491 kB/s | 171 kB 00:00 2023-12-06T18:13:38.539 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:38.540 INFO:teuthology.orchestra.run.smithi061.stdout:Total 488 kB/s | 171 kB 00:00 2023-12-06T18:13:38.540 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:13:38.548 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:13:38.548 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:13:38.607 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:13:38.607 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:13:38.762 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:13:38.857 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : rbd-nbd-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:39.378 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: rbd-nbd-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:39.671 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : rbd-nbd-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:13:39.671 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:39.672 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:13:39.672 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-nbd-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:13:39.672 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:39.672 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:13:39.908 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-base-debuginfo 2023-12-06T18:13:40.536 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:01:50 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:13:40.691 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:13:40.691 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:40.691 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:13:40.692 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:40.692 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:13:40.692 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-base-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 73 M 2023-12-06T18:13:40.692 INFO:teuthology.orchestra.run.smithi061.stdout:Installing dependencies: 2023-12-06T18:13:40.692 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 21 M 2023-12-06T18:13:40.692 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:40.692 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:13:40.692 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:13:40.692 INFO:teuthology.orchestra.run.smithi061.stdout:Install 2 Packages 2023-12-06T18:13:40.692 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:13:40.693 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 94 M 2023-12-06T18:13:40.693 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 508 M 2023-12-06T18:13:40.693 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:13:42.333 INFO:teuthology.orchestra.run.smithi061.stdout:(1/2): ceph-debuginfo-18.0.0-7644.g96fa2578.el9 13 MB/s | 21 MB 00:01 2023-12-06T18:13:44.796 INFO:teuthology.orchestra.run.smithi152.stdout:(2/2): ceph-base-debuginfo-18.0.0-7644.g96fa257 8.3 MB/s | 73 MB 00:08 2023-12-06T18:13:44.798 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:44.798 INFO:teuthology.orchestra.run.smithi152.stdout:Total 11 MB/s | 94 MB 00:08 2023-12-06T18:13:44.798 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:13:44.803 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:13:44.803 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:13:45.412 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:13:45.412 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:13:46.383 INFO:teuthology.orchestra.run.smithi061.stdout:(2/2): ceph-base-debuginfo-18.0.0-7644.g96fa257 13 MB/s | 73 MB 00:05 2023-12-06T18:13:46.385 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:13:46.385 INFO:teuthology.orchestra.run.smithi061.stdout:Total 16 MB/s | 94 MB 00:05 2023-12-06T18:13:46.386 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:13:46.391 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:13:46.391 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:13:47.019 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:13:47.020 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:13:48.374 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:13:50.103 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:13:55.984 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 1/2 2023-12-06T18:13:56.062 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-base-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 2/2 2023-12-06T18:13:58.112 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 1/2 2023-12-06T18:13:58.186 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-base-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 2/2 2023-12-06T18:13:59.613 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-base-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 2/2 2023-12-06T18:13:59.614 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-base-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/2 2023-12-06T18:13:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2/2 2023-12-06T18:13:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:59.920 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:13:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:13:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:13:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:13:59.921 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:14:00.139 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-common-debuginfo 2023-12-06T18:14:00.730 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:02:12 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:14:00.876 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:14:00.876 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:14:00.877 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:14:00.877 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:14:00.877 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:14:00.877 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 404 M 2023-12-06T18:14:00.877 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:14:00.877 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:14:00.877 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:14:00.877 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:14:00.877 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:14:00.877 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 404 M 2023-12-06T18:14:00.877 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 1.6 G 2023-12-06T18:14:00.877 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:14:01.616 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-base-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 2/2 2023-12-06T18:14:01.616 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-base-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/2 2023-12-06T18:14:01.912 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2/2 2023-12-06T18:14:01.912 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:14:01.912 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:14:01.912 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-base-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:14:01.912 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:14:01.912 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:14:01.912 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:14:02.120 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-common-debuginfo 2023-12-06T18:14:02.732 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:02:12 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:14:02.884 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:14:02.885 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:14:02.885 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:14:02.885 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:14:02.885 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:14:02.885 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-common-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 404 M 2023-12-06T18:14:02.885 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:14:02.886 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:14:02.886 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:14:02.886 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:14:02.886 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:14:02.886 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 404 M 2023-12-06T18:14:02.886 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 1.6 G 2023-12-06T18:14:02.886 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:14:28.380 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-common-debuginfo-18.0.0-7644.g96fa2578.el9 15 MB/s | 404 MB 00:27 2023-12-06T18:14:28.381 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:14:28.381 INFO:teuthology.orchestra.run.smithi152.stdout:Total 15 MB/s | 404 MB 00:27 2023-12-06T18:14:28.381 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:14:28.385 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:14:28.386 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:14:30.794 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:14:30.794 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:14:35.418 INFO:teuthology.orchestra.run.smithi061.stdout:ceph-common-debuginfo-18.0.0-7644.g96fa2578.el9 12 MB/s | 404 MB 00:32 2023-12-06T18:14:35.419 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:14:35.419 INFO:teuthology.orchestra.run.smithi061.stdout:Total 12 MB/s | 404 MB 00:32 2023-12-06T18:14:35.420 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:14:35.424 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:14:35.424 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:14:37.947 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:14:37.948 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:15:11.531 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:15:11.656 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-common-debuginfo-2:18.0.0-7644.g96fa2578.el9.x8 1/1 2023-12-06T18:15:20.778 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-debuginfo-2:18.0.0-7644.g96fa2578.el9.x8 1/1 2023-12-06T18:15:21.074 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-common-debuginfo-2:18.0.0-7644.g96fa2578.el9.x8 1/1 2023-12-06T18:15:21.074 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:15:21.074 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:15:21.074 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:15:21.074 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:15:21.074 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:15:21.124 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:15:21.216 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-common-debuginfo-2:18.0.0-7644.g96fa2578.el9.x8 1/1 2023-12-06T18:15:21.287 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2023-12-06T18:15:21.873 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:03:33 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:15:22.017 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:15:22.018 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================== 2023-12-06T18:15:22.018 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:15:22.018 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================== 2023-12-06T18:15:22.018 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:15:22.018 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 1.4 M 2023-12-06T18:15:22.018 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:15:22.019 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:15:22.019 INFO:teuthology.orchestra.run.smithi152.stdout:======================================================================================== 2023-12-06T18:15:22.019 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:15:22.019 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:15:22.019 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 1.4 M 2023-12-06T18:15:22.019 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 6.2 M 2023-12-06T18:15:22.019 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:15:22.344 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-immutable-object-cache-debuginfo-18.0.0-76 4.3 MB/s | 1.4 MB 00:00 2023-12-06T18:15:22.345 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:15:22.345 INFO:teuthology.orchestra.run.smithi152.stdout:Total 4.3 MB/s | 1.4 MB 00:00 2023-12-06T18:15:22.345 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:15:22.349 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:15:22.349 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:15:22.390 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:15:22.390 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:15:22.640 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:15:22.750 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:18.0.0-7644. 1/1 2023-12-06T18:15:23.135 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:18.0.0-7644. 1/1 2023-12-06T18:15:23.430 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:18.0.0-7644. 1/1 2023-12-06T18:15:23.431 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:15:23.431 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:15:23.431 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:15:23.431 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:15:23.431 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:15:23.615 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-radosgw-debuginfo 2023-12-06T18:15:24.194 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:03:36 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:15:24.340 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:15:24.341 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:15:24.341 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:15:24.341 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:15:24.341 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:15:24.341 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 172 M 2023-12-06T18:15:24.341 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:15:24.341 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:15:24.341 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:15:24.341 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:15:24.341 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:15:24.342 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 172 M 2023-12-06T18:15:24.342 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 730 M 2023-12-06T18:15:24.342 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:15:28.723 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-common-debuginfo-2:18.0.0-7644.g96fa2578.el9.x8 1/1 2023-12-06T18:15:29.008 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-common-debuginfo-2:18.0.0-7644.g96fa2578.el9.x8 1/1 2023-12-06T18:15:29.008 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:15:29.008 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:15:29.008 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-common-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:15:29.008 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:15:29.008 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:15:29.231 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2023-12-06T18:15:29.846 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:03:39 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:15:30.001 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:15:30.002 INFO:teuthology.orchestra.run.smithi061.stdout:======================================================================================== 2023-12-06T18:15:30.002 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:15:30.002 INFO:teuthology.orchestra.run.smithi061.stdout:======================================================================================== 2023-12-06T18:15:30.002 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:15:30.002 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 1.4 M 2023-12-06T18:15:30.002 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:15:30.002 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:15:30.002 INFO:teuthology.orchestra.run.smithi061.stdout:======================================================================================== 2023-12-06T18:15:30.002 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:15:30.003 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:15:30.003 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 1.4 M 2023-12-06T18:15:30.003 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 6.2 M 2023-12-06T18:15:30.003 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:15:31.518 INFO:teuthology.orchestra.run.smithi061.stdout:ceph-immutable-object-cache-debuginfo-18.0.0-76 941 kB/s | 1.4 MB 00:01 2023-12-06T18:15:31.518 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:15:31.519 INFO:teuthology.orchestra.run.smithi061.stdout:Total 940 kB/s | 1.4 MB 00:01 2023-12-06T18:15:31.519 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:15:31.523 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:15:31.523 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:15:31.567 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:15:31.567 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:15:31.823 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:15:31.935 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:18.0.0-7644. 1/1 2023-12-06T18:15:32.398 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:18.0.0-7644. 1/1 2023-12-06T18:15:32.658 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:18.0.0-7644. 1/1 2023-12-06T18:15:32.659 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:15:32.659 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:15:32.659 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-immutable-object-cache-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:15:32.659 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:15:32.659 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:15:32.755 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-radosgw-debuginfo-18.0.0-7644.g96fa2578.el 20 MB/s | 172 MB 00:08 2023-12-06T18:15:32.755 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:15:32.756 INFO:teuthology.orchestra.run.smithi152.stdout:Total 20 MB/s | 172 MB 00:08 2023-12-06T18:15:32.756 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:15:32.760 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:15:32.760 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:15:32.815 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-radosgw-debuginfo 2023-12-06T18:15:33.419 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:03:43 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:15:33.572 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:15:33.573 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:15:33.573 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:15:33.573 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:15:33.573 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:15:33.573 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-radosgw-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 172 M 2023-12-06T18:15:33.573 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:15:33.573 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:15:33.573 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:15:33.573 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:15:33.574 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:15:33.574 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 172 M 2023-12-06T18:15:33.574 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 730 M 2023-12-06T18:15:33.574 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:15:33.802 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:15:33.803 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:15:41.503 INFO:teuthology.orchestra.run.smithi061.stdout:ceph-radosgw-debuginfo-18.0.0-7644.g96fa2578.el 22 MB/s | 172 MB 00:07 2023-12-06T18:15:41.504 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:15:41.504 INFO:teuthology.orchestra.run.smithi061.stdout:Total 22 MB/s | 172 MB 00:07 2023-12-06T18:15:41.505 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:15:41.509 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:15:41.509 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:15:42.601 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:15:42.601 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:15:51.499 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:15:51.602 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-radosgw-debuginfo-2:18.0.0-7644.g96fa2578.el9.x 1/1 2023-12-06T18:15:56.284 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:18.0.0-7644.g96fa2578.el9.x 1/1 2023-12-06T18:15:56.602 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-radosgw-debuginfo-2:18.0.0-7644.g96fa2578.el9.x 1/1 2023-12-06T18:15:56.602 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:15:56.602 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:15:56.602 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:15:56.603 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:15:56.603 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:15:56.810 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-test-debuginfo 2023-12-06T18:15:57.391 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:04:09 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:15:57.536 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:15:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:15:57.537 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:15:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:15:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:15:57.538 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 622 M 2023-12-06T18:15:57.538 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:15:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:15:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:15:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:15:57.538 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:15:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 622 M 2023-12-06T18:15:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 2.8 G 2023-12-06T18:15:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:16:01.308 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:16:01.395 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-radosgw-debuginfo-2:18.0.0-7644.g96fa2578.el9.x 1/1 2023-12-06T18:16:05.857 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:18.0.0-7644.g96fa2578.el9.x 1/1 2023-12-06T18:16:06.145 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-radosgw-debuginfo-2:18.0.0-7644.g96fa2578.el9.x 1/1 2023-12-06T18:16:06.145 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:16:06.146 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:16:06.146 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-radosgw-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:16:06.146 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:16:06.146 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:16:06.343 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-test-debuginfo 2023-12-06T18:16:06.952 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:04:16 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:16:07.104 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:16:07.104 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:16:07.104 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:16:07.105 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:16:07.105 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:16:07.105 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-test-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 622 M 2023-12-06T18:16:07.105 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:16:07.105 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:16:07.105 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:16:07.105 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:16:07.105 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:16:07.106 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 622 M 2023-12-06T18:16:07.106 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 2.8 G 2023-12-06T18:16:07.106 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:16:36.258 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-test-debuginfo-18.0.0-7644.g96fa2578.el9.x 16 MB/s | 622 MB 00:38 2023-12-06T18:16:36.259 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:16:36.259 INFO:teuthology.orchestra.run.smithi152.stdout:Total 16 MB/s | 622 MB 00:38 2023-12-06T18:16:36.259 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:16:36.264 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:16:36.264 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:16:39.954 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:16:39.954 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:17:45.217 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:17:45.373 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-test-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/1 2023-12-06T18:17:54.916 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-test-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/1 2023-12-06T18:17:55.226 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-test-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/1 2023-12-06T18:17:55.226 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:17:55.226 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:17:55.226 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:17:55.226 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:17:55.227 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:17:55.531 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-base-debuginfo 2023-12-06T18:17:56.119 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:06:08 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:17:56.214 INFO:teuthology.orchestra.run.smithi152.stdout:Package ceph-base-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:17:56.264 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:17:56.265 INFO:teuthology.orchestra.run.smithi152.stdout:Nothing to do. 2023-12-06T18:17:56.265 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:17:56.321 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-mgr-debuginfo 2023-12-06T18:17:56.910 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:06:08 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:17:57.056 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:17:57.057 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:17:57.057 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:17:57.057 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:17:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:17:57.058 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 19 M 2023-12-06T18:17:57.058 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:17:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:17:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:17:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:17:57.058 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:17:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 19 M 2023-12-06T18:17:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 100 M 2023-12-06T18:17:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:17:58.159 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-mgr-debuginfo-18.0.0-7644.g96fa2578.el9.x8 17 MB/s | 19 MB 00:01 2023-12-06T18:17:58.160 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:17:58.161 INFO:teuthology.orchestra.run.smithi152.stdout:Total 17 MB/s | 19 MB 00:01 2023-12-06T18:17:58.161 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:17:58.164 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:17:58.164 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:17:58.308 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:17:58.308 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:18:00.483 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:18:00.616 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:18:01.630 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:18:01.923 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:18:01.923 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:18:01.923 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:18:01.924 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:18:01.924 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:18:01.924 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:18:02.101 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-mds-debuginfo 2023-12-06T18:18:02.690 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:06:14 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:18:02.835 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:18:02.836 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:18:02.836 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:18:02.836 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:18:02.836 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:18:02.836 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 47 M 2023-12-06T18:18:02.836 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:18:02.836 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:18:02.836 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:18:02.836 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:18:02.836 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:18:02.836 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 47 M 2023-12-06T18:18:02.837 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 168 M 2023-12-06T18:18:02.837 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:18:05.936 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-mds-debuginfo-18.0.0-7644.g96fa2578.el9.x8 15 MB/s | 47 MB 00:03 2023-12-06T18:18:05.937 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:18:05.937 INFO:teuthology.orchestra.run.smithi152.stdout:Total 15 MB/s | 47 MB 00:03 2023-12-06T18:18:05.938 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:18:05.942 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:18:05.942 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:18:06.248 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:18:06.249 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:18:10.832 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:18:10.941 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mds-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:18:12.251 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mds-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:18:12.547 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mds-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:18:12.547 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:18:12.547 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:18:12.547 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:18:12.547 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:18:12.547 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:18:12.727 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-mon-debuginfo 2023-12-06T18:18:13.306 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:06:25 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:18:13.452 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:18:13.452 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:18:13.453 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:18:13.453 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:18:13.453 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:18:13.453 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 111 M 2023-12-06T18:18:13.453 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:18:13.453 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:18:13.453 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:18:13.453 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:18:13.453 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:18:13.453 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 111 M 2023-12-06T18:18:13.453 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 507 M 2023-12-06T18:18:13.453 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:18:20.254 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-mon-debuginfo-18.0.0-7644.g96fa2578.el9.x8 16 MB/s | 111 MB 00:06 2023-12-06T18:18:20.255 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:18:20.255 INFO:teuthology.orchestra.run.smithi152.stdout:Total 16 MB/s | 111 MB 00:06 2023-12-06T18:18:20.255 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:18:20.259 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:18:20.260 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:18:20.942 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:18:20.943 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:18:32.556 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:18:32.658 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mon-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:18:35.975 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mon-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:18:36.274 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mon-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:18:36.274 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:18:36.274 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:18:36.274 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:18:36.274 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:18:36.274 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:18:36.465 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-osd-debuginfo 2023-12-06T18:18:37.055 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:06:49 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:18:37.202 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:18:37.203 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:18:37.203 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:18:37.203 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:18:37.203 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:18:37.203 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 311 M 2023-12-06T18:18:37.203 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:18:37.203 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:18:37.203 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:18:37.203 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:18:37.203 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:18:37.204 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 311 M 2023-12-06T18:18:37.204 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 1.3 G 2023-12-06T18:18:37.204 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:18:56.961 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-osd-debuginfo-18.0.0-7644.g96fa2578.el9.x8 16 MB/s | 311 MB 00:19 2023-12-06T18:18:56.962 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:18:56.962 INFO:teuthology.orchestra.run.smithi152.stdout:Total 16 MB/s | 311 MB 00:19 2023-12-06T18:18:56.962 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:18:56.966 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:18:56.966 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:18:58.820 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:18:58.820 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:19:30.729 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:19:30.889 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-osd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:19:39.271 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-osd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:19:39.598 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-osd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:19:39.598 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:19:39.598 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:19:39.598 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:19:39.598 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:19:39.598 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:19:39.804 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-fuse-debuginfo 2023-12-06T18:19:40.385 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:07:52 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:19:40.531 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:19:40.531 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:19:40.531 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:19:40.532 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:19:40.532 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:19:40.532 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 5.8 M 2023-12-06T18:19:40.532 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:19:40.532 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:19:40.532 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:19:40.532 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:19:40.532 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:19:40.532 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 5.8 M 2023-12-06T18:19:40.532 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 28 M 2023-12-06T18:19:40.532 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:19:41.517 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-fuse-debuginfo-18.0.0-7644.g96fa2578.el9.x 5.9 MB/s | 5.8 MB 00:00 2023-12-06T18:19:41.517 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:19:41.518 INFO:teuthology.orchestra.run.smithi152.stdout:Total 5.8 MB/s | 5.8 MB 00:00 2023-12-06T18:19:41.518 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:19:41.521 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:19:41.522 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:19:41.588 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:19:41.588 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:19:42.304 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:19:42.416 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-fuse-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/1 2023-12-06T18:19:43.037 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-fuse-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/1 2023-12-06T18:19:43.341 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-fuse-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/1 2023-12-06T18:19:43.341 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:19:43.341 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:19:43.341 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:19:43.341 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:19:43.341 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:19:43.524 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install librados-devel-debuginfo 2023-12-06T18:19:44.111 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:07:56 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:19:44.258 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:19:44.258 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:19:44.258 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:19:44.259 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:19:44.259 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:19:44.259 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 108 k 2023-12-06T18:19:44.259 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:19:44.259 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:19:44.259 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:19:44.259 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:19:44.259 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:19:44.259 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 108 k 2023-12-06T18:19:44.259 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 440 k 2023-12-06T18:19:44.259 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:19:44.811 INFO:teuthology.orchestra.run.smithi152.stdout:librados-devel-debuginfo-18.0.0-7644.g96fa2578. 196 kB/s | 108 kB 00:00 2023-12-06T18:19:44.812 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:19:44.812 INFO:teuthology.orchestra.run.smithi152.stdout:Total 196 kB/s | 108 kB 00:00 2023-12-06T18:19:44.812 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:19:44.815 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:19:44.816 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:19:44.850 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:19:44.851 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:19:44.965 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:19:45.067 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librados-devel-debuginfo-2:18.0.0-7644.g96fa2578.el9 1/1 2023-12-06T18:19:45.452 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados-devel-debuginfo-2:18.0.0-7644.g96fa2578.el9 1/1 2023-12-06T18:19:45.748 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados-devel-debuginfo-2:18.0.0-7644.g96fa2578.el9 1/1 2023-12-06T18:19:45.749 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:19:45.749 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:19:45.749 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:19:45.749 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:19:45.749 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:19:45.912 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install libcephfs2-debuginfo 2023-12-06T18:19:46.494 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:07:58 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:19:46.638 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:19:46.639 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:19:46.639 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:19:46.639 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:19:46.640 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:19:46.640 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 5.1 M 2023-12-06T18:19:46.640 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:19:46.640 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:19:46.640 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:19:46.640 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:19:46.640 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:19:46.640 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 5.1 M 2023-12-06T18:19:46.640 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 24 M 2023-12-06T18:19:46.641 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:19:48.522 INFO:teuthology.orchestra.run.smithi152.stdout:libcephfs2-debuginfo-18.0.0-7644.g96fa2578.el9. 2.7 MB/s | 5.1 MB 00:01 2023-12-06T18:19:48.523 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:19:48.523 INFO:teuthology.orchestra.run.smithi152.stdout:Total 2.7 MB/s | 5.1 MB 00:01 2023-12-06T18:19:48.523 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:19:48.527 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:19:48.527 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:19:48.592 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:19:48.592 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:19:49.239 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:19:49.337 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs2-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86 1/1 2023-12-06T18:19:49.896 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephfs2-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86 1/1 2023-12-06T18:19:50.205 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs2-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86 1/1 2023-12-06T18:19:50.205 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:19:50.206 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:19:50.206 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:19:50.206 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:19:50.206 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:19:50.387 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install librados2-debuginfo 2023-12-06T18:19:50.967 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:08:02 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:19:51.113 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:19:51.114 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:19:51.114 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:19:51.114 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:19:51.114 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:19:51.114 INFO:teuthology.orchestra.run.smithi152.stdout: librados2-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 43 M 2023-12-06T18:19:51.114 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:19:51.115 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:19:51.115 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:19:51.115 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:19:51.115 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:19:51.115 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 43 M 2023-12-06T18:19:51.115 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 248 M 2023-12-06T18:19:51.115 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:19:53.783 INFO:teuthology.orchestra.run.smithi152.stdout:librados2-debuginfo-18.0.0-7644.g96fa2578.el9.x 16 MB/s | 43 MB 00:02 2023-12-06T18:19:53.784 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:19:53.784 INFO:teuthology.orchestra.run.smithi152.stdout:Total 16 MB/s | 43 MB 00:02 2023-12-06T18:19:53.785 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:19:53.788 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:19:53.788 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:19:54.085 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:19:54.085 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:19:59.053 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:19:59.156 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librados2-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/1 2023-12-06T18:20:01.042 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/1 2023-12-06T18:20:01.335 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/1 2023-12-06T18:20:01.336 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:01.336 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:20:01.336 INFO:teuthology.orchestra.run.smithi152.stdout: librados2-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:20:01.336 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:01.336 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:20:01.521 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install librbd1-debuginfo 2023-12-06T18:20:02.112 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:08:14 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:20:02.256 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:20:02.257 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:02.257 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:20:02.257 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:02.257 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:20:02.257 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 50 M 2023-12-06T18:20:02.258 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:02.258 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:20:02.258 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:02.258 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:20:02.258 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:02.258 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 50 M 2023-12-06T18:20:02.258 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 294 M 2023-12-06T18:20:02.258 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:20:08.226 INFO:teuthology.orchestra.run.smithi152.stdout:librbd1-debuginfo-18.0.0-7644.g96fa2578.el9.x86 8.3 MB/s | 50 MB 00:05 2023-12-06T18:20:08.227 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:20:08.228 INFO:teuthology.orchestra.run.smithi152.stdout:Total 8.3 MB/s | 50 MB 00:05 2023-12-06T18:20:08.228 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:20:08.232 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:20:08.232 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:20:08.568 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:20:08.568 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:20:14.393 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:20:14.475 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librbd1-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:20:16.643 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:20:16.956 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:20:16.956 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:16.956 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:20:16.956 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:20:16.957 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:16.957 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:20:17.148 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install python3-cephfs-debuginfo 2023-12-06T18:20:17.731 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:08:29 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:20:17.879 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:20:17.880 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:17.880 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:20:17.880 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:17.881 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:20:17.881 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 615 k 2023-12-06T18:20:17.881 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:17.881 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:20:17.881 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:17.881 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:20:17.881 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:17.881 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 615 k 2023-12-06T18:20:17.881 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 2.1 M 2023-12-06T18:20:17.881 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:20:18.596 INFO:teuthology.orchestra.run.smithi152.stdout:python3-cephfs-debuginfo-18.0.0-7644.g96fa2578. 862 kB/s | 615 kB 00:00 2023-12-06T18:20:18.596 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:20:18.596 INFO:teuthology.orchestra.run.smithi152.stdout:Total 859 kB/s | 615 kB 00:00 2023-12-06T18:20:18.597 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:20:18.600 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:20:18.600 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:20:18.638 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:20:18.638 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:20:18.802 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:20:18.894 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cephfs-debuginfo-2:18.0.0-7644.g96fa2578.el9 1/1 2023-12-06T18:20:19.287 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: python3-cephfs-debuginfo-2:18.0.0-7644.g96fa2578.el9 1/1 2023-12-06T18:20:19.595 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cephfs-debuginfo-2:18.0.0-7644.g96fa2578.el9 1/1 2023-12-06T18:20:19.595 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:19.595 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:20:19.595 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:20:19.595 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:19.595 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:20:19.784 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install python3-rados-debuginfo 2023-12-06T18:20:20.364 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:08:32 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:20:20.510 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:20:20.510 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:20.510 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:20:20.511 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:20.511 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:20:20.511 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 770 k 2023-12-06T18:20:20.511 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:20.511 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:20:20.511 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:20.511 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:20:20.511 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:20.511 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 770 k 2023-12-06T18:20:20.511 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 2.7 M 2023-12-06T18:20:20.511 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:20:21.874 INFO:teuthology.orchestra.run.smithi152.stdout:python3-rados-debuginfo-18.0.0-7644.g96fa2578.e 565 kB/s | 770 kB 00:01 2023-12-06T18:20:21.874 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:20:21.875 INFO:teuthology.orchestra.run.smithi152.stdout:Total 564 kB/s | 770 kB 00:01 2023-12-06T18:20:21.875 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:20:21.879 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:20:21.879 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:20:21.919 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:20:21.920 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:20:22.099 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:20:22.205 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rados-debuginfo-2:18.0.0-7644.g96fa2578.el9. 1/1 2023-12-06T18:20:22.617 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: python3-rados-debuginfo-2:18.0.0-7644.g96fa2578.el9. 1/1 2023-12-06T18:20:22.917 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rados-debuginfo-2:18.0.0-7644.g96fa2578.el9. 1/1 2023-12-06T18:20:22.917 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:22.918 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:20:22.918 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:20:22.918 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:22.918 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:20:23.111 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install python3-rbd-debuginfo 2023-12-06T18:20:23.702 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:08:35 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:20:23.851 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:20:23.852 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:23.852 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:20:23.852 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:23.852 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:20:23.852 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 1.1 M 2023-12-06T18:20:23.852 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:23.852 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:20:23.852 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:23.852 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:20:23.852 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:23.852 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 1.1 M 2023-12-06T18:20:23.852 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 4.3 M 2023-12-06T18:20:23.852 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:20:25.186 INFO:teuthology.orchestra.run.smithi152.stdout:python3-rbd-debuginfo-18.0.0-7644.g96fa2578.el9 865 kB/s | 1.1 MB 00:01 2023-12-06T18:20:25.187 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:20:25.187 INFO:teuthology.orchestra.run.smithi152.stdout:Total 864 kB/s | 1.1 MB 00:01 2023-12-06T18:20:25.187 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:20:25.190 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:20:25.190 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:20:25.234 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:20:25.234 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:20:25.472 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:20:25.567 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rbd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x8 1/1 2023-12-06T18:20:25.976 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: python3-rbd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x8 1/1 2023-12-06T18:20:26.275 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rbd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x8 1/1 2023-12-06T18:20:26.275 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:26.275 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:20:26.275 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:20:26.275 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:26.275 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:20:26.468 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install python3-rgw-debuginfo 2023-12-06T18:20:27.051 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:08:39 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:20:27.197 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:20:27.198 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:27.198 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:20:27.198 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:27.198 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:20:27.198 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 337 k 2023-12-06T18:20:27.198 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:27.198 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:20:27.199 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:27.199 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:20:27.199 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:27.199 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 337 k 2023-12-06T18:20:27.199 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 1.0 M 2023-12-06T18:20:27.199 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:20:27.921 INFO:teuthology.orchestra.run.smithi152.stdout:python3-rgw-debuginfo-18.0.0-7644.g96fa2578.el9 467 kB/s | 337 kB 00:00 2023-12-06T18:20:27.921 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:20:27.922 INFO:teuthology.orchestra.run.smithi152.stdout:Total 466 kB/s | 337 kB 00:00 2023-12-06T18:20:27.922 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:20:27.925 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:20:27.925 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:20:27.962 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:20:27.962 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:20:28.098 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:20:28.203 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rgw-debuginfo-2:18.0.0-7644.g96fa2578.el9.x8 1/1 2023-12-06T18:20:28.576 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: python3-rgw-debuginfo-2:18.0.0-7644.g96fa2578.el9.x8 1/1 2023-12-06T18:20:28.883 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rgw-debuginfo-2:18.0.0-7644.g96fa2578.el9.x8 1/1 2023-12-06T18:20:28.884 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:28.884 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:20:28.884 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:20:28.884 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:28.884 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:20:29.068 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install rbd-fuse-debuginfo 2023-12-06T18:20:29.661 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:08:41 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:20:29.808 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:20:29.809 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:29.809 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:20:29.809 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:29.809 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:20:29.809 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 506 k 2023-12-06T18:20:29.809 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:29.809 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:20:29.809 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:29.809 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:20:29.809 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:29.810 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 506 k 2023-12-06T18:20:29.810 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 2.5 M 2023-12-06T18:20:29.810 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:20:30.165 INFO:teuthology.orchestra.run.smithi152.stdout:rbd-fuse-debuginfo-18.0.0-7644.g96fa2578.el9.x8 1.4 MB/s | 506 kB 00:00 2023-12-06T18:20:30.166 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:20:30.166 INFO:teuthology.orchestra.run.smithi152.stdout:Total 1.4 MB/s | 506 kB 00:00 2023-12-06T18:20:30.166 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:20:30.169 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:20:30.169 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:20:30.206 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:20:30.207 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:20:30.370 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:20:30.481 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-fuse-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:20:30.872 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-fuse-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:20:31.172 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-fuse-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:20:31.173 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:31.173 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:20:31.173 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:20:31.173 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:31.173 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:20:31.352 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install rbd-mirror-debuginfo 2023-12-06T18:20:31.937 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:08:43 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:20:32.083 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:20:32.084 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:32.084 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:20:32.084 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:32.084 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:20:32.084 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 50 M 2023-12-06T18:20:32.084 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:32.084 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:20:32.084 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:32.084 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:20:32.084 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:32.084 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 50 M 2023-12-06T18:20:32.084 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 284 M 2023-12-06T18:20:32.085 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:20:37.392 INFO:teuthology.orchestra.run.smithi152.stdout:rbd-mirror-debuginfo-18.0.0-7644.g96fa2578.el9. 9.4 MB/s | 50 MB 00:05 2023-12-06T18:20:37.393 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:20:37.393 INFO:teuthology.orchestra.run.smithi152.stdout:Total 9.4 MB/s | 50 MB 00:05 2023-12-06T18:20:37.394 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:20:37.398 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:20:37.398 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:20:37.723 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:20:37.723 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:20:43.447 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:20:43.540 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-mirror-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86 1/1 2023-12-06T18:20:45.580 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-mirror-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86 1/1 2023-12-06T18:20:45.911 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-mirror-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86 1/1 2023-12-06T18:20:45.911 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:45.912 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:20:45.912 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:20:45.912 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:45.912 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:20:46.110 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install rbd-nbd-debuginfo 2023-12-06T18:20:46.703 INFO:teuthology.orchestra.run.smithi152.stdout:Last metadata expiration check: 0:08:58 ago on Wed 06 Dec 2023 06:11:48 PM UTC. 2023-12-06T18:20:46.849 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2023-12-06T18:20:46.850 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:46.850 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repo Size 2023-12-06T18:20:46.850 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:46.851 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2023-12-06T18:20:46.851 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 1.2 M 2023-12-06T18:20:46.851 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:46.851 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2023-12-06T18:20:46.851 INFO:teuthology.orchestra.run.smithi152.stdout:================================================================================ 2023-12-06T18:20:46.851 INFO:teuthology.orchestra.run.smithi152.stdout:Install 1 Package 2023-12-06T18:20:46.851 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:46.851 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 1.2 M 2023-12-06T18:20:46.851 INFO:teuthology.orchestra.run.smithi152.stdout:Installed size: 5.4 M 2023-12-06T18:20:46.851 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2023-12-06T18:20:47.369 INFO:teuthology.orchestra.run.smithi152.stdout:rbd-nbd-debuginfo-18.0.0-7644.g96fa2578.el9.x86 2.4 MB/s | 1.2 MB 00:00 2023-12-06T18:20:47.370 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:20:47.370 INFO:teuthology.orchestra.run.smithi152.stdout:Total 2.4 MB/s | 1.2 MB 00:00 2023-12-06T18:20:47.370 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2023-12-06T18:20:47.374 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2023-12-06T18:20:47.374 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2023-12-06T18:20:47.416 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2023-12-06T18:20:47.416 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2023-12-06T18:20:47.649 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2023-12-06T18:20:47.743 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-nbd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:20:48.195 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-nbd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:20:48.496 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-nbd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:20:48.496 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:48.496 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2023-12-06T18:20:48.496 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:20:48.496 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:20:48.496 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2023-12-06T18:20:48.683 DEBUG:teuthology.parallel:result is None 2023-12-06T18:20:57.457 INFO:teuthology.orchestra.run.smithi061.stdout:ceph-test-debuginfo-18.0.0-7644.g96fa2578.el9.x 2.1 MB/s | 622 MB 04:50 2023-12-06T18:20:57.458 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:20:57.458 INFO:teuthology.orchestra.run.smithi061.stdout:Total 2.1 MB/s | 622 MB 04:50 2023-12-06T18:20:57.459 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:20:57.464 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:20:57.464 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:21:01.339 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:21:01.339 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:22:10.305 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:22:10.861 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-test-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/1 2023-12-06T18:22:19.379 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-test-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/1 2023-12-06T18:22:19.787 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-test-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/1 2023-12-06T18:22:19.787 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:22:19.788 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:22:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-test-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:22:19.788 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:22:19.788 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:22:20.049 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-base-debuginfo 2023-12-06T18:22:20.661 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:10:30 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:22:20.761 INFO:teuthology.orchestra.run.smithi061.stdout:Package ceph-base-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 is already installed. 2023-12-06T18:22:20.814 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:22:20.824 INFO:teuthology.orchestra.run.smithi061.stdout:Nothing to do. 2023-12-06T18:22:20.824 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:22:20.874 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-mgr-debuginfo 2023-12-06T18:22:21.485 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:10:31 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:22:21.638 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:22:21.639 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:22:21.639 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:22:21.639 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:22:21.639 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:22:21.639 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 19 M 2023-12-06T18:22:21.639 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:22:21.639 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:22:21.639 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:22:21.639 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:22:21.639 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:22:21.639 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 19 M 2023-12-06T18:22:21.639 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 100 M 2023-12-06T18:22:21.639 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:22:22.978 INFO:teuthology.orchestra.run.smithi061.stdout:ceph-mgr-debuginfo-18.0.0-7644.g96fa2578.el9.x8 14 MB/s | 19 MB 00:01 2023-12-06T18:22:22.978 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:22:22.979 INFO:teuthology.orchestra.run.smithi061.stdout:Total 14 MB/s | 19 MB 00:01 2023-12-06T18:22:22.979 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:22:22.983 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:22:22.983 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:22:23.137 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:22:23.137 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:22:25.433 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:22:25.529 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mgr-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:22:26.504 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mgr-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:22:26.812 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mgr-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:22:26.812 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:22:26.812 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:22:26.812 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mgr-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:22:26.812 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:22:26.812 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:22:26.977 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-mds-debuginfo 2023-12-06T18:22:27.593 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:10:37 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:22:27.748 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:22:27.749 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:22:27.749 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:22:27.749 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:22:27.749 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:22:27.749 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mds-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 47 M 2023-12-06T18:22:27.749 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:22:27.749 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:22:27.749 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:22:27.749 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:22:27.749 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:22:27.749 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 47 M 2023-12-06T18:22:27.749 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 168 M 2023-12-06T18:22:27.749 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:22:30.346 INFO:teuthology.orchestra.run.smithi061.stdout:ceph-mds-debuginfo-18.0.0-7644.g96fa2578.el9.x8 18 MB/s | 47 MB 00:02 2023-12-06T18:22:30.347 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:22:30.348 INFO:teuthology.orchestra.run.smithi061.stdout:Total 18 MB/s | 47 MB 00:02 2023-12-06T18:22:30.348 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:22:30.352 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:22:30.352 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:22:30.674 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:22:30.675 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:22:35.558 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:22:35.647 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mds-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:22:37.016 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mds-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:22:37.296 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mds-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:22:37.296 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:22:37.296 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:22:37.296 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mds-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:22:37.296 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:22:37.296 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:22:37.473 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-mon-debuginfo 2023-12-06T18:22:38.087 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:10:48 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:22:38.238 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:22:38.239 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:22:38.239 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:22:38.239 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:22:38.239 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:22:38.239 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mon-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 111 M 2023-12-06T18:22:38.239 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:22:38.240 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:22:38.240 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:22:38.240 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:22:38.240 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:22:38.240 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 111 M 2023-12-06T18:22:38.240 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 507 M 2023-12-06T18:22:38.240 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:22:43.382 INFO:teuthology.orchestra.run.smithi061.stdout:ceph-mon-debuginfo-18.0.0-7644.g96fa2578.el9.x8 22 MB/s | 111 MB 00:05 2023-12-06T18:22:43.385 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:22:43.385 INFO:teuthology.orchestra.run.smithi061.stdout:Total 22 MB/s | 111 MB 00:05 2023-12-06T18:22:43.385 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:22:43.387 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:22:43.387 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:22:44.106 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:22:44.107 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:22:56.524 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:22:56.614 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-mon-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:22:59.810 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-mon-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:23:00.084 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-mon-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:23:00.084 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:23:00.084 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:23:00.084 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-mon-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:23:00.084 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:23:00.084 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:23:00.312 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-osd-debuginfo 2023-12-06T18:23:00.919 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:11:10 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:23:01.072 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:23:01.072 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:23:01.073 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:23:01.073 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:23:01.073 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:23:01.073 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-osd-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 311 M 2023-12-06T18:23:01.073 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:23:01.073 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:23:01.073 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:23:01.073 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:23:01.073 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:23:01.073 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 311 M 2023-12-06T18:23:01.073 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 1.3 G 2023-12-06T18:23:01.073 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:23:19.323 INFO:teuthology.orchestra.run.smithi061.stdout:ceph-osd-debuginfo-18.0.0-7644.g96fa2578.el9.x8 17 MB/s | 311 MB 00:18 2023-12-06T18:23:19.324 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:23:19.324 INFO:teuthology.orchestra.run.smithi061.stdout:Total 17 MB/s | 311 MB 00:18 2023-12-06T18:23:19.325 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:23:19.329 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:23:19.329 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:23:21.271 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:23:21.271 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:23:55.025 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:23:55.130 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-osd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:24:03.085 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-osd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:24:03.398 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-osd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:24:03.398 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:03.399 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:24:03.399 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-osd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:24:03.399 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:03.399 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:24:03.629 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install ceph-fuse-debuginfo 2023-12-06T18:24:04.238 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:12:14 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:24:04.391 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:24:04.392 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:04.392 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:24:04.392 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:04.392 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:24:04.392 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-fuse-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 5.8 M 2023-12-06T18:24:04.392 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:04.392 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:24:04.392 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:04.392 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:24:04.392 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:04.393 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 5.8 M 2023-12-06T18:24:04.393 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 28 M 2023-12-06T18:24:04.393 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:24:05.164 INFO:teuthology.orchestra.run.smithi061.stdout:ceph-fuse-debuginfo-18.0.0-7644.g96fa2578.el9.x 7.5 MB/s | 5.8 MB 00:00 2023-12-06T18:24:05.165 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:24:05.165 INFO:teuthology.orchestra.run.smithi061.stdout:Total 7.4 MB/s | 5.8 MB 00:00 2023-12-06T18:24:05.165 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:24:05.169 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:24:05.169 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:24:05.241 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:24:05.242 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:24:05.995 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:24:06.099 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : ceph-fuse-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/1 2023-12-06T18:24:06.695 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: ceph-fuse-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/1 2023-12-06T18:24:06.977 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : ceph-fuse-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/1 2023-12-06T18:24:06.977 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:06.977 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:24:06.977 INFO:teuthology.orchestra.run.smithi061.stdout: ceph-fuse-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:24:06.977 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:06.978 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:24:07.155 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install librados-devel-debuginfo 2023-12-06T18:24:07.768 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:12:17 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:24:07.921 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:24:07.921 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:07.921 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:24:07.921 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:07.922 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:24:07.922 INFO:teuthology.orchestra.run.smithi061.stdout: librados-devel-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 108 k 2023-12-06T18:24:07.922 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:07.922 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:24:07.922 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:07.922 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:24:07.922 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:07.922 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 108 k 2023-12-06T18:24:07.922 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 440 k 2023-12-06T18:24:07.922 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:24:08.150 INFO:teuthology.orchestra.run.smithi061.stdout:librados-devel-debuginfo-18.0.0-7644.g96fa2578. 475 kB/s | 108 kB 00:00 2023-12-06T18:24:08.151 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:24:08.151 INFO:teuthology.orchestra.run.smithi061.stdout:Total 472 kB/s | 108 kB 00:00 2023-12-06T18:24:08.151 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:24:08.155 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:24:08.155 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:24:08.191 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:24:08.191 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:24:08.308 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:24:08.410 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : librados-devel-debuginfo-2:18.0.0-7644.g96fa2578.el9 1/1 2023-12-06T18:24:08.920 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librados-devel-debuginfo-2:18.0.0-7644.g96fa2578.el9 1/1 2023-12-06T18:24:09.201 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librados-devel-debuginfo-2:18.0.0-7644.g96fa2578.el9 1/1 2023-12-06T18:24:09.202 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:09.202 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:24:09.202 INFO:teuthology.orchestra.run.smithi061.stdout: librados-devel-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:24:09.202 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:09.202 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:24:09.515 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install libcephfs2-debuginfo 2023-12-06T18:24:10.127 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:12:20 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:24:10.283 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:24:10.284 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:10.284 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:24:10.284 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:10.284 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:24:10.284 INFO:teuthology.orchestra.run.smithi061.stdout: libcephfs2-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 5.1 M 2023-12-06T18:24:10.284 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:10.284 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:24:10.284 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:10.284 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:24:10.284 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:10.284 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 5.1 M 2023-12-06T18:24:10.284 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 24 M 2023-12-06T18:24:10.284 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:24:11.528 INFO:teuthology.orchestra.run.smithi061.stdout:libcephfs2-debuginfo-18.0.0-7644.g96fa2578.el9. 4.1 MB/s | 5.1 MB 00:01 2023-12-06T18:24:11.529 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:24:11.529 INFO:teuthology.orchestra.run.smithi061.stdout:Total 4.1 MB/s | 5.1 MB 00:01 2023-12-06T18:24:11.529 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:24:11.533 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:24:11.533 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:24:11.601 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:24:11.601 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:24:12.287 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:24:12.396 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : libcephfs2-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86 1/1 2023-12-06T18:24:12.939 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: libcephfs2-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86 1/1 2023-12-06T18:24:13.226 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : libcephfs2-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86 1/1 2023-12-06T18:24:13.226 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:13.226 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:24:13.226 INFO:teuthology.orchestra.run.smithi061.stdout: libcephfs2-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:24:13.226 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:13.226 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:24:13.413 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install librados2-debuginfo 2023-12-06T18:24:14.021 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:12:24 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:24:14.173 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:24:14.174 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:14.174 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:24:14.174 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:14.174 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:24:14.174 INFO:teuthology.orchestra.run.smithi061.stdout: librados2-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 43 M 2023-12-06T18:24:14.174 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:14.174 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:24:14.174 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:14.174 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:24:14.174 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:14.175 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 43 M 2023-12-06T18:24:14.175 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 248 M 2023-12-06T18:24:14.175 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:24:18.097 INFO:teuthology.orchestra.run.smithi061.stdout:librados2-debuginfo-18.0.0-7644.g96fa2578.el9.x 11 MB/s | 43 MB 00:03 2023-12-06T18:24:18.098 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:24:18.098 INFO:teuthology.orchestra.run.smithi061.stdout:Total 11 MB/s | 43 MB 00:03 2023-12-06T18:24:18.099 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:24:18.103 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:24:18.103 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:24:18.415 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:24:18.415 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:24:23.644 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:24:23.740 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : librados2-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/1 2023-12-06T18:24:25.582 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librados2-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/1 2023-12-06T18:24:25.868 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librados2-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_ 1/1 2023-12-06T18:24:25.869 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:25.869 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:24:25.869 INFO:teuthology.orchestra.run.smithi061.stdout: librados2-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:24:25.869 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:25.869 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:24:26.090 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install librbd1-debuginfo 2023-12-06T18:24:26.703 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:12:36 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:24:26.856 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:24:26.856 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:26.856 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:24:26.857 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:26.857 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:24:26.857 INFO:teuthology.orchestra.run.smithi061.stdout: librbd1-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 50 M 2023-12-06T18:24:26.857 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:26.857 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:24:26.857 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:26.857 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:24:26.857 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:26.857 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 50 M 2023-12-06T18:24:26.858 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 294 M 2023-12-06T18:24:26.858 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:24:31.441 INFO:teuthology.orchestra.run.smithi061.stdout:librbd1-debuginfo-18.0.0-7644.g96fa2578.el9.x86 11 MB/s | 50 MB 00:04 2023-12-06T18:24:31.442 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:24:31.442 INFO:teuthology.orchestra.run.smithi061.stdout:Total 11 MB/s | 50 MB 00:04 2023-12-06T18:24:31.443 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:24:31.447 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:24:31.447 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:24:31.801 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:24:31.801 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:24:38.044 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:24:38.150 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : librbd1-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:24:40.223 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: librbd1-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:24:40.524 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : librbd1-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:24:40.524 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:40.524 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:24:40.524 INFO:teuthology.orchestra.run.smithi061.stdout: librbd1-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:24:40.524 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:40.524 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:24:40.729 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install python3-cephfs-debuginfo 2023-12-06T18:24:41.337 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:12:51 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:24:41.490 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:24:41.491 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:41.491 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:24:41.491 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:41.491 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:24:41.491 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cephfs-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 615 k 2023-12-06T18:24:41.491 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:41.491 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:24:41.491 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:41.491 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:24:41.491 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:41.491 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 615 k 2023-12-06T18:24:41.491 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 2.1 M 2023-12-06T18:24:41.491 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:24:42.110 INFO:teuthology.orchestra.run.smithi061.stdout:python3-cephfs-debuginfo-18.0.0-7644.g96fa2578. 994 kB/s | 615 kB 00:00 2023-12-06T18:24:42.111 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:24:42.111 INFO:teuthology.orchestra.run.smithi061.stdout:Total 992 kB/s | 615 kB 00:00 2023-12-06T18:24:42.111 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:24:42.115 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:24:42.115 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:24:42.155 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:24:42.155 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:24:42.335 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:24:42.448 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-cephfs-debuginfo-2:18.0.0-7644.g96fa2578.el9 1/1 2023-12-06T18:24:42.879 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: python3-cephfs-debuginfo-2:18.0.0-7644.g96fa2578.el9 1/1 2023-12-06T18:24:43.198 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-cephfs-debuginfo-2:18.0.0-7644.g96fa2578.el9 1/1 2023-12-06T18:24:43.198 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:43.198 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:24:43.198 INFO:teuthology.orchestra.run.smithi061.stdout: python3-cephfs-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:24:43.198 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:43.198 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:24:43.398 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install python3-rados-debuginfo 2023-12-06T18:24:44.010 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:12:54 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:24:44.163 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:24:44.164 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:44.164 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:24:44.164 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:44.164 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:24:44.164 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rados-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 770 k 2023-12-06T18:24:44.164 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:44.165 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:24:44.165 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:44.165 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:24:44.165 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:44.165 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 770 k 2023-12-06T18:24:44.165 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 2.7 M 2023-12-06T18:24:44.165 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:24:45.205 INFO:teuthology.orchestra.run.smithi061.stdout:python3-rados-debuginfo-18.0.0-7644.g96fa2578.e 740 kB/s | 770 kB 00:01 2023-12-06T18:24:45.206 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:24:45.206 INFO:teuthology.orchestra.run.smithi061.stdout:Total 739 kB/s | 770 kB 00:01 2023-12-06T18:24:45.206 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:24:45.210 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:24:45.210 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:24:45.254 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:24:45.254 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:24:45.449 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:24:45.557 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-rados-debuginfo-2:18.0.0-7644.g96fa2578.el9. 1/1 2023-12-06T18:24:45.972 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: python3-rados-debuginfo-2:18.0.0-7644.g96fa2578.el9. 1/1 2023-12-06T18:24:46.272 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-rados-debuginfo-2:18.0.0-7644.g96fa2578.el9. 1/1 2023-12-06T18:24:46.272 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:46.272 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:24:46.272 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rados-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:24:46.272 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:46.272 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:24:46.498 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install python3-rbd-debuginfo 2023-12-06T18:24:47.108 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:12:57 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:24:47.260 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:24:47.261 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:47.261 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:24:47.261 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:47.261 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:24:47.261 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rbd-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 1.1 M 2023-12-06T18:24:47.261 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:47.261 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:24:47.261 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:47.261 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:24:47.262 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:47.262 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 1.1 M 2023-12-06T18:24:47.262 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 4.3 M 2023-12-06T18:24:47.262 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:24:48.675 INFO:teuthology.orchestra.run.smithi061.stdout:python3-rbd-debuginfo-18.0.0-7644.g96fa2578.el9 815 kB/s | 1.1 MB 00:01 2023-12-06T18:24:48.675 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:24:48.676 INFO:teuthology.orchestra.run.smithi061.stdout:Total 814 kB/s | 1.1 MB 00:01 2023-12-06T18:24:48.676 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:24:48.680 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:24:48.680 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:24:48.728 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:24:48.728 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:24:48.965 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:24:49.076 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-rbd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x8 1/1 2023-12-06T18:24:49.496 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: python3-rbd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x8 1/1 2023-12-06T18:24:49.930 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-rbd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x8 1/1 2023-12-06T18:24:49.930 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:49.930 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:24:49.930 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rbd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:24:49.930 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:49.930 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:24:50.158 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install python3-rgw-debuginfo 2023-12-06T18:24:50.765 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:13:00 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:24:50.919 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:24:50.919 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:50.920 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:24:50.920 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:50.920 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:24:50.920 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rgw-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 337 k 2023-12-06T18:24:50.920 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:50.920 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:24:50.920 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:50.920 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:24:50.920 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:50.920 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 337 k 2023-12-06T18:24:50.920 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 1.0 M 2023-12-06T18:24:50.920 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:24:51.527 INFO:teuthology.orchestra.run.smithi061.stdout:python3-rgw-debuginfo-18.0.0-7644.g96fa2578.el9 555 kB/s | 337 kB 00:00 2023-12-06T18:24:51.528 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:24:51.528 INFO:teuthology.orchestra.run.smithi061.stdout:Total 554 kB/s | 337 kB 00:00 2023-12-06T18:24:51.528 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:24:51.532 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:24:51.532 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:24:51.572 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:24:51.572 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:24:51.718 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:24:51.820 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : python3-rgw-debuginfo-2:18.0.0-7644.g96fa2578.el9.x8 1/1 2023-12-06T18:24:52.222 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: python3-rgw-debuginfo-2:18.0.0-7644.g96fa2578.el9.x8 1/1 2023-12-06T18:24:52.503 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : python3-rgw-debuginfo-2:18.0.0-7644.g96fa2578.el9.x8 1/1 2023-12-06T18:24:52.503 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:52.503 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:24:52.503 INFO:teuthology.orchestra.run.smithi061.stdout: python3-rgw-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:24:52.503 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:52.503 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:24:52.693 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install rbd-fuse-debuginfo 2023-12-06T18:24:53.303 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:13:03 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:24:53.456 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:24:53.457 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:53.457 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:24:53.457 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:53.457 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:24:53.457 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-fuse-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 506 k 2023-12-06T18:24:53.458 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:53.458 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:24:53.458 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:53.458 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:24:53.458 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:53.458 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 506 k 2023-12-06T18:24:53.458 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 2.5 M 2023-12-06T18:24:53.458 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:24:53.756 INFO:teuthology.orchestra.run.smithi061.stdout:rbd-fuse-debuginfo-18.0.0-7644.g96fa2578.el9.x8 1.7 MB/s | 506 kB 00:00 2023-12-06T18:24:53.756 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:24:53.756 INFO:teuthology.orchestra.run.smithi061.stdout:Total 1.7 MB/s | 506 kB 00:00 2023-12-06T18:24:53.756 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:24:53.760 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:24:53.760 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:24:53.800 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:24:53.800 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:24:53.966 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:24:54.083 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : rbd-fuse-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:24:54.523 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: rbd-fuse-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:24:54.821 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : rbd-fuse-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_6 1/1 2023-12-06T18:24:54.821 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:54.821 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:24:54.821 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-fuse-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:24:54.821 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:54.821 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:24:55.023 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install rbd-mirror-debuginfo 2023-12-06T18:24:55.634 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:13:05 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:24:55.790 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:24:55.790 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:55.790 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:24:55.790 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:55.790 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:24:55.791 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-mirror-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 50 M 2023-12-06T18:24:55.791 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:55.791 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:24:55.791 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:24:55.791 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:24:55.791 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:24:55.791 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 50 M 2023-12-06T18:24:55.791 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 284 M 2023-12-06T18:24:55.791 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:25:00.483 INFO:teuthology.orchestra.run.smithi061.stdout:rbd-mirror-debuginfo-18.0.0-7644.g96fa2578.el9. 11 MB/s | 50 MB 00:04 2023-12-06T18:25:00.484 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:25:00.484 INFO:teuthology.orchestra.run.smithi061.stdout:Total 11 MB/s | 50 MB 00:04 2023-12-06T18:25:00.485 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:25:00.489 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:25:00.489 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:25:00.830 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:25:00.830 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:25:06.862 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:25:06.959 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : rbd-mirror-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86 1/1 2023-12-06T18:25:08.952 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: rbd-mirror-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86 1/1 2023-12-06T18:25:09.254 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : rbd-mirror-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86 1/1 2023-12-06T18:25:09.254 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:09.254 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:25:09.255 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-mirror-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:25:09.255 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:09.255 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:25:09.466 DEBUG:teuthology.orchestra.run.smithi061:> sudo yum -y install rbd-nbd-debuginfo 2023-12-06T18:25:10.089 INFO:teuthology.orchestra.run.smithi061.stdout:Last metadata expiration check: 0:13:20 ago on Wed 06 Dec 2023 06:11:50 PM UTC. 2023-12-06T18:25:10.242 INFO:teuthology.orchestra.run.smithi061.stdout:Dependencies resolved. 2023-12-06T18:25:10.242 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:25:10.242 INFO:teuthology.orchestra.run.smithi061.stdout: Package Arch Version Repo Size 2023-12-06T18:25:10.243 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:25:10.243 INFO:teuthology.orchestra.run.smithi061.stdout:Installing: 2023-12-06T18:25:10.243 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-nbd-debuginfo x86_64 2:18.0.0-7644.g96fa2578.el9 ceph 1.2 M 2023-12-06T18:25:10.243 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:10.243 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction Summary 2023-12-06T18:25:10.243 INFO:teuthology.orchestra.run.smithi061.stdout:================================================================================ 2023-12-06T18:25:10.243 INFO:teuthology.orchestra.run.smithi061.stdout:Install 1 Package 2023-12-06T18:25:10.243 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:10.243 INFO:teuthology.orchestra.run.smithi061.stdout:Total download size: 1.2 M 2023-12-06T18:25:10.243 INFO:teuthology.orchestra.run.smithi061.stdout:Installed size: 5.4 M 2023-12-06T18:25:10.243 INFO:teuthology.orchestra.run.smithi061.stdout:Downloading Packages: 2023-12-06T18:25:10.735 INFO:teuthology.orchestra.run.smithi061.stdout:rbd-nbd-debuginfo-18.0.0-7644.g96fa2578.el9.x86 2.5 MB/s | 1.2 MB 00:00 2023-12-06T18:25:10.736 INFO:teuthology.orchestra.run.smithi061.stdout:-------------------------------------------------------------------------------- 2023-12-06T18:25:10.736 INFO:teuthology.orchestra.run.smithi061.stdout:Total 2.5 MB/s | 1.2 MB 00:00 2023-12-06T18:25:10.736 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction check 2023-12-06T18:25:10.740 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction check succeeded. 2023-12-06T18:25:10.740 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction test 2023-12-06T18:25:10.785 INFO:teuthology.orchestra.run.smithi061.stdout:Transaction test succeeded. 2023-12-06T18:25:10.785 INFO:teuthology.orchestra.run.smithi061.stdout:Running transaction 2023-12-06T18:25:11.042 INFO:teuthology.orchestra.run.smithi061.stdout: Preparing : 1/1 2023-12-06T18:25:11.145 INFO:teuthology.orchestra.run.smithi061.stdout: Installing : rbd-nbd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:25:11.563 INFO:teuthology.orchestra.run.smithi061.stdout: Running scriptlet: rbd-nbd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:25:11.860 INFO:teuthology.orchestra.run.smithi061.stdout: Verifying : rbd-nbd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 1/1 2023-12-06T18:25:11.860 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:11.860 INFO:teuthology.orchestra.run.smithi061.stdout:Installed: 2023-12-06T18:25:11.860 INFO:teuthology.orchestra.run.smithi061.stdout: rbd-nbd-debuginfo-2:18.0.0-7644.g96fa2578.el9.x86_64 2023-12-06T18:25:11.860 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:11.860 INFO:teuthology.orchestra.run.smithi061.stdout:Complete! 2023-12-06T18:25:12.059 DEBUG:teuthology.parallel:result is None 2023-12-06T18:25:12.060 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96fa2578c622ca9c169af6f426ca8b1616130e58 2023-12-06T18:25:12.308 DEBUG:teuthology.orchestra.run.smithi061:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-06T18:25:12.330 INFO:teuthology.orchestra.run.smithi061.stdout:18.0.0-7644.g96fa2578.el9 2023-12-06T18:25:12.330 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7644.g96fa2578.el9 2023-12-06T18:25:12.330 INFO:teuthology.task.install:The correct ceph version 18.0.0-7644.g96fa2578 is installed. 2023-12-06T18:25:12.331 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96fa2578c622ca9c169af6f426ca8b1616130e58 2023-12-06T18:25:12.466 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-06T18:25:12.490 INFO:teuthology.orchestra.run.smithi152.stdout:18.0.0-7644.g96fa2578.el9 2023-12-06T18:25:12.490 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7644.g96fa2578.el9 2023-12-06T18:25:12.490 INFO:teuthology.task.install:The correct ceph version 18.0.0-7644.g96fa2578 is installed. 2023-12-06T18:25:12.491 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-06T18:25:12.491 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:25:12.491 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-06T18:25:12.523 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:25:12.523 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-06T18:25:12.561 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-06T18:25:12.561 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:25:12.561 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/usr/bin/daemon-helper 2023-12-06T18:25:12.595 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-06T18:25:12.667 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:25:12.667 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/daemon-helper 2023-12-06T18:25:12.697 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-06T18:25:12.767 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-06T18:25:12.767 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:25:12.767 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-06T18:25:12.798 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-06T18:25:12.869 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:25:12.869 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-06T18:25:12.900 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-06T18:25:12.969 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-06T18:25:12.969 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:25:12.970 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/usr/bin/stdin-killer 2023-12-06T18:25:13.000 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-06T18:25:13.070 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:25:13.070 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/stdin-killer 2023-12-06T18:25:13.099 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-06T18:25:13.168 INFO:teuthology.run_tasks:Running task ceph... 2023-12-06T18:25:13.237 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-12-06T18:25:13.237 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod 777 /var/log/ceph 2023-12-06T18:25:13.240 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 777 /var/log/ceph 2023-12-06T18:25:13.268 INFO:tasks.ceph:Disabling ceph logrotate... 2023-12-06T18:25:13.268 DEBUG:teuthology.orchestra.run.smithi061:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-06T18:25:13.311 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-06T18:25:13.339 INFO:tasks.ceph:Creating extra log directories... 2023-12-06T18:25:13.339 DEBUG:teuthology.orchestra.run.smithi061:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-06T18:25:13.382 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-06T18:25:13.420 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-12-06T18:25:13.420 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(OSDMAP_FLAGS\\)', '\\(SMALLER_PGP_NUM\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW OPS\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-12-06T18:25:13.420 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/sseshasa-2023-12-06_17:50:03-rados-wip-sseshasa-testing-2023-12-04-1848-distro-default-smithi/7480579', 'branch': 'wip-sseshasa-testing-2023-12-04-1848', 'description': 'rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/mon_recovery validater/valgrind}', 'email': None, 'first_in_suite': False, 'job_id': '7480579', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'sseshasa-2023-12-06_17:50:03-rados-wip-sseshasa-testing-2023-12-04-1848-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-sseshasa-testing-2023-12-04-1848'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(OSDMAP_FLAGS\\)', '\\(SMALLER_PGP_NUM\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW OPS\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request'], 'sha1': '96fa2578c622ca9c169af6f426ca8b1616130e58', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': '96fa2578c622ca9c169af6f426ca8b1616130e58'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-sseshasa-testing-2023-12-04-1848', 'sha1': '96fa2578c622ca9c169af6f426ca8b1616130e58'}}, 'owner': 'scheduled_sseshasa@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 5341, 'sha1': '96fa2578c622ca9c169af6f426ca8b1616130e58', 'sleep_before_teardown': 0, 'subset': '222/150000', 'suite': 'rados', 'suite_branch': 'wip-sseshasa-testing-2023-12-04-1848', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '96fa2578c622ca9c169af6f426ca8b1616130e58', 'targets': {'smithi061.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOOmUHjTe45/NHixBF/csb/zjzk1zdv1ekMBaylRiQCGZ9jpEg1Wr/laHEkskFVFyHo59Ut5qZjChz0i6UlJ+Gw=', 'smithi152.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMB/1Zaef7NTky5wA6AE8OcK28GIDiVUGrOtfy53WeFfxKrMrUh4n/UUWyealTa1cbnzMBnXCmaa32+rVaKVxfM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200}}, {'mon_recovery': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'f2de108beb0a6dde7f3e6c96039071828a1f6269', 'timestamp': '2023-12-06_17:50:03', 'tube': 'smithi', 'user': 'sseshasa', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-12-06T18:25:13.421 DEBUG:teuthology.orchestra.run.smithi061:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-06T18:25:13.453 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-06T18:25:13.468 DEBUG:teuthology.orchestra.run.smithi061:> sudo install -d -m0777 -- /var/run/ceph 2023-12-06T18:25:13.508 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m0777 -- /var/run/ceph 2023-12-06T18:25:13.536 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:25:13.536 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/scratch_devs of=/dev/stdout 2023-12-06T18:25:13.592 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-06T18:25:13.592 DEBUG:teuthology.orchestra.run.smithi061:> stat /dev/vg_nvme/lv_1 2023-12-06T18:25:13.648 INFO:teuthology.orchestra.run.smithi061.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-06T18:25:13.648 INFO:teuthology.orchestra.run.smithi061.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-06T18:25:13.648 INFO:teuthology.orchestra.run.smithi061.stdout:Device: 5h/5d Inode: 663 Links: 1 2023-12-06T18:25:13.648 INFO:teuthology.orchestra.run.smithi061.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-06T18:25:13.648 INFO:teuthology.orchestra.run.smithi061.stdout:Context: system_u:object_r:device_t:s0 2023-12-06T18:25:13.648 INFO:teuthology.orchestra.run.smithi061.stdout:Access: 2023-12-06 18:13:36.296893735 +0000 2023-12-06T18:25:13.648 INFO:teuthology.orchestra.run.smithi061.stdout:Modify: 2023-12-06 18:10:57.671840055 +0000 2023-12-06T18:25:13.648 INFO:teuthology.orchestra.run.smithi061.stdout:Change: 2023-12-06 18:10:57.671840055 +0000 2023-12-06T18:25:13.648 INFO:teuthology.orchestra.run.smithi061.stdout: Birth: 2023-12-06 18:10:57.671840055 +0000 2023-12-06T18:25:13.648 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-06T18:25:13.718 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records in 2023-12-06T18:25:13.718 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records out 2023-12-06T18:25:13.718 INFO:teuthology.orchestra.run.smithi061.stderr:512 bytes copied, 0.000205173 s, 2.5 MB/s 2023-12-06T18:25:13.719 DEBUG:teuthology.orchestra.run.smithi061:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-06T18:25:13.776 DEBUG:teuthology.orchestra.run.smithi061:> stat /dev/vg_nvme/lv_2 2023-12-06T18:25:13.831 INFO:teuthology.orchestra.run.smithi061.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-06T18:25:13.831 INFO:teuthology.orchestra.run.smithi061.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-06T18:25:13.832 INFO:teuthology.orchestra.run.smithi061.stdout:Device: 5h/5d Inode: 682 Links: 1 2023-12-06T18:25:13.832 INFO:teuthology.orchestra.run.smithi061.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-06T18:25:13.832 INFO:teuthology.orchestra.run.smithi061.stdout:Context: system_u:object_r:device_t:s0 2023-12-06T18:25:13.832 INFO:teuthology.orchestra.run.smithi061.stdout:Access: 2023-12-06 18:13:36.297893718 +0000 2023-12-06T18:25:13.832 INFO:teuthology.orchestra.run.smithi061.stdout:Modify: 2023-12-06 18:10:58.184831238 +0000 2023-12-06T18:25:13.832 INFO:teuthology.orchestra.run.smithi061.stdout:Change: 2023-12-06 18:10:58.184831238 +0000 2023-12-06T18:25:13.832 INFO:teuthology.orchestra.run.smithi061.stdout: Birth: 2023-12-06 18:10:58.184831238 +0000 2023-12-06T18:25:13.832 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-06T18:25:13.901 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records in 2023-12-06T18:25:13.901 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records out 2023-12-06T18:25:13.901 INFO:teuthology.orchestra.run.smithi061.stderr:512 bytes copied, 0.000204664 s, 2.5 MB/s 2023-12-06T18:25:13.902 DEBUG:teuthology.orchestra.run.smithi061:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-06T18:25:13.958 DEBUG:teuthology.orchestra.run.smithi061:> stat /dev/vg_nvme/lv_3 2023-12-06T18:25:14.013 INFO:teuthology.orchestra.run.smithi061.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-06T18:25:14.013 INFO:teuthology.orchestra.run.smithi061.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-06T18:25:14.014 INFO:teuthology.orchestra.run.smithi061.stdout:Device: 5h/5d Inode: 702 Links: 1 2023-12-06T18:25:14.014 INFO:teuthology.orchestra.run.smithi061.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-06T18:25:14.014 INFO:teuthology.orchestra.run.smithi061.stdout:Context: system_u:object_r:device_t:s0 2023-12-06T18:25:14.014 INFO:teuthology.orchestra.run.smithi061.stdout:Access: 2023-12-06 18:13:36.297893718 +0000 2023-12-06T18:25:14.014 INFO:teuthology.orchestra.run.smithi061.stdout:Modify: 2023-12-06 18:10:58.680822713 +0000 2023-12-06T18:25:14.014 INFO:teuthology.orchestra.run.smithi061.stdout:Change: 2023-12-06 18:10:58.680822713 +0000 2023-12-06T18:25:14.014 INFO:teuthology.orchestra.run.smithi061.stdout: Birth: 2023-12-06 18:10:58.680822713 +0000 2023-12-06T18:25:14.014 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-06T18:25:14.082 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records in 2023-12-06T18:25:14.082 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records out 2023-12-06T18:25:14.082 INFO:teuthology.orchestra.run.smithi061.stderr:512 bytes copied, 0.000169733 s, 3.0 MB/s 2023-12-06T18:25:14.083 DEBUG:teuthology.orchestra.run.smithi061:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-06T18:25:14.139 DEBUG:teuthology.orchestra.run.smithi061:> stat /dev/vg_nvme/lv_4 2023-12-06T18:25:14.194 INFO:teuthology.orchestra.run.smithi061.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-06T18:25:14.195 INFO:teuthology.orchestra.run.smithi061.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-06T18:25:14.195 INFO:teuthology.orchestra.run.smithi061.stdout:Device: 5h/5d Inode: 719 Links: 1 2023-12-06T18:25:14.195 INFO:teuthology.orchestra.run.smithi061.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-06T18:25:14.195 INFO:teuthology.orchestra.run.smithi061.stdout:Context: system_u:object_r:device_t:s0 2023-12-06T18:25:14.195 INFO:teuthology.orchestra.run.smithi061.stdout:Access: 2023-12-06 18:13:36.298893701 +0000 2023-12-06T18:25:14.195 INFO:teuthology.orchestra.run.smithi061.stdout:Modify: 2023-12-06 18:10:59.185814033 +0000 2023-12-06T18:25:14.195 INFO:teuthology.orchestra.run.smithi061.stdout:Change: 2023-12-06 18:10:59.185814033 +0000 2023-12-06T18:25:14.195 INFO:teuthology.orchestra.run.smithi061.stdout: Birth: 2023-12-06 18:10:59.185814033 +0000 2023-12-06T18:25:14.195 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-06T18:25:14.263 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records in 2023-12-06T18:25:14.263 INFO:teuthology.orchestra.run.smithi061.stderr:1+0 records out 2023-12-06T18:25:14.263 INFO:teuthology.orchestra.run.smithi061.stderr:512 bytes copied, 0.000203265 s, 2.5 MB/s 2023-12-06T18:25:14.264 DEBUG:teuthology.orchestra.run.smithi061:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-06T18:25:14.320 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-06T18:25:14.320 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:25:14.320 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/scratch_devs of=/dev/stdout 2023-12-06T18:25:14.338 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-06T18:25:14.338 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_1 2023-12-06T18:25:14.395 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-06T18:25:14.396 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-06T18:25:14.396 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 662 Links: 1 2023-12-06T18:25:14.396 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-06T18:25:14.396 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2023-12-06T18:25:14.396 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2023-12-06 18:13:31.718868106 +0000 2023-12-06T18:25:14.396 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2023-12-06 18:10:49.731572306 +0000 2023-12-06T18:25:14.396 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2023-12-06 18:10:49.731572306 +0000 2023-12-06T18:25:14.396 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2023-12-06 18:10:49.731572306 +0000 2023-12-06T18:25:14.396 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-06T18:25:14.463 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2023-12-06T18:25:14.464 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2023-12-06T18:25:14.464 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000253368 s, 2.0 MB/s 2023-12-06T18:25:14.465 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-06T18:25:14.523 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_2 2023-12-06T18:25:14.579 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-06T18:25:14.580 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-06T18:25:14.580 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 682 Links: 1 2023-12-06T18:25:14.580 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-06T18:25:14.580 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2023-12-06T18:25:14.580 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2023-12-06 18:13:31.719868095 +0000 2023-12-06T18:25:14.580 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2023-12-06 18:10:50.237566494 +0000 2023-12-06T18:25:14.580 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2023-12-06 18:10:50.237566494 +0000 2023-12-06T18:25:14.580 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2023-12-06 18:10:50.237566494 +0000 2023-12-06T18:25:14.580 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-06T18:25:14.648 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2023-12-06T18:25:14.648 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2023-12-06T18:25:14.648 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000230274 s, 2.2 MB/s 2023-12-06T18:25:14.649 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-06T18:25:14.707 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_3 2023-12-06T18:25:14.763 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-06T18:25:14.764 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-06T18:25:14.764 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 703 Links: 1 2023-12-06T18:25:14.764 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-06T18:25:14.764 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2023-12-06T18:25:14.764 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2023-12-06 18:13:31.719868095 +0000 2023-12-06T18:25:14.764 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2023-12-06 18:10:50.747560637 +0000 2023-12-06T18:25:14.764 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2023-12-06 18:10:50.747560637 +0000 2023-12-06T18:25:14.764 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2023-12-06 18:10:50.747560637 +0000 2023-12-06T18:25:14.764 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-06T18:25:14.830 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2023-12-06T18:25:14.830 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2023-12-06T18:25:14.830 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000220365 s, 2.3 MB/s 2023-12-06T18:25:14.832 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-06T18:25:14.889 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_4 2023-12-06T18:25:14.945 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-06T18:25:14.945 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-06T18:25:14.945 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 720 Links: 1 2023-12-06T18:25:14.945 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-06T18:25:14.945 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2023-12-06T18:25:14.945 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2023-12-06 18:13:31.719868095 +0000 2023-12-06T18:25:14.945 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2023-12-06 18:10:51.238554998 +0000 2023-12-06T18:25:14.946 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2023-12-06 18:10:51.238554998 +0000 2023-12-06T18:25:14.946 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2023-12-06 18:10:51.238554998 +0000 2023-12-06T18:25:14.946 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-06T18:25:15.012 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2023-12-06T18:25:15.012 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2023-12-06T18:25:15.012 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000231272 s, 2.2 MB/s 2023-12-06T18:25:15.013 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-06T18:25:15.069 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-06T18:25:15.070 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi061.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi152.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-06T18:25:15.070 INFO:tasks.ceph:Generating config... 2023-12-06T18:25:15.075 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-12-06T18:25:15.075 INFO:tasks.ceph:[global] mon election default strategy = 3 2023-12-06T18:25:15.075 INFO:tasks.ceph:[global] ms bind msgr1 = False 2023-12-06T18:25:15.075 INFO:tasks.ceph:[global] ms bind msgr2 = True 2023-12-06T18:25:15.075 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2023-12-06T18:25:15.076 INFO:tasks.ceph:[global] ms type = async 2023-12-06T18:25:15.076 INFO:tasks.ceph:[global] osd heartbeat grace = 80 2023-12-06T18:25:15.076 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-12-06T18:25:15.076 INFO:tasks.ceph:[mgr] debug ms = 1 2023-12-06T18:25:15.076 INFO:tasks.ceph:[mon] debug mon = 20 2023-12-06T18:25:15.076 INFO:tasks.ceph:[mon] debug ms = 1 2023-12-06T18:25:15.076 INFO:tasks.ceph:[mon] debug paxos = 20 2023-12-06T18:25:15.076 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-12-06T18:25:15.076 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2023-12-06T18:25:15.076 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-12-06T18:25:15.076 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-12-06T18:25:15.076 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-12-06T18:25:15.076 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-12-06T18:25:15.076 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-12-06T18:25:15.076 INFO:tasks.ceph:[osd] bdev async discard = True 2023-12-06T18:25:15.076 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-12-06T18:25:15.077 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-12-06T18:25:15.077 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-12-06T18:25:15.077 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-12-06T18:25:15.077 INFO:tasks.ceph:[osd] debug bluefs = 1 2023-12-06T18:25:15.077 INFO:tasks.ceph:[osd] debug bluestore = 1 2023-12-06T18:25:15.077 INFO:tasks.ceph:[osd] debug monc = 20 2023-12-06T18:25:15.077 INFO:tasks.ceph:[osd] debug ms = 1 2023-12-06T18:25:15.077 INFO:tasks.ceph:[osd] debug osd = 20 2023-12-06T18:25:15.077 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-12-06T18:25:15.077 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-12-06T18:25:15.077 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-12-06T18:25:15.077 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-12-06T18:25:15.077 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2023-12-06T18:25:15.078 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-12-06T18:25:15.078 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-12-06T18:25:15.078 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-12-06T18:25:15.078 INFO:tasks.ceph:[osd] osd fast shutdown = False 2023-12-06T18:25:15.078 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-12-06T18:25:15.078 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-12-06T18:25:15.078 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2023-12-06T18:25:15.078 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2023-12-06T18:25:15.078 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-12-06T18:25:15.078 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-12-06T18:25:15.078 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-12-06T18:25:15.078 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-12-06T18:25:15.078 INFO:tasks.ceph:Setting up mon.a... 2023-12-06T18:25:15.079 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-12-06T18:25:15.140 INFO:teuthology.orchestra.run.smithi061.stdout:creating /etc/ceph/ceph.keyring 2023-12-06T18:25:15.143 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-12-06T18:25:15.194 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-06T18:25:15.226 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.61'), ('mon.c', '[v2:172.21.15.61:3301,v1:172.21.15.61:6790]'), ('mon.b', '172.21.15.152')] 2023-12-06T18:25:15.227 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790],172.21.15.152', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-12-06T18:25:15.227 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:25:15.227 DEBUG:teuthology.orchestra.run.smithi061:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-12-06T18:25:15.282 DEBUG:teuthology.orchestra.run.smithi061:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.61 --addv c '[v2:172.21.15.61:3301,v1:172.21.15.61:6790]' --add b 172.21.15.152 --print /home/ubuntu/cephtest/ceph.monmap 2023-12-06T18:25:15.369 INFO:teuthology.orchestra.run.smithi061.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-12-06T18:25:15.369 INFO:teuthology.orchestra.run.smithi061.stdout:monmaptool: generated fsid 497e3283-cc16-40ff-a72c-a7487474fb4b 2023-12-06T18:25:15.369 INFO:teuthology.orchestra.run.smithi061.stdout:setting min_mon_release = pacific 2023-12-06T18:25:15.369 INFO:teuthology.orchestra.run.smithi061.stdout:epoch 0 2023-12-06T18:25:15.369 INFO:teuthology.orchestra.run.smithi061.stdout:fsid 497e3283-cc16-40ff-a72c-a7487474fb4b 2023-12-06T18:25:15.369 INFO:teuthology.orchestra.run.smithi061.stdout:last_changed 2023-12-06T18:25:15.366722+0000 2023-12-06T18:25:15.369 INFO:teuthology.orchestra.run.smithi061.stdout:created 2023-12-06T18:25:15.366722+0000 2023-12-06T18:25:15.369 INFO:teuthology.orchestra.run.smithi061.stdout:min_mon_release 16 (pacific) 2023-12-06T18:25:15.369 INFO:teuthology.orchestra.run.smithi061.stdout:election_strategy: 3 2023-12-06T18:25:15.370 INFO:teuthology.orchestra.run.smithi061.stdout:0: [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] mon.a 2023-12-06T18:25:15.370 INFO:teuthology.orchestra.run.smithi061.stdout:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2023-12-06T18:25:15.370 INFO:teuthology.orchestra.run.smithi061.stdout:2: [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] mon.c 2023-12-06T18:25:15.370 INFO:teuthology.orchestra.run.smithi061.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-12-06T18:25:15.371 DEBUG:teuthology.orchestra.run.smithi061:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-12-06T18:25:15.386 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 497e3283-cc16-40ff-a72c-a7487474fb4b... 2023-12-06T18:25:15.387 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-06T18:25:15.429 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-06T18:25:15.483 INFO:teuthology.orchestra.run.smithi152.stdout:[global] 2023-12-06T18:25:15.483 INFO:teuthology.orchestra.run.smithi152.stdout: chdir = "" 2023-12-06T18:25:15.483 INFO:teuthology.orchestra.run.smithi152.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-06T18:25:15.483 INFO:teuthology.orchestra.run.smithi152.stdout: auth supported = cephx 2023-12-06T18:25:15.483 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.483 INFO:teuthology.orchestra.run.smithi152.stdout: filestore xattr use omap = true 2023-12-06T18:25:15.483 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.483 INFO:teuthology.orchestra.run.smithi152.stdout: mon clock drift allowed = 1.000 2023-12-06T18:25:15.484 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.484 INFO:teuthology.orchestra.run.smithi152.stdout: osd crush chooseleaf type = 0 2023-12-06T18:25:15.484 INFO:teuthology.orchestra.run.smithi152.stdout: auth debug = true 2023-12-06T18:25:15.484 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.484 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on old message = true 2023-12-06T18:25:15.484 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on bug = true 2023-12-06T18:25:15.484 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.484 INFO:teuthology.orchestra.run.smithi152.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-06T18:25:15.484 INFO:teuthology.orchestra.run.smithi152.stdout: mon pg warn max object skew = 0 2023-12-06T18:25:15.484 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.484 INFO:teuthology.orchestra.run.smithi152.stdout: # disable pg_autoscaler by default for new pools 2023-12-06T18:25:15.484 INFO:teuthology.orchestra.run.smithi152.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-06T18:25:15.484 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.484 INFO:teuthology.orchestra.run.smithi152.stdout: osd pool default size = 2 2023-12-06T18:25:15.484 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.484 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd allow primary affinity = true 2023-12-06T18:25:15.485 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd allow pg remap = true 2023-12-06T18:25:15.485 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on legacy crush tunables = false 2023-12-06T18:25:15.485 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on crush straw calc version zero = false 2023-12-06T18:25:15.485 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on no sortbitwise = false 2023-12-06T18:25:15.485 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on osd down out interval zero = false 2023-12-06T18:25:15.485 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on too few osds = false 2023-12-06T18:25:15.485 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-06T18:25:15.485 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-06T18:25:15.485 INFO:teuthology.orchestra.run.smithi152.stdout: mon_allow_pool_size_one = true 2023-12-06T18:25:15.485 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.485 INFO:teuthology.orchestra.run.smithi152.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-06T18:25:15.485 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.485 INFO:teuthology.orchestra.run.smithi152.stdout: osd default data pool replay window = 5 2023-12-06T18:25:15.485 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.485 INFO:teuthology.orchestra.run.smithi152.stdout: mon allow pool delete = true 2023-12-06T18:25:15.485 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.486 INFO:teuthology.orchestra.run.smithi152.stdout: mon cluster log file level = debug 2023-12-06T18:25:15.486 INFO:teuthology.orchestra.run.smithi152.stdout: debug asserts on shutdown = true 2023-12-06T18:25:15.486 INFO:teuthology.orchestra.run.smithi152.stdout: mon health detail to clog = false 2023-12-06T18:25:15.486 INFO:teuthology.orchestra.run.smithi152.stdout: mon host = "172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790],172.21.15.152" 2023-12-06T18:25:15.486 INFO:teuthology.orchestra.run.smithi152.stdout: mon client directed command retry = 5 2023-12-06T18:25:15.486 INFO:teuthology.orchestra.run.smithi152.stdout: mon election default strategy = 3 2023-12-06T18:25:15.486 INFO:teuthology.orchestra.run.smithi152.stdout: ms bind msgr1 = False 2023-12-06T18:25:15.486 INFO:teuthology.orchestra.run.smithi152.stdout: ms bind msgr2 = True 2023-12-06T18:25:15.486 INFO:teuthology.orchestra.run.smithi152.stdout: ms inject socket failures = 5000 2023-12-06T18:25:15.486 INFO:teuthology.orchestra.run.smithi152.stdout: ms type = async 2023-12-06T18:25:15.486 INFO:teuthology.orchestra.run.smithi152.stdout: osd heartbeat grace = 80 2023-12-06T18:25:15.486 INFO:teuthology.orchestra.run.smithi152.stdout: fsid = 497e3283-cc16-40ff-a72c-a7487474fb4b 2023-12-06T18:25:15.486 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.486 INFO:teuthology.orchestra.run.smithi152.stdout:[osd] 2023-12-06T18:25:15.487 INFO:teuthology.orchestra.run.smithi152.stdout: osd journal size = 100 2023-12-06T18:25:15.487 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.487 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub load threshold = 5.0 2023-12-06T18:25:15.487 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub max interval = 600 2023-12-06T18:25:15.487 INFO:teuthology.orchestra.run.smithi152.stdout: osd mclock profile = high_recovery_ops 2023-12-06T18:25:15.487 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.487 INFO:teuthology.orchestra.run.smithi152.stdout: osd recover clone overlap = true 2023-12-06T18:25:15.487 INFO:teuthology.orchestra.run.smithi152.stdout: osd recovery max chunk = 1048576 2023-12-06T18:25:15.487 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.487 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug shutdown = true 2023-12-06T18:25:15.487 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug op order = true 2023-12-06T18:25:15.487 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug verify stray on activate = true 2023-12-06T18:25:15.487 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug trim objects = true 2023-12-06T18:25:15.487 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.487 INFO:teuthology.orchestra.run.smithi152.stdout: osd open classes on start = true 2023-12-06T18:25:15.487 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug pg log writeout = true 2023-12-06T18:25:15.487 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.488 INFO:teuthology.orchestra.run.smithi152.stdout: osd deep scrub update digest min age = 30 2023-12-06T18:25:15.488 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.488 INFO:teuthology.orchestra.run.smithi152.stdout: osd map max advance = 10 2023-12-06T18:25:15.488 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.488 INFO:teuthology.orchestra.run.smithi152.stdout: journal zero on create = true 2023-12-06T18:25:15.488 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.488 INFO:teuthology.orchestra.run.smithi152.stdout: filestore ondisk finisher threads = 3 2023-12-06T18:25:15.488 INFO:teuthology.orchestra.run.smithi152.stdout: filestore apply finisher threads = 3 2023-12-06T18:25:15.488 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.488 INFO:teuthology.orchestra.run.smithi152.stdout: bdev debug aio = true 2023-12-06T18:25:15.488 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug misdirected ops = true 2023-12-06T18:25:15.488 INFO:teuthology.orchestra.run.smithi152.stdout: bdev async discard = True 2023-12-06T18:25:15.488 INFO:teuthology.orchestra.run.smithi152.stdout: bdev enable discard = True 2023-12-06T18:25:15.488 INFO:teuthology.orchestra.run.smithi152.stdout: bluestore block size = 96636764160 2023-12-06T18:25:15.488 INFO:teuthology.orchestra.run.smithi152.stdout: bluestore fsck on mount = True 2023-12-06T18:25:15.488 INFO:teuthology.orchestra.run.smithi152.stdout: bluestore zero block detection = True 2023-12-06T18:25:15.488 INFO:teuthology.orchestra.run.smithi152.stdout: debug bluefs = 1 2023-12-06T18:25:15.489 INFO:teuthology.orchestra.run.smithi152.stdout: debug bluestore = 1 2023-12-06T18:25:15.489 INFO:teuthology.orchestra.run.smithi152.stdout: debug monc = 20 2023-12-06T18:25:15.489 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2023-12-06T18:25:15.489 INFO:teuthology.orchestra.run.smithi152.stdout: debug osd = 20 2023-12-06T18:25:15.489 INFO:teuthology.orchestra.run.smithi152.stdout: debug rocksdb = 10 2023-12-06T18:25:15.489 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd backfillfull_ratio = 0.85 2023-12-06T18:25:15.489 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd full ratio = 0.9 2023-12-06T18:25:15.489 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd nearfull ratio = 0.8 2023-12-06T18:25:15.489 INFO:teuthology.orchestra.run.smithi152.stdout: osd blocked scrub grace period = 3600 2023-12-06T18:25:15.489 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug verify cached snaps = True 2023-12-06T18:25:15.489 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug verify missing on start = True 2023-12-06T18:25:15.489 INFO:teuthology.orchestra.run.smithi152.stdout: osd failsafe full ratio = 0.95 2023-12-06T18:25:15.489 INFO:teuthology.orchestra.run.smithi152.stdout: osd fast shutdown = False 2023-12-06T18:25:15.489 INFO:teuthology.orchestra.run.smithi152.stdout: osd max markdown count = 1000 2023-12-06T18:25:15.489 INFO:teuthology.orchestra.run.smithi152.stdout: osd mclock override recovery settings = True 2023-12-06T18:25:15.489 INFO:teuthology.orchestra.run.smithi152.stdout: osd memory target = 1610612736 2023-12-06T18:25:15.489 INFO:teuthology.orchestra.run.smithi152.stdout: osd objectstore = bluestore 2023-12-06T18:25:15.490 INFO:teuthology.orchestra.run.smithi152.stdout: osd op queue = debug_random 2023-12-06T18:25:15.490 INFO:teuthology.orchestra.run.smithi152.stdout: osd op queue cut off = debug_random 2023-12-06T18:25:15.490 INFO:teuthology.orchestra.run.smithi152.stdout: osd shutdown pgref assert = True 2023-12-06T18:25:15.490 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.490 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr] 2023-12-06T18:25:15.490 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2023-12-06T18:25:15.490 INFO:teuthology.orchestra.run.smithi152.stdout: debug mgr = 20 2023-12-06T18:25:15.490 INFO:teuthology.orchestra.run.smithi152.stdout: debug mon = 20 2023-12-06T18:25:15.490 INFO:teuthology.orchestra.run.smithi152.stdout: debug auth = 20 2023-12-06T18:25:15.490 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min pgs per osd = 4 2023-12-06T18:25:15.490 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min bytes per osd = 10 2023-12-06T18:25:15.490 INFO:teuthology.orchestra.run.smithi152.stdout: mgr/telemetry/nag = false 2023-12-06T18:25:15.490 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.490 INFO:teuthology.orchestra.run.smithi152.stdout:[mon] 2023-12-06T18:25:15.490 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2023-12-06T18:25:15.490 INFO:teuthology.orchestra.run.smithi152.stdout: debug mon = 20 2023-12-06T18:25:15.491 INFO:teuthology.orchestra.run.smithi152.stdout: debug paxos = 20 2023-12-06T18:25:15.491 INFO:teuthology.orchestra.run.smithi152.stdout: debug auth = 20 2023-12-06T18:25:15.491 INFO:teuthology.orchestra.run.smithi152.stdout: mon data avail warn = 5 2023-12-06T18:25:15.491 INFO:teuthology.orchestra.run.smithi152.stdout: mon mgr mkfs grace = 240 2023-12-06T18:25:15.491 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min pgs per osd = 4 2023-12-06T18:25:15.491 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd reporter subtree level = osd 2023-12-06T18:25:15.491 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd prime pg temp = true 2023-12-06T18:25:15.491 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min bytes per osd = 10 2023-12-06T18:25:15.491 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.491 INFO:teuthology.orchestra.run.smithi152.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-06T18:25:15.491 INFO:teuthology.orchestra.run.smithi152.stdout: auth mon ticket ttl = 660 # 11m 2023-12-06T18:25:15.491 INFO:teuthology.orchestra.run.smithi152.stdout: auth service ticket ttl = 240 # 4m 2023-12-06T18:25:15.491 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.491 INFO:teuthology.orchestra.run.smithi152.stdout: # don't complain about insecure global_id in the test suite 2023-12-06T18:25:15.491 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-06T18:25:15.491 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-06T18:25:15.492 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.492 INFO:teuthology.orchestra.run.smithi152.stdout: # 1m isn't quite enough 2023-12-06T18:25:15.492 INFO:teuthology.orchestra.run.smithi152.stdout: mon_down_mkfs_grace = 2m 2023-12-06T18:25:15.492 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.492 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_filestore_osds = false 2023-12-06T18:25:15.492 INFO:teuthology.orchestra.run.smithi152.stdout: mon min osdmap epochs = 50 2023-12-06T18:25:15.492 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd crush smoke test = False 2023-12-06T18:25:15.492 INFO:teuthology.orchestra.run.smithi152.stdout: mon osdmap full prune interval = 2 2023-12-06T18:25:15.492 INFO:teuthology.orchestra.run.smithi152.stdout: mon osdmap full prune min = 15 2023-12-06T18:25:15.492 INFO:teuthology.orchestra.run.smithi152.stdout: mon osdmap full prune txsize = 2 2023-12-06T18:25:15.492 INFO:teuthology.orchestra.run.smithi152.stdout: mon scrub interval = 300 2023-12-06T18:25:15.492 INFO:teuthology.orchestra.run.smithi152.stdout: paxos service trim min = 10 2023-12-06T18:25:15.492 INFO:teuthology.orchestra.run.smithi152.stdout: 2023-12-06T18:25:15.492 INFO:teuthology.orchestra.run.smithi152.stdout:[client] 2023-12-06T18:25:15.492 INFO:teuthology.orchestra.run.smithi152.stdout: rgw cache enabled = true 2023-12-06T18:25:15.492 INFO:teuthology.orchestra.run.smithi152.stdout: rgw enable ops log = true 2023-12-06T18:25:15.492 INFO:teuthology.orchestra.run.smithi152.stdout: rgw enable usage log = true 2023-12-06T18:25:15.493 INFO:teuthology.orchestra.run.smithi152.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-06T18:25:15.493 INFO:teuthology.orchestra.run.smithi152.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-06T18:25:15.493 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.a] 2023-12-06T18:25:15.493 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.c] 2023-12-06T18:25:15.493 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.b] 2023-12-06T18:25:15.493 INFO:teuthology.orchestra.run.smithi061.stdout:[global] 2023-12-06T18:25:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: chdir = "" 2023-12-06T18:25:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-06T18:25:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: auth supported = cephx 2023-12-06T18:25:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: filestore xattr use omap = true 2023-12-06T18:25:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: mon clock drift allowed = 1.000 2023-12-06T18:25:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: osd crush chooseleaf type = 0 2023-12-06T18:25:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: auth debug = true 2023-12-06T18:25:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: ms die on old message = true 2023-12-06T18:25:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: ms die on bug = true 2023-12-06T18:25:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-06T18:25:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: mon pg warn max object skew = 0 2023-12-06T18:25:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: # disable pg_autoscaler by default for new pools 2023-12-06T18:25:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-06T18:25:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: osd pool default size = 2 2023-12-06T18:25:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd allow primary affinity = true 2023-12-06T18:25:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd allow pg remap = true 2023-12-06T18:25:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on legacy crush tunables = false 2023-12-06T18:25:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on crush straw calc version zero = false 2023-12-06T18:25:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on no sortbitwise = false 2023-12-06T18:25:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on osd down out interval zero = false 2023-12-06T18:25:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: mon warn on too few osds = false 2023-12-06T18:25:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-06T18:25:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-06T18:25:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: mon_allow_pool_size_one = true 2023-12-06T18:25:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-06T18:25:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: osd default data pool replay window = 5 2023-12-06T18:25:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: mon allow pool delete = true 2023-12-06T18:25:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: mon cluster log file level = debug 2023-12-06T18:25:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: debug asserts on shutdown = true 2023-12-06T18:25:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: mon health detail to clog = false 2023-12-06T18:25:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: mon host = "172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790],172.21.15.152" 2023-12-06T18:25:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: mon client directed command retry = 5 2023-12-06T18:25:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: mon election default strategy = 3 2023-12-06T18:25:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: ms bind msgr1 = False 2023-12-06T18:25:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: ms bind msgr2 = True 2023-12-06T18:25:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: ms inject socket failures = 5000 2023-12-06T18:25:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: ms type = async 2023-12-06T18:25:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: osd heartbeat grace = 80 2023-12-06T18:25:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: fsid = 497e3283-cc16-40ff-a72c-a7487474fb4b 2023-12-06T18:25:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.496 INFO:teuthology.orchestra.run.smithi061.stdout:[osd] 2023-12-06T18:25:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: osd journal size = 100 2023-12-06T18:25:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: osd scrub load threshold = 5.0 2023-12-06T18:25:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: osd scrub max interval = 600 2023-12-06T18:25:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: osd mclock profile = high_recovery_ops 2023-12-06T18:25:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: osd recover clone overlap = true 2023-12-06T18:25:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: osd recovery max chunk = 1048576 2023-12-06T18:25:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug shutdown = true 2023-12-06T18:25:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug op order = true 2023-12-06T18:25:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug verify stray on activate = true 2023-12-06T18:25:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug trim objects = true 2023-12-06T18:25:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: osd open classes on start = true 2023-12-06T18:25:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug pg log writeout = true 2023-12-06T18:25:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: osd deep scrub update digest min age = 30 2023-12-06T18:25:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: osd map max advance = 10 2023-12-06T18:25:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: journal zero on create = true 2023-12-06T18:25:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: filestore ondisk finisher threads = 3 2023-12-06T18:25:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: filestore apply finisher threads = 3 2023-12-06T18:25:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: bdev debug aio = true 2023-12-06T18:25:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug misdirected ops = true 2023-12-06T18:25:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: bdev async discard = True 2023-12-06T18:25:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: bdev enable discard = True 2023-12-06T18:25:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: bluestore block size = 96636764160 2023-12-06T18:25:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: bluestore fsck on mount = True 2023-12-06T18:25:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: bluestore zero block detection = True 2023-12-06T18:25:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: debug bluefs = 1 2023-12-06T18:25:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: debug bluestore = 1 2023-12-06T18:25:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: debug monc = 20 2023-12-06T18:25:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2023-12-06T18:25:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: debug osd = 20 2023-12-06T18:25:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: debug rocksdb = 10 2023-12-06T18:25:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd backfillfull_ratio = 0.85 2023-12-06T18:25:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd full ratio = 0.9 2023-12-06T18:25:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd nearfull ratio = 0.8 2023-12-06T18:25:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: osd blocked scrub grace period = 3600 2023-12-06T18:25:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug verify cached snaps = True 2023-12-06T18:25:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: osd debug verify missing on start = True 2023-12-06T18:25:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: osd failsafe full ratio = 0.95 2023-12-06T18:25:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: osd fast shutdown = False 2023-12-06T18:25:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: osd max markdown count = 1000 2023-12-06T18:25:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: osd mclock override recovery settings = True 2023-12-06T18:25:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: osd memory target = 1610612736 2023-12-06T18:25:15.500 INFO:teuthology.orchestra.run.smithi061.stdout: osd objectstore = bluestore 2023-12-06T18:25:15.500 INFO:teuthology.orchestra.run.smithi061.stdout: osd op queue = debug_random 2023-12-06T18:25:15.500 INFO:teuthology.orchestra.run.smithi061.stdout: osd op queue cut off = debug_random 2023-12-06T18:25:15.500 INFO:teuthology.orchestra.run.smithi061.stdout: osd shutdown pgref assert = True 2023-12-06T18:25:15.500 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.500 INFO:teuthology.orchestra.run.smithi061.stdout:[mgr] 2023-12-06T18:25:15.500 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2023-12-06T18:25:15.500 INFO:teuthology.orchestra.run.smithi061.stdout: debug mgr = 20 2023-12-06T18:25:15.500 INFO:teuthology.orchestra.run.smithi061.stdout: debug mon = 20 2023-12-06T18:25:15.500 INFO:teuthology.orchestra.run.smithi061.stdout: debug auth = 20 2023-12-06T18:25:15.500 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min pgs per osd = 4 2023-12-06T18:25:15.500 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min bytes per osd = 10 2023-12-06T18:25:15.500 INFO:teuthology.orchestra.run.smithi061.stdout: mgr/telemetry/nag = false 2023-12-06T18:25:15.500 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.500 INFO:teuthology.orchestra.run.smithi061.stdout:[mon] 2023-12-06T18:25:15.500 INFO:teuthology.orchestra.run.smithi061.stdout: debug ms = 1 2023-12-06T18:25:15.500 INFO:teuthology.orchestra.run.smithi061.stdout: debug mon = 20 2023-12-06T18:25:15.501 INFO:teuthology.orchestra.run.smithi061.stdout: debug paxos = 20 2023-12-06T18:25:15.501 INFO:teuthology.orchestra.run.smithi061.stdout: debug auth = 20 2023-12-06T18:25:15.501 INFO:teuthology.orchestra.run.smithi061.stdout: mon data avail warn = 5 2023-12-06T18:25:15.501 INFO:teuthology.orchestra.run.smithi061.stdout: mon mgr mkfs grace = 240 2023-12-06T18:25:15.501 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min pgs per osd = 4 2023-12-06T18:25:15.501 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd reporter subtree level = osd 2023-12-06T18:25:15.501 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd prime pg temp = true 2023-12-06T18:25:15.501 INFO:teuthology.orchestra.run.smithi061.stdout: mon reweight min bytes per osd = 10 2023-12-06T18:25:15.501 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.501 INFO:teuthology.orchestra.run.smithi061.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-06T18:25:15.501 INFO:teuthology.orchestra.run.smithi061.stdout: auth mon ticket ttl = 660 # 11m 2023-12-06T18:25:15.501 INFO:teuthology.orchestra.run.smithi061.stdout: auth service ticket ttl = 240 # 4m 2023-12-06T18:25:15.501 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.501 INFO:teuthology.orchestra.run.smithi061.stdout: # don't complain about insecure global_id in the test suite 2023-12-06T18:25:15.501 INFO:teuthology.orchestra.run.smithi061.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-06T18:25:15.501 INFO:teuthology.orchestra.run.smithi061.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-06T18:25:15.502 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.502 INFO:teuthology.orchestra.run.smithi061.stdout: # 1m isn't quite enough 2023-12-06T18:25:15.502 INFO:teuthology.orchestra.run.smithi061.stdout: mon_down_mkfs_grace = 2m 2023-12-06T18:25:15.502 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.502 INFO:teuthology.orchestra.run.smithi061.stdout: mon_warn_on_filestore_osds = false 2023-12-06T18:25:15.502 INFO:teuthology.orchestra.run.smithi061.stdout: mon min osdmap epochs = 50 2023-12-06T18:25:15.502 INFO:teuthology.orchestra.run.smithi061.stdout: mon osd crush smoke test = False 2023-12-06T18:25:15.502 INFO:teuthology.orchestra.run.smithi061.stdout: mon osdmap full prune interval = 2 2023-12-06T18:25:15.502 INFO:teuthology.orchestra.run.smithi061.stdout: mon osdmap full prune min = 15 2023-12-06T18:25:15.502 INFO:teuthology.orchestra.run.smithi061.stdout: mon osdmap full prune txsize = 2 2023-12-06T18:25:15.502 INFO:teuthology.orchestra.run.smithi061.stdout: mon scrub interval = 300 2023-12-06T18:25:15.502 INFO:teuthology.orchestra.run.smithi061.stdout: paxos service trim min = 10 2023-12-06T18:25:15.502 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:25:15.502 INFO:teuthology.orchestra.run.smithi061.stdout:[client] 2023-12-06T18:25:15.502 INFO:teuthology.orchestra.run.smithi061.stdout: rgw cache enabled = true 2023-12-06T18:25:15.502 INFO:teuthology.orchestra.run.smithi061.stdout: rgw enable ops log = true 2023-12-06T18:25:15.503 INFO:teuthology.orchestra.run.smithi061.stdout: rgw enable usage log = true 2023-12-06T18:25:15.503 INFO:teuthology.orchestra.run.smithi061.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-06T18:25:15.503 INFO:teuthology.orchestra.run.smithi061.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-06T18:25:15.503 INFO:teuthology.orchestra.run.smithi061.stdout:[mon.a] 2023-12-06T18:25:15.503 INFO:teuthology.orchestra.run.smithi061.stdout:[mon.c] 2023-12-06T18:25:15.503 INFO:teuthology.orchestra.run.smithi061.stdout:[mon.b] 2023-12-06T18:25:15.504 INFO:tasks.ceph:Creating admin key on mon.a... 2023-12-06T18:25:15.504 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-12-06T18:25:15.604 INFO:tasks.ceph:Copying monmap to all nodes... 2023-12-06T18:25:15.604 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:25:15.604 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-12-06T18:25:15.619 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:25:15.619 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-12-06T18:25:15.674 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi061.front.sepia.ceph.com 2023-12-06T18:25:15.675 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:25:15.675 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-06T18:25:15.675 DEBUG:teuthology.orchestra.run.smithi061:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-06T18:25:15.761 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:25:15.761 DEBUG:teuthology.orchestra.run.smithi061:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-06T18:25:15.815 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi152.front.sepia.ceph.com 2023-12-06T18:25:15.815 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:25:15.815 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-06T18:25:15.816 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-06T18:25:15.859 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:25:15.859 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-06T18:25:15.915 INFO:tasks.ceph:Setting up mon nodes... 2023-12-06T18:25:15.915 INFO:tasks.ceph:Setting up mgr nodes... 2023-12-06T18:25:15.915 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-12-06T18:25:15.978 INFO:teuthology.orchestra.run.smithi061.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-12-06T18:25:15.981 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-12-06T18:25:16.052 INFO:teuthology.orchestra.run.smithi152.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-12-06T18:25:16.054 INFO:tasks.ceph:Setting up mds nodes... 2023-12-06T18:25:16.054 INFO:tasks.ceph_client:Setting up client nodes... 2023-12-06T18:25:16.055 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-06T18:25:16.101 INFO:teuthology.orchestra.run.smithi061.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-12-06T18:25:16.119 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-12-06T18:25:16.162 INFO:teuthology.orchestra.run.smithi152.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-12-06T18:25:16.177 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-12-06T18:25:16.177 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi061.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi152.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-06T18:25:16.177 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-12-06T18:25:16.207 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-06T18:25:16.207 INFO:tasks.ceph:role: osd.0 2023-12-06T18:25:16.207 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi061.front.sepia.ceph.com 2023-12-06T18:25:16.208 DEBUG:teuthology.orchestra.run.smithi061:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-06T18:25:16.285 INFO:teuthology.orchestra.run.smithi061.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-06T18:25:16.285 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-06T18:25:16.285 INFO:teuthology.orchestra.run.smithi061.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-06T18:25:16.285 INFO:teuthology.orchestra.run.smithi061.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-06T18:25:16.285 INFO:teuthology.orchestra.run.smithi061.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-06T18:25:16.285 INFO:teuthology.orchestra.run.smithi061.stdout: = sunit=0 swidth=0 blks 2023-12-06T18:25:16.285 INFO:teuthology.orchestra.run.smithi061.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-06T18:25:16.285 INFO:teuthology.orchestra.run.smithi061.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-06T18:25:16.286 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-06T18:25:16.286 INFO:teuthology.orchestra.run.smithi061.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-06T18:25:16.808 INFO:teuthology.orchestra.run.smithi061.stdout:Discarding blocks...Done. 2023-12-06T18:25:16.810 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi061.front.sepia.ceph.com -o noatime 2023-12-06T18:25:16.811 DEBUG:teuthology.orchestra.run.smithi061:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-12-06T18:25:16.953 DEBUG:teuthology.orchestra.run.smithi061:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-12-06T18:25:16.992 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-12-06T18:25:17.066 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-06T18:25:17.066 INFO:tasks.ceph:role: osd.1 2023-12-06T18:25:17.066 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi061.front.sepia.ceph.com 2023-12-06T18:25:17.066 DEBUG:teuthology.orchestra.run.smithi061:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-06T18:25:17.146 INFO:teuthology.orchestra.run.smithi061.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-06T18:25:17.146 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-06T18:25:17.146 INFO:teuthology.orchestra.run.smithi061.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-06T18:25:17.146 INFO:teuthology.orchestra.run.smithi061.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-06T18:25:17.146 INFO:teuthology.orchestra.run.smithi061.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-06T18:25:17.146 INFO:teuthology.orchestra.run.smithi061.stdout: = sunit=0 swidth=0 blks 2023-12-06T18:25:17.146 INFO:teuthology.orchestra.run.smithi061.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-06T18:25:17.146 INFO:teuthology.orchestra.run.smithi061.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-06T18:25:17.147 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-06T18:25:17.147 INFO:teuthology.orchestra.run.smithi061.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-06T18:25:17.602 INFO:teuthology.orchestra.run.smithi061.stdout:Discarding blocks...Done. 2023-12-06T18:25:17.604 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi061.front.sepia.ceph.com -o noatime 2023-12-06T18:25:17.604 DEBUG:teuthology.orchestra.run.smithi061:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-12-06T18:25:17.705 DEBUG:teuthology.orchestra.run.smithi061:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-12-06T18:25:17.746 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-12-06T18:25:17.819 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-06T18:25:17.819 INFO:tasks.ceph:role: osd.2 2023-12-06T18:25:17.819 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi061.front.sepia.ceph.com 2023-12-06T18:25:17.820 DEBUG:teuthology.orchestra.run.smithi061:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-06T18:25:17.900 INFO:teuthology.orchestra.run.smithi061.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-06T18:25:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-06T18:25:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-06T18:25:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-06T18:25:17.900 INFO:teuthology.orchestra.run.smithi061.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-06T18:25:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: = sunit=0 swidth=0 blks 2023-12-06T18:25:17.900 INFO:teuthology.orchestra.run.smithi061.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-06T18:25:17.900 INFO:teuthology.orchestra.run.smithi061.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-06T18:25:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-06T18:25:17.900 INFO:teuthology.orchestra.run.smithi061.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-06T18:25:18.390 INFO:teuthology.orchestra.run.smithi061.stdout:Discarding blocks...Done. 2023-12-06T18:25:18.392 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi061.front.sepia.ceph.com -o noatime 2023-12-06T18:25:18.392 DEBUG:teuthology.orchestra.run.smithi061:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-12-06T18:25:18.557 DEBUG:teuthology.orchestra.run.smithi061:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-12-06T18:25:18.598 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-12-06T18:25:18.671 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-06T18:25:18.671 INFO:tasks.ceph:role: osd.3 2023-12-06T18:25:18.672 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi061.front.sepia.ceph.com 2023-12-06T18:25:18.672 DEBUG:teuthology.orchestra.run.smithi061:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-06T18:25:18.751 INFO:teuthology.orchestra.run.smithi061.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-06T18:25:18.751 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-06T18:25:18.751 INFO:teuthology.orchestra.run.smithi061.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-06T18:25:18.751 INFO:teuthology.orchestra.run.smithi061.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-06T18:25:18.751 INFO:teuthology.orchestra.run.smithi061.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-06T18:25:18.751 INFO:teuthology.orchestra.run.smithi061.stdout: = sunit=0 swidth=0 blks 2023-12-06T18:25:18.751 INFO:teuthology.orchestra.run.smithi061.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-06T18:25:18.751 INFO:teuthology.orchestra.run.smithi061.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-06T18:25:18.751 INFO:teuthology.orchestra.run.smithi061.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-06T18:25:18.751 INFO:teuthology.orchestra.run.smithi061.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-06T18:25:19.258 INFO:teuthology.orchestra.run.smithi061.stdout:Discarding blocks...Done. 2023-12-06T18:25:19.260 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi061.front.sepia.ceph.com -o noatime 2023-12-06T18:25:19.260 DEBUG:teuthology.orchestra.run.smithi061:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-12-06T18:25:19.382 DEBUG:teuthology.orchestra.run.smithi061:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-12-06T18:25:19.422 DEBUG:teuthology.orchestra.run.smithi061:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-06T18:25:19.533 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:19.529+0000 7fa558a0c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-12-06T18:25:19.533 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:19.529+0000 7fa558a0c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-12-06T18:25:19.533 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:19.529+0000 7fa558a0c500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-06T18:25:19.533 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:19.529+0000 7fa558a0c500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-06T18:25:19.533 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:19.529+0000 7fa558a0c500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-06T18:25:19.534 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:19.530+0000 7fa558a0c500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-12-06T18:25:22.381 DEBUG:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-12-06T18:25:22.417 DEBUG:teuthology.orchestra.run.smithi061:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-06T18:25:22.513 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:22.509+0000 7fa7364c3500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-12-06T18:25:22.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:22.510+0000 7fa7364c3500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-12-06T18:25:22.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:22.510+0000 7fa7364c3500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-06T18:25:22.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:22.510+0000 7fa7364c3500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-06T18:25:22.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:22.510+0000 7fa7364c3500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-06T18:25:22.514 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:22.510+0000 7fa7364c3500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-12-06T18:25:25.354 DEBUG:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-12-06T18:25:25.389 DEBUG:teuthology.orchestra.run.smithi061:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-06T18:25:25.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:25.482+0000 7ff085e25500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-12-06T18:25:25.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:25.482+0000 7ff085e25500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-12-06T18:25:25.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:25.482+0000 7ff085e25500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-06T18:25:25.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:25.482+0000 7ff085e25500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-06T18:25:25.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:25.482+0000 7ff085e25500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-06T18:25:25.485 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:25.483+0000 7ff085e25500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-12-06T18:25:28.327 DEBUG:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-12-06T18:25:28.363 DEBUG:teuthology.orchestra.run.smithi061:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-06T18:25:28.459 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:28.456+0000 7f1611810500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-12-06T18:25:28.459 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:28.456+0000 7f1611810500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-12-06T18:25:28.460 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:28.456+0000 7f1611810500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-06T18:25:28.460 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:28.457+0000 7f1611810500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-06T18:25:28.460 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:28.457+0000 7f1611810500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-06T18:25:28.460 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:25:28.457+0000 7f1611810500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-12-06T18:25:31.305 DEBUG:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-12-06T18:25:31.341 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-12-06T18:25:31.372 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-06T18:25:31.372 INFO:tasks.ceph:role: osd.4 2023-12-06T18:25:31.372 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi152.front.sepia.ceph.com 2023-12-06T18:25:31.372 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-06T18:25:31.450 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-06T18:25:31.450 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-06T18:25:31.450 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-06T18:25:31.450 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-06T18:25:31.450 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-06T18:25:31.450 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2023-12-06T18:25:31.451 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-06T18:25:31.451 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-06T18:25:31.451 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-06T18:25:31.451 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-06T18:25:31.975 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2023-12-06T18:25:31.977 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2023-12-06T18:25:31.977 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-12-06T18:25:32.135 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-12-06T18:25:32.171 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-12-06T18:25:32.241 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-06T18:25:32.241 INFO:tasks.ceph:role: osd.5 2023-12-06T18:25:32.241 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi152.front.sepia.ceph.com 2023-12-06T18:25:32.241 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-06T18:25:32.318 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-06T18:25:32.318 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-06T18:25:32.318 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-06T18:25:32.318 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-06T18:25:32.318 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-06T18:25:32.318 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2023-12-06T18:25:32.318 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-06T18:25:32.318 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-06T18:25:32.318 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-06T18:25:32.318 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-06T18:25:32.770 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2023-12-06T18:25:32.772 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2023-12-06T18:25:32.773 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-12-06T18:25:32.879 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-12-06T18:25:32.915 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-12-06T18:25:32.984 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-06T18:25:32.985 INFO:tasks.ceph:role: osd.6 2023-12-06T18:25:32.985 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi152.front.sepia.ceph.com 2023-12-06T18:25:32.985 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-06T18:25:33.061 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-06T18:25:33.061 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-06T18:25:33.061 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-06T18:25:33.061 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-06T18:25:33.062 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-06T18:25:33.062 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2023-12-06T18:25:33.062 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-06T18:25:33.062 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-06T18:25:33.062 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-06T18:25:33.062 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-06T18:25:33.555 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2023-12-06T18:25:33.557 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2023-12-06T18:25:33.557 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-12-06T18:25:33.717 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-12-06T18:25:33.752 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-12-06T18:25:33.821 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-06T18:25:33.821 INFO:tasks.ceph:role: osd.7 2023-12-06T18:25:33.821 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi152.front.sepia.ceph.com 2023-12-06T18:25:33.821 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-06T18:25:33.897 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-06T18:25:33.897 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-06T18:25:33.898 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-06T18:25:33.898 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-06T18:25:33.898 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-06T18:25:33.898 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2023-12-06T18:25:33.898 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-06T18:25:33.898 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-06T18:25:33.898 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-06T18:25:33.898 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-06T18:25:34.407 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2023-12-06T18:25:34.409 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2023-12-06T18:25:34.409 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-12-06T18:25:34.530 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-12-06T18:25:34.567 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-06T18:25:34.692 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:34.689+0000 7f624c012500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-12-06T18:25:34.693 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:34.689+0000 7f624c012500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-12-06T18:25:34.693 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:34.689+0000 7f624c012500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-06T18:25:34.693 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:34.689+0000 7f624c012500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-06T18:25:34.693 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:34.689+0000 7f624c012500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-06T18:25:34.693 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:34.689+0000 7f624c012500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-12-06T18:25:37.538 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-12-06T18:25:37.570 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-06T18:25:37.661 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:37.657+0000 7fdb52649500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-12-06T18:25:37.661 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:37.657+0000 7fdb52649500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-12-06T18:25:37.661 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:37.657+0000 7fdb52649500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-06T18:25:37.661 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:37.657+0000 7fdb52649500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-06T18:25:37.661 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:37.657+0000 7fdb52649500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-06T18:25:37.662 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:37.658+0000 7fdb52649500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-12-06T18:25:40.502 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-12-06T18:25:40.534 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-06T18:25:40.627 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:40.623+0000 7fc855a0f500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-12-06T18:25:40.627 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:40.623+0000 7fc855a0f500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-12-06T18:25:40.628 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:40.624+0000 7fc855a0f500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-06T18:25:40.628 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:40.624+0000 7fc855a0f500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-06T18:25:40.628 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:40.624+0000 7fc855a0f500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-06T18:25:40.628 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:40.624+0000 7fc855a0f500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-12-06T18:25:43.485 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-12-06T18:25:43.518 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-06T18:25:43.609 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:43.605+0000 7f9cb2e10500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-12-06T18:25:43.609 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:43.605+0000 7f9cb2e10500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-12-06T18:25:43.609 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:43.605+0000 7f9cb2e10500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-06T18:25:43.609 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:43.605+0000 7f9cb2e10500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-06T18:25:43.610 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:43.605+0000 7f9cb2e10500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-06T18:25:43.610 INFO:teuthology.orchestra.run.smithi152.stderr:2023-12-06T18:25:43.606+0000 7f9cb2e10500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-12-06T18:25:46.456 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-12-06T18:25:46.487 INFO:tasks.ceph:Reading keys from all nodes... 2023-12-06T18:25:46.487 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:25:46.487 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-12-06T18:25:46.523 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:25:46.524 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-12-06T18:25:46.597 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:25:46.598 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-12-06T18:25:46.671 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:25:46.671 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-12-06T18:25:46.743 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:25:46.744 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-12-06T18:25:46.815 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:25:46.815 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-12-06T18:25:46.846 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:25:46.847 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-12-06T18:25:46.916 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:25:46.916 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-12-06T18:25:46.985 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:25:46.985 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-12-06T18:25:47.054 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:25:47.054 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-12-06T18:25:47.122 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:25:47.122 DEBUG:teuthology.orchestra.run.smithi061:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-12-06T18:25:47.140 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:25:47.140 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-12-06T18:25:47.178 INFO:tasks.ceph:Adding keys to all mons... 2023-12-06T18:25:47.178 DEBUG:teuthology.orchestra.run.smithi061:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-06T18:25:47.182 DEBUG:teuthology.orchestra.run.smithi152:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-06T18:25:47.222 INFO:teuthology.orchestra.run.smithi061.stdout:[mgr.y] 2023-12-06T18:25:47.222 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQCLvHBlyKY2OhAA+FfhwIh59r7uke6ccZzTug== 2023-12-06T18:25:47.222 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.0] 2023-12-06T18:25:47.222 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQCPvHBld92gHxAADEpnN+LkjU1rc4otDi5d4w== 2023-12-06T18:25:47.223 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.1] 2023-12-06T18:25:47.223 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQCSvHBlrzlzHhAAvxIh0beJspx6bskIgn5gkw== 2023-12-06T18:25:47.223 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.2] 2023-12-06T18:25:47.223 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQCVvHBlIMXBHBAAk6ve47aMO32KrVm1Dgp7ww== 2023-12-06T18:25:47.223 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.3] 2023-12-06T18:25:47.223 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQCYvHBlfmQ7GxAA4qozIzdsac96wzqn7ZKpaQ== 2023-12-06T18:25:47.223 INFO:teuthology.orchestra.run.smithi061.stdout:[mgr.x] 2023-12-06T18:25:47.223 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQCMvHBlcB/2AhAAKiWNUKFzYzRgoQdo9bhPaA== 2023-12-06T18:25:47.223 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.4] 2023-12-06T18:25:47.223 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQCevHBl++EhKRAAV05O8ioFvRG6X5EDs51KbQ== 2023-12-06T18:25:47.223 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.5] 2023-12-06T18:25:47.223 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQChvHBlXLM+JxAAKfa9eGbxeiDF84V0/KSmbg== 2023-12-06T18:25:47.223 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.6] 2023-12-06T18:25:47.224 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQCkvHBlMaQ8JRAAUCQPbXEaNyYEsz96kzCTHQ== 2023-12-06T18:25:47.224 INFO:teuthology.orchestra.run.smithi061.stdout:[osd.7] 2023-12-06T18:25:47.224 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQCnvHBlsConJBAAhgbgkAr7Wys0hoxHRJgzbg== 2023-12-06T18:25:47.224 INFO:teuthology.orchestra.run.smithi061.stdout:[client.0] 2023-12-06T18:25:47.224 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQCMvHBlUYDoBRAAWLn+ivW25jThLLftywrU2w== 2023-12-06T18:25:47.224 INFO:teuthology.orchestra.run.smithi061.stdout:[client.1] 2023-12-06T18:25:47.224 INFO:teuthology.orchestra.run.smithi061.stdout: key = AQCMvHBl0R6ICRAADgARPxchHtIp6p2hbP5kjQ== 2023-12-06T18:25:47.244 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr.y] 2023-12-06T18:25:47.245 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCLvHBlyKY2OhAA+FfhwIh59r7uke6ccZzTug== 2023-12-06T18:25:47.245 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.0] 2023-12-06T18:25:47.245 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCPvHBld92gHxAADEpnN+LkjU1rc4otDi5d4w== 2023-12-06T18:25:47.245 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.1] 2023-12-06T18:25:47.245 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCSvHBlrzlzHhAAvxIh0beJspx6bskIgn5gkw== 2023-12-06T18:25:47.245 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.2] 2023-12-06T18:25:47.245 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCVvHBlIMXBHBAAk6ve47aMO32KrVm1Dgp7ww== 2023-12-06T18:25:47.245 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.3] 2023-12-06T18:25:47.245 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCYvHBlfmQ7GxAA4qozIzdsac96wzqn7ZKpaQ== 2023-12-06T18:25:47.245 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr.x] 2023-12-06T18:25:47.245 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCMvHBlcB/2AhAAKiWNUKFzYzRgoQdo9bhPaA== 2023-12-06T18:25:47.245 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.4] 2023-12-06T18:25:47.246 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCevHBl++EhKRAAV05O8ioFvRG6X5EDs51KbQ== 2023-12-06T18:25:47.246 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.5] 2023-12-06T18:25:47.246 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQChvHBlXLM+JxAAKfa9eGbxeiDF84V0/KSmbg== 2023-12-06T18:25:47.246 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.6] 2023-12-06T18:25:47.246 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCkvHBlMaQ8JRAAUCQPbXEaNyYEsz96kzCTHQ== 2023-12-06T18:25:47.246 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.7] 2023-12-06T18:25:47.246 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCnvHBlsConJBAAhgbgkAr7Wys0hoxHRJgzbg== 2023-12-06T18:25:47.246 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2023-12-06T18:25:47.246 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCMvHBlUYDoBRAAWLn+ivW25jThLLftywrU2w== 2023-12-06T18:25:47.246 INFO:teuthology.orchestra.run.smithi152.stdout:[client.1] 2023-12-06T18:25:47.246 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCMvHBl0R6ICRAADgARPxchHtIp6p2hbP5kjQ== 2023-12-06T18:25:47.247 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-06T18:25:47.268 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-06T18:25:47.339 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-06T18:25:47.342 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-06T18:25:47.397 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-06T18:25:47.400 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-06T18:25:47.454 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-06T18:25:47.456 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-06T18:25:47.509 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-06T18:25:47.512 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-06T18:25:47.565 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-06T18:25:47.567 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-06T18:25:47.620 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-06T18:25:47.623 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-06T18:25:47.676 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-06T18:25:47.679 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-06T18:25:47.732 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-06T18:25:47.735 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-06T18:25:47.788 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-06T18:25:47.790 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-06T18:25:47.843 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-06T18:25:47.845 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-06T18:25:47.902 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-06T18:25:47.904 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-06T18:25:47.960 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-12-06T18:25:47.961 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-12-06T18:25:47.991 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-06T18:25:48.092 DEBUG:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-12-06T18:25:48.123 DEBUG:teuthology.orchestra.run.smithi061:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-12-06T18:25:48.193 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-06T18:25:48.294 DEBUG:teuthology.orchestra.run.smithi061:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-12-06T18:25:48.325 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-12-06T18:25:48.353 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-06T18:25:48.451 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-12-06T18:25:48.478 DEBUG:teuthology.orchestra.run.smithi061:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-06T18:25:48.480 DEBUG:teuthology.orchestra.run.smithi152:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-06T18:25:48.533 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-12-06T18:25:48.533 DEBUG:tasks.ceph_manager:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2023-12-06T18:25:48.533 INFO:tasks.ceph.mon.a:Restarting daemon 2023-12-06T18:25:48.534 DEBUG:teuthology.orchestra.run.smithi061:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2023-12-06T18:25:48.536 INFO:tasks.ceph.mon.a:Started 2023-12-06T18:25:48.536 DEBUG:tasks.ceph_manager:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2023-12-06T18:25:48.536 INFO:tasks.ceph.mon.c:Restarting daemon 2023-12-06T18:25:48.536 DEBUG:teuthology.orchestra.run.smithi061:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2023-12-06T18:25:48.538 INFO:tasks.ceph.mon.c:Started 2023-12-06T18:25:48.538 DEBUG:tasks.ceph_manager:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2023-12-06T18:25:48.538 INFO:tasks.ceph.mon.b:Restarting daemon 2023-12-06T18:25:48.538 DEBUG:teuthology.orchestra.run.smithi152:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2023-12-06T18:25:48.576 INFO:tasks.ceph.mon.b:Started 2023-12-06T18:25:48.576 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-12-06T18:25:48.576 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-12-06T18:25:48.576 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2023-12-06T18:25:48.578 INFO:tasks.ceph.mgr.y:Started 2023-12-06T18:25:48.579 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-12-06T18:25:48.579 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2023-12-06T18:25:48.581 INFO:tasks.ceph.mgr.x:Started 2023-12-06T18:25:48.581 INFO:tasks.ceph:Setting crush tunables to default 2023-12-06T18:25:48.581 DEBUG:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd crush tunables default 2023-12-06T18:25:56.149 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:07.469 86580== Warning: unimplemented fcntl command: 1036 2023-12-06T18:25:56.462 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:07.783 86580== Warning: unimplemented fcntl command: 1036 2023-12-06T18:25:56.792 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:08.131 86952== Warning: unimplemented fcntl command: 1036 2023-12-06T18:25:56.793 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:08.131 86953== Warning: unimplemented fcntl command: 1036 2023-12-06T18:25:57.124 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:08.463 86952== Warning: unimplemented fcntl command: 1036 2023-12-06T18:25:57.125 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:08.463 86953== Warning: unimplemented fcntl command: 1036 2023-12-06T18:26:08.976 INFO:teuthology.orchestra.run.smithi061.stderr:adjusted tunables profile to default 2023-12-06T18:26:08.989 INFO:tasks.ceph:check_enable_crimson: False 2023-12-06T18:26:08.989 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-12-06T18:26:08.989 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:26:08.989 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-12-06T18:26:09.020 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:26:09.020 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-12-06T18:26:09.092 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:26:09.092 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-12-06T18:26:09.096 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:09.093+0000 7f05d76fa040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-06T18:26:09.130 DEBUG:teuthology.orchestra.run.smithi061:> set -ex 2023-12-06T18:26:09.131 DEBUG:teuthology.orchestra.run.smithi061:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-12-06T18:26:09.158 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:09.154+0000 7f05d76fa040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-06T18:26:09.190 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:26:09.190 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-12-06T18:26:09.217 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:26:09.217 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-12-06T18:26:09.284 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:26:09.284 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-12-06T18:26:09.354 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2023-12-06T18:26:09.354 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-12-06T18:26:09.416 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 5e6e17b9-e72e-4e34-a531-0830d659bfe0 0 2023-12-06T18:26:09.710 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:09.707+0000 7f05d76fa040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-06T18:26:09.765 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:09.761+0000 7f05d76fa040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-06T18:26:09.812 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:09.809+0000 7f05d76fa040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-06T18:26:09.872 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:09.868+0000 7f05d76fa040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-06T18:26:09.918 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:09.914+0000 7f05d76fa040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-06T18:26:09.918 INFO:teuthology.orchestra.run.smithi152.stdout:0 2023-12-06T18:26:09.930 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new b78c49b0-360d-4351-b44e-65ccaaeb768f 1 2023-12-06T18:26:10.026 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:10.022+0000 7f05d76fa040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-06T18:26:10.103 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:10.099+0000 7f05d76fa040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-06T18:26:10.151 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:10.147+0000 7f05d76fa040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-06T18:26:10.207 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:10.203+0000 7f05d76fa040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-06T18:26:10.208 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:10.204+0000 7fab29417040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-06T18:26:10.259 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:10.255+0000 7f05d76fa040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-06T18:26:10.276 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:10.272+0000 7fab29417040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-06T18:26:10.319 INFO:teuthology.orchestra.run.smithi152.stdout:1 2023-12-06T18:26:10.331 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new a2af3151-a4da-4c34-8fb6-78abf035785b 2 2023-12-06T18:26:10.674 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:10.670+0000 7f05d76fa040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-06T18:26:10.681 INFO:teuthology.orchestra.run.smithi152.stdout:2 2023-12-06T18:26:10.693 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 1c754ed6-c535-454f-b35b-00f62cd6ce5a 3 2023-12-06T18:26:10.822 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:10.819+0000 7f05d76fa040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-06T18:26:10.836 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:10.832+0000 7fab29417040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-06T18:26:10.888 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:10.884+0000 7fab29417040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-06T18:26:10.934 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:10.930+0000 7fab29417040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-06T18:26:10.957 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:10.954+0000 7f05d76fa040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-06T18:26:10.989 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:10.985+0000 7fab29417040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-06T18:26:11.005 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:11.002+0000 7f05d76fa040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-06T18:26:11.032 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:11.028+0000 7fab29417040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-06T18:26:11.032 INFO:teuthology.orchestra.run.smithi152.stdout:3 2023-12-06T18:26:11.044 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 766572af-b9ca-4f38-927d-36f26101ba51 4 2023-12-06T18:26:11.112 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:11.108+0000 7f05d76fa040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-06T18:26:11.141 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:11.137+0000 7fab29417040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-06T18:26:11.215 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:11.211+0000 7fab29417040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-06T18:26:11.223 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:11.212+0000 7f05d76fa040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-06T18:26:11.261 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:11.257+0000 7fab29417040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-06T18:26:11.315 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:11.311+0000 7fab29417040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-06T18:26:11.366 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:11.362+0000 7fab29417040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-06T18:26:11.373 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:11.370+0000 7f05d76fa040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-06T18:26:11.393 INFO:teuthology.orchestra.run.smithi152.stdout:4 2023-12-06T18:26:11.407 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 7a566b60-1543-41d7-894c-e2fbb63e48dc 5 2023-12-06T18:26:11.420 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:11.417+0000 7f05d76fa040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-06T18:26:11.572 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:11.569+0000 7f05d76fa040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-06T18:26:11.766 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:11.762+0000 7fab29417040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-06T18:26:11.790 INFO:teuthology.orchestra.run.smithi152.stdout:5 2023-12-06T18:26:11.801 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 28cc2358-4ac4-40b2-91a4-20edfe625a73 6 2023-12-06T18:26:11.908 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:11.905+0000 7fab29417040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-06T18:26:12.019 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:12.015+0000 7fab29417040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-06T18:26:12.067 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:12.063+0000 7fab29417040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-06T18:26:12.168 INFO:teuthology.orchestra.run.smithi152.stdout:6 2023-12-06T18:26:12.168 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:12.165+0000 7fab29417040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-06T18:26:12.180 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 8f79279b-d2d0-438e-b3c2-88df1241ff30 7 2023-12-06T18:26:12.269 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:12.266+0000 7fab29417040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-06T18:26:12.411 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:12.408+0000 7fab29417040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-06T18:26:12.433 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:12.429+0000 7f05d76fa040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-06T18:26:12.452 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:12.449+0000 7fab29417040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-06T18:26:12.595 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:12.592+0000 7fab29417040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-06T18:26:12.722 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:12.718+0000 7f05d76fa040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-06T18:26:12.723 INFO:teuthology.orchestra.run.smithi152.stdout:7 2023-12-06T18:26:12.735 DEBUG:tasks.ceph_manager:running osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.0.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-12-06T18:26:12.735 INFO:tasks.ceph.osd.0:Restarting daemon 2023-12-06T18:26:12.735 DEBUG:teuthology.orchestra.run.smithi061:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 2023-12-06T18:26:12.738 INFO:tasks.ceph.osd.0:Started 2023-12-06T18:26:12.738 DEBUG:tasks.ceph_manager:running osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.1.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-12-06T18:26:12.738 INFO:tasks.ceph.osd.1:Restarting daemon 2023-12-06T18:26:12.738 DEBUG:teuthology.orchestra.run.smithi061:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 2023-12-06T18:26:12.740 INFO:tasks.ceph.osd.1:Started 2023-12-06T18:26:12.740 DEBUG:tasks.ceph_manager:running osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.2.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-12-06T18:26:12.740 INFO:tasks.ceph.osd.2:Restarting daemon 2023-12-06T18:26:12.740 DEBUG:teuthology.orchestra.run.smithi061:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 2023-12-06T18:26:12.742 INFO:tasks.ceph.osd.2:Started 2023-12-06T18:26:12.742 DEBUG:tasks.ceph_manager:running osd.3 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.3.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-12-06T18:26:12.742 INFO:tasks.ceph.osd.3:Restarting daemon 2023-12-06T18:26:12.743 DEBUG:teuthology.orchestra.run.smithi061:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 2023-12-06T18:26:12.745 INFO:tasks.ceph.osd.3:Started 2023-12-06T18:26:12.745 DEBUG:tasks.ceph_manager:running osd.4 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.4.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-12-06T18:26:12.745 INFO:tasks.ceph.osd.4:Restarting daemon 2023-12-06T18:26:12.745 DEBUG:teuthology.orchestra.run.smithi152:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 2023-12-06T18:26:12.747 INFO:tasks.ceph.osd.4:Started 2023-12-06T18:26:12.747 DEBUG:tasks.ceph_manager:running osd.5 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.5.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-12-06T18:26:12.747 INFO:tasks.ceph.osd.5:Restarting daemon 2023-12-06T18:26:12.747 DEBUG:teuthology.orchestra.run.smithi152:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 2023-12-06T18:26:12.749 INFO:tasks.ceph.osd.5:Started 2023-12-06T18:26:12.749 DEBUG:tasks.ceph_manager:running osd.6 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.6.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-12-06T18:26:12.750 INFO:tasks.ceph.osd.6:Restarting daemon 2023-12-06T18:26:12.750 DEBUG:teuthology.orchestra.run.smithi152:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 2023-12-06T18:26:12.752 INFO:tasks.ceph.osd.6:Started 2023-12-06T18:26:12.752 DEBUG:tasks.ceph_manager:running osd.7 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.7.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2023-12-06T18:26:12.752 INFO:tasks.ceph.osd.7:Restarting daemon 2023-12-06T18:26:12.752 DEBUG:teuthology.orchestra.run.smithi152:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 2023-12-06T18:26:12.754 INFO:tasks.ceph.osd.7:Started 2023-12-06T18:26:12.757 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-06T18:26:12.790 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:12.786+0000 7f05d76fa040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-06T18:26:12.966 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:12.963+0000 7f05d76fa040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-06T18:26:13.521 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:13.517+0000 7f05d269a640 -1 mgr handle_mgr_map I was active but no longer am 2023-12-06T18:26:13.681 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:13.678+0000 7fab29417040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-06T18:26:13.800 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:26:13.800 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":10,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","created":"2023-12-06T18:26:07.122862+0000","modified":"2023-12-06T18:26:12.654320+0000","last_up_change":"0.000000","last_in_change":"2023-12-06T18:26:12.654320+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5e6e17b9-e72e-4e34-a531-0830d659bfe0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b78c49b0-360d-4351-b44e-65ccaaeb768f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a2af3151-a4da-4c34-8fb6-78abf035785b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1c754ed6-c535-454f-b35b-00f62cd6ce5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"766572af-b9ca-4f38-927d-36f26101ba51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7a566b60-1543-41d7-894c-e2fbb63e48dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"28cc2358-4ac4-40b2-91a4-20edfe625a73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8f79279b-d2d0-438e-b3c2-88df1241ff30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-06T18:26:13.818 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-12-06T18:26:13.819 INFO:tasks.ceph:Waiting for OSDs to come up 2023-12-06T18:26:13.994 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:13.990+0000 7fab29417040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-06T18:26:14.062 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:14.059+0000 7fab29417040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-06T18:26:14.175 INFO:tasks.ceph.mgr.x.smithi152.stderr:2023-12-06T18:26:14.172+0000 7fab29417040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-06T18:26:14.252 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:14.248+0000 7f4711613040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-06T18:26:14.332 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:14.328+0000 7f4711613040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-06T18:26:14.625 DEBUG:teuthology.orchestra.run.smithi061:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-06T18:26:15.069 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:15.065+0000 7f4711613040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-06T18:26:15.149 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:15.145+0000 7f4711613040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-06T18:26:15.224 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:15.221+0000 7f4711613040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-06T18:26:15.255 INFO:teuthology.misc.health.smithi061.stdout: 2023-12-06T18:26:15.256 INFO:teuthology.misc.health.smithi061.stdout:{"epoch":10,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","created":"2023-12-06T18:26:07.122862+0000","modified":"2023-12-06T18:26:12.654320+0000","last_up_change":"0.000000","last_in_change":"2023-12-06T18:26:12.654320+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5e6e17b9-e72e-4e34-a531-0830d659bfe0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b78c49b0-360d-4351-b44e-65ccaaeb768f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a2af3151-a4da-4c34-8fb6-78abf035785b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1c754ed6-c535-454f-b35b-00f62cd6ce5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"766572af-b9ca-4f38-927d-36f26101ba51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7a566b60-1543-41d7-894c-e2fbb63e48dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"28cc2358-4ac4-40b2-91a4-20edfe625a73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8f79279b-d2d0-438e-b3c2-88df1241ff30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-06T18:26:15.272 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-06T18:26:15.319 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:15.315+0000 7f4711613040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-06T18:26:15.393 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:15.390+0000 7f4711613040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-06T18:26:15.591 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:15.587+0000 7f4711613040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-06T18:26:15.698 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:15.694+0000 7f4711613040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-06T18:26:15.777 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:15.774+0000 7f4711613040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-06T18:26:15.876 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:15.872+0000 7f4711613040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-06T18:26:15.961 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:15.958+0000 7f4711613040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-06T18:26:16.542 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:16.539+0000 7f4711613040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-06T18:26:16.778 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:16.774+0000 7f4711613040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-06T18:26:16.941 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:16.937+0000 7f4711613040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-06T18:26:17.018 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:17.014+0000 7f4711613040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-06T18:26:17.222 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:17.219+0000 7f4711613040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-06T18:26:17.382 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:17.379+0000 7f4711613040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-06T18:26:17.631 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:17.627+0000 7f4711613040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-06T18:26:17.701 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:17.697+0000 7f4711613040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-06T18:26:17.887 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:17.883+0000 7f4711613040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-06T18:26:18.972 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:18.968+0000 7f4711613040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-06T18:26:19.465 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:19.461+0000 7f4711613040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-06T18:26:19.551 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:19.547+0000 7f4711613040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-06T18:26:19.740 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:19.736+0000 7f4711613040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-06T18:26:21.216 INFO:tasks.ceph.osd.4.smithi152.stderr:==00:00:00:08.402 87113== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:21.302 INFO:tasks.ceph.osd.4.smithi152.stderr:==00:00:00:08.488 87113== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:21.409 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:08.589 87115== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:21.468 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:08.648 87115== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:21.661 INFO:tasks.ceph.osd.2.smithi061.stderr:==00:00:00:08.827 87263== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:21.665 INFO:tasks.ceph.osd.6.smithi152.stderr:==00:00:00:08.848 87114== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:21.678 INFO:tasks.ceph.osd.5.smithi152.stderr:==00:00:00:08.843 87116== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:21.725 INFO:tasks.ceph.osd.6.smithi152.stderr:==00:00:00:08.907 87114== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:21.725 INFO:tasks.ceph.osd.0.smithi061.stderr:==00:00:00:08.891 87262== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:21.737 INFO:tasks.ceph.osd.2.smithi061.stderr:==00:00:00:08.903 87263== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:21.742 INFO:tasks.ceph.osd.5.smithi152.stderr:==00:00:00:08.907 87116== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:21.746 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:21.743+0000 7f46b2672640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-12-06T18:26:21.790 INFO:tasks.ceph.osd.0.smithi061.stderr:==00:00:00:08.956 87262== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:21.828 INFO:tasks.ceph.osd.1.smithi061.stderr:==00:00:00:08.995 87264== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:21.837 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:26:21.828+0000 5dc2340 -1 Falling back to public interface 2023-12-06T18:26:21.908 INFO:tasks.ceph.osd.1.smithi061.stderr:==00:00:00:09.075 87264== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:21.955 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:26:21.952+0000 7f46b2672640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-12-06T18:26:22.027 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:26:22.018+0000 5dc2340 -1 Falling back to public interface 2023-12-06T18:26:22.079 DEBUG:teuthology.orchestra.run.smithi061:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-06T18:26:22.246 INFO:tasks.ceph.osd.4.smithi152.stderr:==00:00:00:09.432 87113== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:22.270 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:26:22.260+0000 5dc2340 -1 Falling back to public interface 2023-12-06T18:26:22.324 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:26:22.313+0000 5dc2340 -1 Falling back to public interface 2023-12-06T18:26:22.327 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:26:22.315+0000 5dc2340 -1 Falling back to public interface 2023-12-06T18:26:22.330 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:26:22.325+0000 8c79640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2023-12-06T18:26:22.348 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:26:22.338+0000 5dc2340 -1 Falling back to public interface 2023-12-06T18:26:22.485 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:09.665 87115== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:22.489 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:26:22.477+0000 5dc2340 -1 Falling back to public interface 2023-12-06T18:26:22.614 INFO:tasks.ceph.osd.4.smithi152.stderr:==00:00:00:09.801 87113== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:22.732 INFO:tasks.ceph.osd.5.smithi152.stderr:==00:00:00:09.896 87116== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:22.742 INFO:tasks.ceph.osd.6.smithi152.stderr:==00:00:00:09.924 87114== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:22.768 INFO:teuthology.misc.health.smithi061.stdout: 2023-12-06T18:26:22.768 INFO:teuthology.misc.health.smithi061.stdout:{"epoch":11,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","created":"2023-12-06T18:26:07.122862+0000","modified":"2023-12-06T18:26:19.860888+0000","last_up_change":"0.000000","last_in_change":"2023-12-06T18:26:12.654320+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5e6e17b9-e72e-4e34-a531-0830d659bfe0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b78c49b0-360d-4351-b44e-65ccaaeb768f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a2af3151-a4da-4c34-8fb6-78abf035785b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1c754ed6-c535-454f-b35b-00f62cd6ce5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"766572af-b9ca-4f38-927d-36f26101ba51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7a566b60-1543-41d7-894c-e2fbb63e48dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"28cc2358-4ac4-40b2-91a4-20edfe625a73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8f79279b-d2d0-438e-b3c2-88df1241ff30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-06T18:26:22.777 INFO:tasks.ceph.osd.0.smithi061.stderr:==00:00:00:09.943 87262== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:22.780 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-06T18:26:22.794 INFO:tasks.ceph.osd.2.smithi061.stderr:==00:00:00:09.960 87263== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:22.857 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:10.035 87115== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:22.928 INFO:tasks.ceph.osd.4.smithi152.stderr:==00:00:00:10.114 87113== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.056 INFO:tasks.ceph.osd.1.smithi061.stderr:==00:00:00:10.222 87264== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.091 INFO:tasks.ceph.osd.5.smithi152.stderr:==00:00:00:10.255 87116== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.101 INFO:tasks.ceph.osd.6.smithi152.stderr:==00:00:00:10.284 87114== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.144 INFO:tasks.ceph.osd.0.smithi061.stderr:==00:00:00:10.310 87262== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.174 INFO:tasks.ceph.osd.5.smithi152.stderr:==00:00:00:10.339 87116== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.177 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:10.357 87115== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.186 INFO:tasks.ceph.osd.2.smithi061.stderr:==00:00:00:10.352 87263== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.197 INFO:tasks.ceph.osd.4.smithi152.stderr:==00:00:00:10.383 87113== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.417 INFO:tasks.ceph.osd.6.smithi152.stderr:==00:00:00:10.600 87114== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.419 INFO:tasks.ceph.osd.1.smithi061.stderr:==00:00:00:10.586 87264== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.443 INFO:tasks.ceph.osd.5.smithi152.stderr:==00:00:00:10.608 87116== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.443 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:10.623 87115== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.465 INFO:tasks.ceph.osd.4.smithi152.stderr:==00:00:00:10.651 87113== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.469 INFO:tasks.ceph.osd.0.smithi061.stderr:==00:00:00:10.634 87262== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.516 INFO:tasks.ceph.osd.2.smithi061.stderr:==00:00:00:10.681 87263== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.693 INFO:tasks.ceph.osd.6.smithi152.stderr:==00:00:00:10.876 87114== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.720 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:10.900 87115== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.722 INFO:tasks.ceph.osd.5.smithi152.stderr:==00:00:00:10.888 87116== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.735 INFO:tasks.ceph.osd.1.smithi061.stderr:==00:00:00:10.902 87264== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.738 INFO:tasks.ceph.osd.0.smithi061.stderr:==00:00:00:10.903 87262== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.742 INFO:tasks.ceph.osd.4.smithi152.stderr:==00:00:00:10.928 87113== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.795 INFO:tasks.ceph.osd.2.smithi061.stderr:==00:00:00:10.960 87263== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.956 INFO:tasks.ceph.osd.6.smithi152.stderr:==00:00:00:11.138 87114== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:23.993 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:11.173 87115== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.003 INFO:tasks.ceph.osd.5.smithi152.stderr:==00:00:00:11.168 87116== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.023 INFO:tasks.ceph.osd.1.smithi061.stderr:==00:00:00:11.189 87264== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.027 INFO:tasks.ceph.osd.0.smithi061.stderr:==00:00:00:11.193 87262== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.030 INFO:tasks.ceph.osd.4.smithi152.stderr:==00:00:00:11.217 87113== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.041 INFO:tasks.ceph.osd.3.smithi061.stderr:==00:00:00:11.222 87261== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.087 INFO:tasks.ceph.osd.2.smithi061.stderr:==00:00:00:11.252 87263== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.102 INFO:tasks.ceph.osd.3.smithi061.stderr:==00:00:00:11.284 87261== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.235 INFO:tasks.ceph.osd.6.smithi152.stderr:==00:00:00:11.418 87114== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.273 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:11.453 87115== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.293 INFO:tasks.ceph.osd.5.smithi152.stderr:==00:00:00:11.458 87116== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.315 INFO:tasks.ceph.osd.1.smithi061.stderr:==00:00:00:11.482 87264== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.320 INFO:tasks.ceph.osd.0.smithi061.stderr:==00:00:00:11.486 87262== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.324 INFO:tasks.ceph.osd.4.smithi152.stderr:==00:00:00:11.511 87113== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.377 INFO:tasks.ceph.osd.2.smithi061.stderr:==00:00:00:11.542 87263== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.524 INFO:tasks.ceph.osd.6.smithi152.stderr:==00:00:00:11.706 87114== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.572 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:11.752 87115== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.589 INFO:tasks.ceph.osd.5.smithi152.stderr:==00:00:00:11.754 87116== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.614 INFO:tasks.ceph.osd.1.smithi061.stderr:==00:00:00:11.781 87264== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.617 INFO:tasks.ceph.osd.4.smithi152.stderr:==00:00:00:11.804 87113== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.620 INFO:tasks.ceph.osd.0.smithi061.stderr:==00:00:00:11.786 87262== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.670 INFO:tasks.ceph.osd.2.smithi061.stderr:==00:00:00:11.835 87263== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.687 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:26:24.679+0000 5dc2340 -1 Falling back to public interface 2023-12-06T18:26:24.811 INFO:tasks.ceph.osd.6.smithi152.stderr:==00:00:00:11.994 87114== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.860 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:12.040 87115== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.887 INFO:tasks.ceph.osd.5.smithi152.stderr:==00:00:00:12.052 87116== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.906 INFO:tasks.ceph.osd.1.smithi061.stderr:==00:00:00:12.073 87264== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.923 INFO:tasks.ceph.osd.0.smithi061.stderr:==00:00:00:12.088 87262== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:24.959 INFO:tasks.ceph.osd.2.smithi061.stderr:==00:00:00:12.125 87263== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.103 INFO:tasks.ceph.osd.6.smithi152.stderr:==00:00:00:12.285 87114== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.119 INFO:tasks.ceph.osd.3.smithi061.stderr:==00:00:00:12.300 87261== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.156 INFO:tasks.ceph.osd.4.smithi152.stderr:==00:00:00:12.342 87113== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.196 INFO:tasks.ceph.osd.1.smithi061.stderr:==00:00:00:12.362 87264== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.210 INFO:tasks.ceph.osd.4.smithi152.stderr:==00:00:00:12.397 87113== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.240 INFO:tasks.ceph.osd.0.smithi061.stderr:==00:00:00:12.405 87262== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.258 INFO:tasks.ceph.osd.2.smithi061.stderr:==00:00:00:12.424 87263== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.392 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:12.572 87115== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.411 INFO:tasks.ceph.osd.5.smithi152.stderr:==00:00:00:12.576 87116== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.447 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:12.627 87115== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.466 INFO:tasks.ceph.osd.5.smithi152.stderr:==00:00:00:12.631 87116== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.481 INFO:tasks.ceph.osd.3.smithi061.stderr:==00:00:00:12.662 87261== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.485 INFO:tasks.ceph.osd.1.smithi061.stderr:==00:00:00:12.652 87264== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.684 INFO:tasks.ceph.osd.6.smithi152.stderr:==00:00:00:12.867 87114== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.738 INFO:tasks.ceph.osd.6.smithi152.stderr:==00:00:00:12.921 87114== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.778 INFO:tasks.ceph.osd.0.smithi061.stderr:==00:00:00:12.944 87262== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.809 INFO:tasks.ceph.osd.2.smithi061.stderr:==00:00:00:12.974 87263== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.861 INFO:tasks.ceph.osd.3.smithi061.stderr:==00:00:00:13.043 87261== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.863 INFO:tasks.ceph.osd.0.smithi061.stderr:==00:00:00:13.029 87262== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:25.884 INFO:tasks.ceph.osd.2.smithi061.stderr:==00:00:00:13.050 87263== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:26.035 INFO:tasks.ceph.osd.1.smithi061.stderr:==00:00:00:13.202 87264== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:26.113 INFO:tasks.ceph.osd.1.smithi061.stderr:==00:00:00:13.279 87264== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:26.173 INFO:tasks.ceph.osd.3.smithi061.stderr:==00:00:00:13.354 87261== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:26.217 INFO:tasks.ceph.osd.3.smithi061.stderr:==00:00:00:13.399 87261== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:26.510 INFO:tasks.ceph.osd.3.smithi061.stderr:==00:00:00:13.692 87261== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:26.833 INFO:tasks.ceph.osd.3.smithi061.stderr:==00:00:00:14.015 87261== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:27.141 INFO:tasks.ceph.osd.3.smithi061.stderr:==00:00:00:14.323 87261== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:27.411 INFO:tasks.ceph.osd.3.smithi061.stderr:==00:00:00:14.592 87261== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:27.950 INFO:tasks.ceph.osd.3.smithi061.stderr:==00:00:00:15.131 87261== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:28.006 INFO:tasks.ceph.osd.3.smithi061.stderr:==00:00:00:15.187 87261== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:29.586 DEBUG:teuthology.orchestra.run.smithi061:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-06T18:26:30.307 INFO:teuthology.misc.health.smithi061.stdout: 2023-12-06T18:26:30.307 INFO:teuthology.misc.health.smithi061.stdout:{"epoch":11,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","created":"2023-12-06T18:26:07.122862+0000","modified":"2023-12-06T18:26:19.860888+0000","last_up_change":"0.000000","last_in_change":"2023-12-06T18:26:12.654320+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5e6e17b9-e72e-4e34-a531-0830d659bfe0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b78c49b0-360d-4351-b44e-65ccaaeb768f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a2af3151-a4da-4c34-8fb6-78abf035785b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1c754ed6-c535-454f-b35b-00f62cd6ce5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"766572af-b9ca-4f38-927d-36f26101ba51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7a566b60-1543-41d7-894c-e2fbb63e48dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"28cc2358-4ac4-40b2-91a4-20edfe625a73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8f79279b-d2d0-438e-b3c2-88df1241ff30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-06T18:26:30.326 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-06T18:26:30.856 INFO:tasks.ceph.osd.4.smithi152.stderr:==00:00:00:18.042 87113== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:31.089 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:18.268 87115== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:31.299 INFO:tasks.ceph.osd.6.smithi152.stderr:==00:00:00:18.481 87114== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:31.386 INFO:tasks.ceph.osd.5.smithi152.stderr:==00:00:00:18.551 87116== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:31.427 INFO:tasks.ceph.osd.2.smithi061.stderr:==00:00:00:18.592 87263== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:31.653 INFO:tasks.ceph.osd.0.smithi061.stderr:==00:00:00:18.818 87262== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:31.966 INFO:tasks.ceph.osd.1.smithi061.stderr:==00:00:00:19.132 87264== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:33.501 INFO:tasks.ceph.osd.3.smithi061.stderr:==00:00:00:20.681 87261== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:35.579 INFO:tasks.ceph.osd.4.smithi152.stderr:==00:00:00:22.765 87113== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:35.582 INFO:tasks.ceph.osd.4.smithi152.stderr:==00:00:00:22.769 87113== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:35.620 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:22.799 87115== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:35.623 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:22.803 87115== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:35.931 INFO:tasks.ceph.osd.6.smithi152.stderr:==00:00:00:23.113 87114== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:35.934 INFO:tasks.ceph.osd.6.smithi152.stderr:==00:00:00:23.117 87114== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:36.039 INFO:tasks.ceph.osd.5.smithi152.stderr:==00:00:00:23.204 87116== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:36.045 INFO:tasks.ceph.osd.5.smithi152.stderr:==00:00:00:23.210 87116== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:36.693 INFO:tasks.ceph.osd.2.smithi061.stderr:==00:00:00:23.858 87263== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:36.697 INFO:tasks.ceph.osd.2.smithi061.stderr:==00:00:00:23.862 87263== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:36.764 INFO:tasks.ceph.osd.0.smithi061.stderr:==00:00:00:23.930 87262== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:36.771 INFO:tasks.ceph.osd.0.smithi061.stderr:==00:00:00:23.936 87262== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:37.132 DEBUG:teuthology.orchestra.run.smithi061:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-06T18:26:37.141 INFO:tasks.ceph.osd.1.smithi061.stderr:==00:00:00:24.307 87264== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:37.147 INFO:tasks.ceph.osd.1.smithi061.stderr:==00:00:00:24.313 87264== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:37.984 INFO:teuthology.misc.health.smithi061.stdout: 2023-12-06T18:26:37.984 INFO:teuthology.misc.health.smithi061.stdout:{"epoch":11,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","created":"2023-12-06T18:26:07.122862+0000","modified":"2023-12-06T18:26:19.860888+0000","last_up_change":"0.000000","last_in_change":"2023-12-06T18:26:12.654320+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5e6e17b9-e72e-4e34-a531-0830d659bfe0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b78c49b0-360d-4351-b44e-65ccaaeb768f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a2af3151-a4da-4c34-8fb6-78abf035785b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1c754ed6-c535-454f-b35b-00f62cd6ce5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"766572af-b9ca-4f38-927d-36f26101ba51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7a566b60-1543-41d7-894c-e2fbb63e48dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"28cc2358-4ac4-40b2-91a4-20edfe625a73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8f79279b-d2d0-438e-b3c2-88df1241ff30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-06T18:26:37.997 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-06T18:26:38.397 INFO:tasks.ceph.osd.3.smithi061.stderr:==00:00:00:25.578 87261== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:38.403 INFO:tasks.ceph.osd.3.smithi061.stderr:==00:00:00:25.585 87261== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:39.977 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:27.156 87115== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:39.977 INFO:tasks.ceph.osd.6.smithi152.stderr:==00:00:00:27.159 87114== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:40.271 INFO:tasks.ceph.osd.4.smithi152.stderr:==00:00:00:27.457 87113== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:40.538 INFO:tasks.ceph.osd.5.smithi152.stderr:==00:00:00:27.703 87116== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:41.140 INFO:tasks.ceph.osd.2.smithi061.stderr:==00:00:00:28.305 87263== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:41.461 INFO:tasks.ceph.osd.0.smithi061.stderr:==00:00:00:28.626 87262== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:41.987 INFO:tasks.ceph.osd.1.smithi061.stderr:==00:00:00:29.153 87264== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:42.841 INFO:tasks.ceph.osd.3.smithi061.stderr:==00:00:00:30.022 87261== Warning: unimplemented fcntl command: 1038 2023-12-06T18:26:44.802 DEBUG:teuthology.orchestra.run.smithi061:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-06T18:26:45.664 INFO:teuthology.misc.health.smithi061.stdout: 2023-12-06T18:26:45.664 INFO:teuthology.misc.health.smithi061.stdout:{"epoch":11,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","created":"2023-12-06T18:26:07.122862+0000","modified":"2023-12-06T18:26:19.860888+0000","last_up_change":"0.000000","last_in_change":"2023-12-06T18:26:12.654320+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5e6e17b9-e72e-4e34-a531-0830d659bfe0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b78c49b0-360d-4351-b44e-65ccaaeb768f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a2af3151-a4da-4c34-8fb6-78abf035785b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1c754ed6-c535-454f-b35b-00f62cd6ce5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"766572af-b9ca-4f38-927d-36f26101ba51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7a566b60-1543-41d7-894c-e2fbb63e48dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"28cc2358-4ac4-40b2-91a4-20edfe625a73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"8f79279b-d2d0-438e-b3c2-88df1241ff30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-06T18:26:45.680 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-06T18:26:47.700 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:26:47.694+0000 5dc2340 -1 osd.6 0 log_to_monitors true 2023-12-06T18:26:47.719 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:26:47.713+0000 5dc2340 -1 osd.7 0 log_to_monitors true 2023-12-06T18:26:48.067 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:26:48.061+0000 5dc2340 -1 osd.4 0 log_to_monitors true 2023-12-06T18:26:48.404 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:26:48.398+0000 5dc2340 -1 osd.5 0 log_to_monitors true 2023-12-06T18:26:49.558 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:26:49.545+0000 5dc2340 -1 osd.2 0 log_to_monitors true 2023-12-06T18:26:50.077 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:26:50.070+0000 5dc2340 -1 osd.0 0 log_to_monitors true 2023-12-06T18:26:50.336 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:26:50.329+0000 5dc2340 -1 osd.1 0 log_to_monitors true 2023-12-06T18:26:51.154 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:26:51.148+0000 ac7d640 -1 osd.7 0 waiting for initial osdmap 2023-12-06T18:26:51.259 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:26:51.251+0000 ac7d640 -1 osd.6 0 waiting for initial osdmap 2023-12-06T18:26:51.267 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:26:51.260+0000 c493640 -1 osd.4 0 waiting for initial osdmap 2023-12-06T18:26:51.268 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:26:51.263+0000 c493640 -1 osd.5 0 waiting for initial osdmap 2023-12-06T18:26:51.566 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:26:51.548+0000 12257640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-06T18:26:51.609 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:26:51.602+0000 5dc2340 -1 osd.3 0 log_to_monitors true 2023-12-06T18:26:51.678 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:26:51.660+0000 12a58640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-06T18:26:51.729 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:26:51.713+0000 12257640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-06T18:26:51.754 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:26:51.737+0000 12257640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-06T18:26:52.487 DEBUG:teuthology.orchestra.run.smithi061:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-06T18:26:53.433 INFO:teuthology.misc.health.smithi061.stdout: 2023-12-06T18:26:53.433 INFO:teuthology.misc.health.smithi061.stdout:{"epoch":15,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","created":"2023-12-06T18:26:07.122862+0000","modified":"2023-12-06T18:26:52.919636+0000","last_up_change":"2023-12-06T18:26:52.919636+0000","last_in_change":"2023-12-06T18:26:12.654320+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-06T18:26:52.648377+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_acting":7.1399998664855957,"score_stable":7.1399998664855957,"optimal_score":0.14000000059604645,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5e6e17b9-e72e-4e34-a531-0830d659bfe0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b78c49b0-360d-4351-b44e-65ccaaeb768f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a2af3151-a4da-4c34-8fb6-78abf035785b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1c754ed6-c535-454f-b35b-00f62cd6ce5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"766572af-b9ca-4f38-927d-36f26101ba51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2778770069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6801","nonce":2778770069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6803","nonce":2778770069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2778770069}]},"public_addr":"172.21.15.152:6800/2778770069","cluster_addr":"172.21.15.152:6801/2778770069","heartbeat_back_addr":"172.21.15.152:6803/2778770069","heartbeat_front_addr":"172.21.15.152:6802/2778770069","state":["exists","up"]},{"osd":5,"uuid":"7a566b60-1543-41d7-894c-e2fbb63e48dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6809","nonce":263671335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":263671335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":263671335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":263671335}]},"public_addr":"172.21.15.152:6809/263671335","cluster_addr":"172.21.15.152:6810/263671335","heartbeat_back_addr":"172.21.15.152:6814/263671335","heartbeat_front_addr":"172.21.15.152:6812/263671335","state":["exists","up"]},{"osd":6,"uuid":"28cc2358-4ac4-40b2-91a4-20edfe625a73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":4245639079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":4245639079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":4245639079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6813","nonce":4245639079}]},"public_addr":"172.21.15.152:6808/4245639079","cluster_addr":"172.21.15.152:6811/4245639079","heartbeat_back_addr":"172.21.15.152:6815/4245639079","heartbeat_front_addr":"172.21.15.152:6813/4245639079","state":["exists","up"]},{"osd":7,"uuid":"8f79279b-d2d0-438e-b3c2-88df1241ff30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":3940055127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6805","nonce":3940055127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6807","nonce":3940055127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":3940055127}]},"public_addr":"172.21.15.152:6804/3940055127","cluster_addr":"172.21.15.152:6805/3940055127","heartbeat_back_addr":"172.21.15.152:6807/3940055127","heartbeat_front_addr":"172.21.15.152:6806/3940055127","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:49.750276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:49.770987+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-06T18:26:53.447 DEBUG:teuthology.misc:4 of 8 OSDs are up 2023-12-06T18:26:53.642 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:26:53.634+0000 c493640 -1 osd.0 0 waiting for initial osdmap 2023-12-06T18:26:53.687 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:26:53.674+0000 c493640 -1 osd.1 0 waiting for initial osdmap 2023-12-06T18:26:53.815 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:26:53.807+0000 c493640 -1 osd.2 0 waiting for initial osdmap 2023-12-06T18:26:54.212 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:26:54.194+0000 13a5a640 -1 osd.1 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-06T18:26:54.231 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:26:54.213+0000 13259640 -1 osd.0 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-06T18:26:54.304 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:26:54.287+0000 13259640 -1 osd.2 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-06T18:26:54.892 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:26:54.884+0000 c493640 -1 osd.3 0 waiting for initial osdmap 2023-12-06T18:26:55.424 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:26:55.406+0000 12a58640 -1 osd.3 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-06T18:27:00.253 DEBUG:teuthology.orchestra.run.smithi061:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-06T18:27:01.070 INFO:teuthology.misc.health.smithi061.stdout: 2023-12-06T18:27:01.070 INFO:teuthology.misc.health.smithi061.stdout:{"epoch":20,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","created":"2023-12-06T18:26:07.122862+0000","modified":"2023-12-06T18:26:58.566099+0000","last_up_change":"2023-12-06T18:26:56.529403+0000","last_in_change":"2023-12-06T18:26:12.654320+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-06T18:26:52.648377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5e6e17b9-e72e-4e34-a531-0830d659bfe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6805","nonce":281956997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":281956997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":281956997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6807","nonce":281956997}]},"public_addr":"172.21.15.61:6805/281956997","cluster_addr":"172.21.15.61:6806/281956997","heartbeat_back_addr":"172.21.15.61:6808/281956997","heartbeat_front_addr":"172.21.15.61:6807/281956997","state":["exists","up"]},{"osd":1,"uuid":"b78c49b0-360d-4351-b44e-65ccaaeb768f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6809","nonce":3306625622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":3306625622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":3306625622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6811","nonce":3306625622}]},"public_addr":"172.21.15.61:6809/3306625622","cluster_addr":"172.21.15.61:6810/3306625622","heartbeat_back_addr":"172.21.15.61:6812/3306625622","heartbeat_front_addr":"172.21.15.61:6811/3306625622","state":["exists","up"]},{"osd":2,"uuid":"a2af3151-a4da-4c34-8fb6-78abf035785b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6801","nonce":3589270120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3589270120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3589270120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6803","nonce":3589270120}]},"public_addr":"172.21.15.61:6801/3589270120","cluster_addr":"172.21.15.61:6802/3589270120","heartbeat_back_addr":"172.21.15.61:6804/3589270120","heartbeat_front_addr":"172.21.15.61:6803/3589270120","state":["exists","up"]},{"osd":3,"uuid":"1c754ed6-c535-454f-b35b-00f62cd6ce5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":2849920842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":2849920842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":2849920842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6815","nonce":2849920842}]},"public_addr":"172.21.15.61:6813/2849920842","cluster_addr":"172.21.15.61:6814/2849920842","heartbeat_back_addr":"172.21.15.61:6816/2849920842","heartbeat_front_addr":"172.21.15.61:6815/2849920842","state":["exists","up"]},{"osd":4,"uuid":"766572af-b9ca-4f38-927d-36f26101ba51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2778770069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6801","nonce":2778770069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6803","nonce":2778770069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2778770069}]},"public_addr":"172.21.15.152:6800/2778770069","cluster_addr":"172.21.15.152:6801/2778770069","heartbeat_back_addr":"172.21.15.152:6803/2778770069","heartbeat_front_addr":"172.21.15.152:6802/2778770069","state":["exists","up"]},{"osd":5,"uuid":"7a566b60-1543-41d7-894c-e2fbb63e48dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6809","nonce":263671335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":263671335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":263671335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":263671335}]},"public_addr":"172.21.15.152:6809/263671335","cluster_addr":"172.21.15.152:6810/263671335","heartbeat_back_addr":"172.21.15.152:6814/263671335","heartbeat_front_addr":"172.21.15.152:6812/263671335","state":["exists","up"]},{"osd":6,"uuid":"28cc2358-4ac4-40b2-91a4-20edfe625a73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":4245639079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":4245639079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":4245639079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6813","nonce":4245639079}]},"public_addr":"172.21.15.152:6808/4245639079","cluster_addr":"172.21.15.152:6811/4245639079","heartbeat_back_addr":"172.21.15.152:6815/4245639079","heartbeat_front_addr":"172.21.15.152:6813/4245639079","state":["exists","up"]},{"osd":7,"uuid":"8f79279b-d2d0-438e-b3c2-88df1241ff30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":3940055127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6805","nonce":3940055127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6807","nonce":3940055127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":3940055127}]},"public_addr":"172.21.15.152:6804/3940055127","cluster_addr":"172.21.15.152:6805/3940055127","heartbeat_back_addr":"172.21.15.152:6807/3940055127","heartbeat_front_addr":"172.21.15.152:6806/3940055127","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:52.171904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:52.428840+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:51.681954+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:53.687417+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:50.158682+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:50.443807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:49.750276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:49.770987+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-06T18:27:01.082 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-12-06T18:27:01.082 INFO:tasks.ceph:Creating RBD pool 2023-12-06T18:27:01.082 DEBUG:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd pool create rbd 8 2023-12-06T18:27:02.850 INFO:teuthology.orchestra.run.smithi061.stderr:pool 'rbd' created 2023-12-06T18:27:02.864 DEBUG:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-12-06T18:27:03.910 INFO:teuthology.orchestra.run.smithi061.stderr:enabled application 'rbd' on pool 'rbd' 2023-12-06T18:27:03.923 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-12-06T18:27:03.924 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-06T18:27:03.924 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-12-06T18:27:03.924 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-12-06T18:27:03.925 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-12-06T18:27:04.854 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:27:04.869 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":7,"active_gid":4168,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6800","nonce":55528665}]},"active_addr":"172.21.15.61:6800/55528665","active_change":"2023-12-06T18:26:19.865869+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4113,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":11,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.61:0","nonce":6621916}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.61:0","nonce":2601653201}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.61:0","nonce":4232608308}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.61:0","nonce":3096550705}]}]} 2023-12-06T18:27:04.873 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-12-06T18:27:04.873 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-12-06T18:27:04.873 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-06T18:27:05.520 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:27:05.520 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":23,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","created":"2023-12-06T18:26:07.122862+0000","modified":"2023-12-06T18:27:04.895843+0000","last_up_change":"2023-12-06T18:26:56.529403+0000","last_in_change":"2023-12-06T18:26:12.654320+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-06T18:26:52.648377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-12-06T18:27:01.712581+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5e6e17b9-e72e-4e34-a531-0830d659bfe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6805","nonce":281956997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":281956997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":281956997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6807","nonce":281956997}]},"public_addr":"172.21.15.61:6805/281956997","cluster_addr":"172.21.15.61:6806/281956997","heartbeat_back_addr":"172.21.15.61:6808/281956997","heartbeat_front_addr":"172.21.15.61:6807/281956997","state":["exists","up"]},{"osd":1,"uuid":"b78c49b0-360d-4351-b44e-65ccaaeb768f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6809","nonce":3306625622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":3306625622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":3306625622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6811","nonce":3306625622}]},"public_addr":"172.21.15.61:6809/3306625622","cluster_addr":"172.21.15.61:6810/3306625622","heartbeat_back_addr":"172.21.15.61:6812/3306625622","heartbeat_front_addr":"172.21.15.61:6811/3306625622","state":["exists","up"]},{"osd":2,"uuid":"a2af3151-a4da-4c34-8fb6-78abf035785b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6801","nonce":3589270120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3589270120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3589270120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6803","nonce":3589270120}]},"public_addr":"172.21.15.61:6801/3589270120","cluster_addr":"172.21.15.61:6802/3589270120","heartbeat_back_addr":"172.21.15.61:6804/3589270120","heartbeat_front_addr":"172.21.15.61:6803/3589270120","state":["exists","up"]},{"osd":3,"uuid":"1c754ed6-c535-454f-b35b-00f62cd6ce5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":2849920842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":2849920842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":2849920842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6815","nonce":2849920842}]},"public_addr":"172.21.15.61:6813/2849920842","cluster_addr":"172.21.15.61:6814/2849920842","heartbeat_back_addr":"172.21.15.61:6816/2849920842","heartbeat_front_addr":"172.21.15.61:6815/2849920842","state":["exists","up"]},{"osd":4,"uuid":"766572af-b9ca-4f38-927d-36f26101ba51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2778770069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6801","nonce":2778770069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6803","nonce":2778770069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2778770069}]},"public_addr":"172.21.15.152:6800/2778770069","cluster_addr":"172.21.15.152:6801/2778770069","heartbeat_back_addr":"172.21.15.152:6803/2778770069","heartbeat_front_addr":"172.21.15.152:6802/2778770069","state":["exists","up"]},{"osd":5,"uuid":"7a566b60-1543-41d7-894c-e2fbb63e48dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6809","nonce":263671335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":263671335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":263671335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":263671335}]},"public_addr":"172.21.15.152:6809/263671335","cluster_addr":"172.21.15.152:6810/263671335","heartbeat_back_addr":"172.21.15.152:6814/263671335","heartbeat_front_addr":"172.21.15.152:6812/263671335","state":["exists","up"]},{"osd":6,"uuid":"28cc2358-4ac4-40b2-91a4-20edfe625a73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":4245639079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":4245639079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":4245639079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6813","nonce":4245639079}]},"public_addr":"172.21.15.152:6808/4245639079","cluster_addr":"172.21.15.152:6811/4245639079","heartbeat_back_addr":"172.21.15.152:6815/4245639079","heartbeat_front_addr":"172.21.15.152:6813/4245639079","state":["exists","up"]},{"osd":7,"uuid":"8f79279b-d2d0-438e-b3c2-88df1241ff30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":3940055127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6805","nonce":3940055127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6807","nonce":3940055127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":3940055127}]},"public_addr":"172.21.15.152:6804/3940055127","cluster_addr":"172.21.15.152:6805/3940055127","heartbeat_back_addr":"172.21.15.152:6807/3940055127","heartbeat_front_addr":"172.21.15.152:6806/3940055127","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:52.171904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:52.428840+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:51.681954+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:53.687417+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:50.158682+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:50.443807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:49.750276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:49.770987+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-06T18:27:05.537 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-12-06T18:27:05.537 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-06T18:27:06.182 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:27:06.182 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":24,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","created":"2023-12-06T18:26:07.122862+0000","modified":"2023-12-06T18:27:06.066062+0000","last_up_change":"2023-12-06T18:26:56.529403+0000","last_in_change":"2023-12-06T18:26:12.654320+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-06T18:26:52.648377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-12-06T18:27:01.712581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5e6e17b9-e72e-4e34-a531-0830d659bfe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6805","nonce":281956997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":281956997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":281956997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6807","nonce":281956997}]},"public_addr":"172.21.15.61:6805/281956997","cluster_addr":"172.21.15.61:6806/281956997","heartbeat_back_addr":"172.21.15.61:6808/281956997","heartbeat_front_addr":"172.21.15.61:6807/281956997","state":["exists","up"]},{"osd":1,"uuid":"b78c49b0-360d-4351-b44e-65ccaaeb768f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6809","nonce":3306625622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":3306625622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":3306625622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6811","nonce":3306625622}]},"public_addr":"172.21.15.61:6809/3306625622","cluster_addr":"172.21.15.61:6810/3306625622","heartbeat_back_addr":"172.21.15.61:6812/3306625622","heartbeat_front_addr":"172.21.15.61:6811/3306625622","state":["exists","up"]},{"osd":2,"uuid":"a2af3151-a4da-4c34-8fb6-78abf035785b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6801","nonce":3589270120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3589270120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3589270120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6803","nonce":3589270120}]},"public_addr":"172.21.15.61:6801/3589270120","cluster_addr":"172.21.15.61:6802/3589270120","heartbeat_back_addr":"172.21.15.61:6804/3589270120","heartbeat_front_addr":"172.21.15.61:6803/3589270120","state":["exists","up"]},{"osd":3,"uuid":"1c754ed6-c535-454f-b35b-00f62cd6ce5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":2849920842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":2849920842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":2849920842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6815","nonce":2849920842}]},"public_addr":"172.21.15.61:6813/2849920842","cluster_addr":"172.21.15.61:6814/2849920842","heartbeat_back_addr":"172.21.15.61:6816/2849920842","heartbeat_front_addr":"172.21.15.61:6815/2849920842","state":["exists","up"]},{"osd":4,"uuid":"766572af-b9ca-4f38-927d-36f26101ba51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2778770069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6801","nonce":2778770069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6803","nonce":2778770069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2778770069}]},"public_addr":"172.21.15.152:6800/2778770069","cluster_addr":"172.21.15.152:6801/2778770069","heartbeat_back_addr":"172.21.15.152:6803/2778770069","heartbeat_front_addr":"172.21.15.152:6802/2778770069","state":["exists","up"]},{"osd":5,"uuid":"7a566b60-1543-41d7-894c-e2fbb63e48dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6809","nonce":263671335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":263671335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":263671335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":263671335}]},"public_addr":"172.21.15.152:6809/263671335","cluster_addr":"172.21.15.152:6810/263671335","heartbeat_back_addr":"172.21.15.152:6814/263671335","heartbeat_front_addr":"172.21.15.152:6812/263671335","state":["exists","up"]},{"osd":6,"uuid":"28cc2358-4ac4-40b2-91a4-20edfe625a73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":4245639079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":4245639079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":4245639079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6813","nonce":4245639079}]},"public_addr":"172.21.15.152:6808/4245639079","cluster_addr":"172.21.15.152:6811/4245639079","heartbeat_back_addr":"172.21.15.152:6815/4245639079","heartbeat_front_addr":"172.21.15.152:6813/4245639079","state":["exists","up"]},{"osd":7,"uuid":"8f79279b-d2d0-438e-b3c2-88df1241ff30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":3940055127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6805","nonce":3940055127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6807","nonce":3940055127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":3940055127}]},"public_addr":"172.21.15.152:6804/3940055127","cluster_addr":"172.21.15.152:6805/3940055127","heartbeat_back_addr":"172.21.15.152:6807/3940055127","heartbeat_front_addr":"172.21.15.152:6806/3940055127","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:52.171904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:52.428840+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:51.681954+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:53.687417+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:50.158682+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:50.443807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:49.750276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:49.770987+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-06T18:27:06.195 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-12-06T18:27:06.196 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-12-06T18:27:06.196 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-12-06T18:27:06.196 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-12-06T18:27:06.197 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-12-06T18:27:06.197 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-12-06T18:27:06.198 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-12-06T18:27:06.198 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-12-06T18:27:06.624 INFO:teuthology.orchestra.run.smithi061.stdout:77309411332 2023-12-06T18:27:06.624 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-06T18:27:06.635 INFO:teuthology.orchestra.run.smithi061.stdout:60129542148 2023-12-06T18:27:06.635 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-06T18:27:06.661 INFO:teuthology.orchestra.run.smithi061.stdout:68719476742 2023-12-06T18:27:06.661 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-06T18:27:06.695 INFO:teuthology.orchestra.run.smithi061.stdout:64424509447 2023-12-06T18:27:06.695 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-06T18:27:06.714 INFO:teuthology.orchestra.run.smithi061.stdout:64424509446 2023-12-06T18:27:06.715 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-06T18:27:06.742 INFO:teuthology.orchestra.run.smithi061.stdout:68719476740 2023-12-06T18:27:06.742 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-06T18:27:06.748 INFO:teuthology.orchestra.run.smithi061.stdout:68719476741 2023-12-06T18:27:06.748 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-06T18:27:06.826 INFO:teuthology.orchestra.run.smithi061.stdout:60129542148 2023-12-06T18:27:06.826 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-06T18:27:07.567 INFO:teuthology.orchestra.run.smithi061.stdout:77309411331 2023-12-06T18:27:07.582 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411332 got 77309411331 for osd.3 2023-12-06T18:27:07.596 INFO:teuthology.orchestra.run.smithi061.stdout:68719476740 2023-12-06T18:27:07.611 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476740 for osd.0 2023-12-06T18:27:07.725 INFO:teuthology.orchestra.run.smithi061.stdout:60129542147 2023-12-06T18:27:07.744 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.6 2023-12-06T18:27:07.793 INFO:teuthology.orchestra.run.smithi061.stdout:60129542147 2023-12-06T18:27:07.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.7 2023-12-06T18:27:07.818 INFO:teuthology.orchestra.run.smithi061.stdout:68719476739 2023-12-06T18:27:07.833 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476740 got 68719476739 for osd.2 2023-12-06T18:27:07.998 INFO:teuthology.orchestra.run.smithi061.stdout:64424509444 2023-12-06T18:27:08.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.5 2023-12-06T18:27:08.052 INFO:teuthology.orchestra.run.smithi061.stdout:64424509446 2023-12-06T18:27:08.066 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.4 2023-12-06T18:27:08.214 INFO:teuthology.orchestra.run.smithi061.stdout:68719476740 2023-12-06T18:27:08.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476741 got 68719476740 for osd.1 2023-12-06T18:27:08.583 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-06T18:27:08.612 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-06T18:27:08.744 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-06T18:27:08.809 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-06T18:27:08.833 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-06T18:27:09.012 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-06T18:27:09.066 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-06T18:27:09.228 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-06T18:27:09.541 INFO:teuthology.orchestra.run.smithi061.stdout:68719476741 2023-12-06T18:27:09.558 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476741 for osd.0 2023-12-06T18:27:09.606 INFO:teuthology.orchestra.run.smithi061.stdout:77309411331 2023-12-06T18:27:09.607 INFO:teuthology.orchestra.run.smithi061.stdout:60129542147 2023-12-06T18:27:09.621 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411332 got 77309411331 for osd.3 2023-12-06T18:27:09.627 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.7 2023-12-06T18:27:09.757 INFO:teuthology.orchestra.run.smithi061.stdout:68719476739 2023-12-06T18:27:09.771 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476740 got 68719476739 for osd.2 2023-12-06T18:27:09.773 INFO:teuthology.orchestra.run.smithi061.stdout:60129542147 2023-12-06T18:27:09.788 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.6 2023-12-06T18:27:09.867 INFO:teuthology.orchestra.run.smithi061.stdout:64424509446 2023-12-06T18:27:09.882 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.4 2023-12-06T18:27:09.961 INFO:teuthology.orchestra.run.smithi061.stdout:64424509446 2023-12-06T18:27:09.974 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.5 2023-12-06T18:27:09.974 DEBUG:teuthology.parallel:result is None 2023-12-06T18:27:10.309 INFO:teuthology.orchestra.run.smithi061.stdout:68719476741 2023-12-06T18:27:10.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476741 got 68719476741 for osd.1 2023-12-06T18:27:10.322 DEBUG:teuthology.parallel:result is None 2023-12-06T18:27:10.559 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-06T18:27:10.623 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-06T18:27:10.628 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-06T18:27:10.772 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-06T18:27:10.788 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-06T18:27:10.884 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-06T18:27:11.346 INFO:teuthology.orchestra.run.smithi061.stdout:68719476742 2023-12-06T18:27:11.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476742 for osd.0 2023-12-06T18:27:11.366 DEBUG:teuthology.parallel:result is None 2023-12-06T18:27:11.539 INFO:teuthology.orchestra.run.smithi061.stdout:64424509447 2023-12-06T18:27:11.540 INFO:teuthology.orchestra.run.smithi061.stdout:60129542149 2023-12-06T18:27:11.556 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.4 2023-12-06T18:27:11.556 DEBUG:teuthology.parallel:result is None 2023-12-06T18:27:11.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.7 2023-12-06T18:27:11.561 DEBUG:teuthology.parallel:result is None 2023-12-06T18:27:11.566 INFO:teuthology.orchestra.run.smithi061.stdout:77309411333 2023-12-06T18:27:11.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411332 got 77309411333 for osd.3 2023-12-06T18:27:11.583 DEBUG:teuthology.parallel:result is None 2023-12-06T18:27:11.693 INFO:teuthology.orchestra.run.smithi061.stdout:60129542149 2023-12-06T18:27:11.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.6 2023-12-06T18:27:11.707 DEBUG:teuthology.parallel:result is None 2023-12-06T18:27:11.748 INFO:teuthology.orchestra.run.smithi061.stdout:68719476741 2023-12-06T18:27:11.761 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476740 got 68719476741 for osd.2 2023-12-06T18:27:11.762 DEBUG:teuthology.parallel:result is None 2023-12-06T18:27:11.762 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-06T18:27:11.762 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:27:12.374 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:27:12.375 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:27:12.387 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-12-06T18:27:12.216387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218948,"kb_used_data":2384,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755772,"statfs":{"total":773094113280,"available":772869910528,"internally_reserved":0,"allocated":2441216,"data_stored":1372112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001634"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.712401+0000","last_change":"2023-12-06T18:27:04.849231+0000","last_active":"2023-12-06T18:27:07.712401+0000","last_peered":"2023-12-06T18:27:07.712401+0000","last_clean":"2023-12-06T18:27:07.712401+0000","last_became_active":"2023-12-06T18:27:04.837862+0000","last_became_peered":"2023-12-06T18:27:04.837862+0000","last_unstale":"2023-12-06T18:27:07.712401+0000","last_undegraded":"2023-12-06T18:27:07.712401+0000","last_fullsized":"2023-12-06T18:27:07.712401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:03:55.366586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:06.147130+0000","last_change":"2023-12-06T18:27:04.883840+0000","last_active":"2023-12-06T18:27:06.147130+0000","last_peered":"2023-12-06T18:27:06.147130+0000","last_clean":"2023-12-06T18:27:06.147130+0000","last_became_active":"2023-12-06T18:27:04.858001+0000","last_became_peered":"2023-12-06T18:27:04.858001+0000","last_unstale":"2023-12-06T18:27:06.147130+0000","last_undegraded":"2023-12-06T18:27:06.147130+0000","last_fullsized":"2023-12-06T18:27:06.147130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:44:02.916244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.704793+0000","last_change":"2023-12-06T18:27:04.336778+0000","last_active":"2023-12-06T18:27:07.704793+0000","last_peered":"2023-12-06T18:27:07.704793+0000","last_clean":"2023-12-06T18:27:07.704793+0000","last_became_active":"2023-12-06T18:27:04.321269+0000","last_became_peered":"2023-12-06T18:27:04.321269+0000","last_unstale":"2023-12-06T18:27:07.704793+0000","last_undegraded":"2023-12-06T18:27:07.704793+0000","last_fullsized":"2023-12-06T18:27:07.704793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T22:25:15.703579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.175299+0000","last_change":"2023-12-06T18:27:04.453102+0000","last_active":"2023-12-06T18:27:07.175299+0000","last_peered":"2023-12-06T18:27:07.175299+0000","last_clean":"2023-12-06T18:27:07.175299+0000","last_became_active":"2023-12-06T18:27:04.420809+0000","last_became_peered":"2023-12-06T18:27:04.420809+0000","last_unstale":"2023-12-06T18:27:07.175299+0000","last_undegraded":"2023-12-06T18:27:07.175299+0000","last_fullsized":"2023-12-06T18:27:07.175299+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T19:05:57.586399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.688352+0000","last_change":"2023-12-06T18:27:04.408081+0000","last_active":"2023-12-06T18:27:07.688352+0000","last_peered":"2023-12-06T18:27:07.688352+0000","last_clean":"2023-12-06T18:27:07.688352+0000","last_became_active":"2023-12-06T18:27:04.399765+0000","last_became_peered":"2023-12-06T18:27:04.399765+0000","last_unstale":"2023-12-06T18:27:07.688352+0000","last_undegraded":"2023-12-06T18:27:07.688352+0000","last_fullsized":"2023-12-06T18:27:07.688352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:40:33.570114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:06.178486+0000","last_change":"2023-12-06T18:27:04.308250+0000","last_active":"2023-12-06T18:27:06.178486+0000","last_peered":"2023-12-06T18:27:06.178486+0000","last_clean":"2023-12-06T18:27:06.178486+0000","last_became_active":"2023-12-06T18:27:04.273881+0000","last_became_peered":"2023-12-06T18:27:04.273881+0000","last_unstale":"2023-12-06T18:27:06.178486+0000","last_undegraded":"2023-12-06T18:27:06.178486+0000","last_fullsized":"2023-12-06T18:27:06.178486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T00:55:59.429987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.679250+0000","last_change":"2023-12-06T18:27:04.274499+0000","last_active":"2023-12-06T18:27:07.679250+0000","last_peered":"2023-12-06T18:27:07.679250+0000","last_clean":"2023-12-06T18:27:07.679250+0000","last_became_active":"2023-12-06T18:27:04.243006+0000","last_became_peered":"2023-12-06T18:27:04.243006+0000","last_unstale":"2023-12-06T18:27:07.679250+0000","last_undegraded":"2023-12-06T18:27:07.679250+0000","last_fullsized":"2023-12-06T18:27:07.679250+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T06:17:30.361169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.699366+0000","last_change":"2023-12-06T18:27:04.609381+0000","last_active":"2023-12-06T18:27:07.699366+0000","last_peered":"2023-12-06T18:27:07.699366+0000","last_clean":"2023-12-06T18:27:07.699366+0000","last_became_active":"2023-12-06T18:27:04.598701+0000","last_became_peered":"2023-12-06T18:27:04.598701+0000","last_unstale":"2023-12-06T18:27:07.699366+0000","last_undegraded":"2023-12-06T18:27:07.699366+0000","last_fullsized":"2023-12-06T18:27:07.699366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:29:58.962692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":120,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-06T18:27:05.549823+0000","last_change":"2023-12-06T18:26:56.215945+0000","last_active":"2023-12-06T18:27:05.549823+0000","last_peered":"2023-12-06T18:27:05.549823+0000","last_clean":"2023-12-06T18:27:05.549823+0000","last_became_active":"2023-12-06T18:26:56.180726+0000","last_became_peered":"2023-12-06T18:26:56.180726+0000","last_unstale":"2023-12-06T18:27:05.549823+0000","last_undegraded":"2023-12-06T18:27:05.549823+0000","last_fullsized":"2023-12-06T18:27:05.549823+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T03:39:20.100471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411334,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.7289999999999992}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.074}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.620999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.311}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":24.875}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":20.170000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":33.475000000000001}]}]},{"osd":2,"up_from":16,"seq":68719476741,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.7789999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8919999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.519}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.6200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.132}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.618}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.271999999999998}]}]},{"osd":1,"up_from":16,"seq":68719476742,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":606208,"data_stored":469894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.458}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.513}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.858000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.967000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.698}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8339999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.693}]}]},{"osd":0,"up_from":16,"seq":68719476743,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.446999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.629}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.646000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":39.851999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.324}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":20.968}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":46.796999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":606208,"data_stored":469894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.731999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.27}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.356999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.945}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.318999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.741}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.762}]}]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.867000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":49.183}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.045999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.23}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.2880000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.84}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.809999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":198.46700000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.6470000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":69.350999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.891}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.516}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.401999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.484999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2850000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.170999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.5039999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.115}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.721}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.224}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.018000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:27:12.388 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:27:12.959 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:27:12.959 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:27:12.971 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-12-06T18:27:12.216387+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218948,"kb_used_data":2384,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755772,"statfs":{"total":773094113280,"available":772869910528,"internally_reserved":0,"allocated":2441216,"data_stored":1372112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001634"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.712401+0000","last_change":"2023-12-06T18:27:04.849231+0000","last_active":"2023-12-06T18:27:07.712401+0000","last_peered":"2023-12-06T18:27:07.712401+0000","last_clean":"2023-12-06T18:27:07.712401+0000","last_became_active":"2023-12-06T18:27:04.837862+0000","last_became_peered":"2023-12-06T18:27:04.837862+0000","last_unstale":"2023-12-06T18:27:07.712401+0000","last_undegraded":"2023-12-06T18:27:07.712401+0000","last_fullsized":"2023-12-06T18:27:07.712401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:03:55.366586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:06.147130+0000","last_change":"2023-12-06T18:27:04.883840+0000","last_active":"2023-12-06T18:27:06.147130+0000","last_peered":"2023-12-06T18:27:06.147130+0000","last_clean":"2023-12-06T18:27:06.147130+0000","last_became_active":"2023-12-06T18:27:04.858001+0000","last_became_peered":"2023-12-06T18:27:04.858001+0000","last_unstale":"2023-12-06T18:27:06.147130+0000","last_undegraded":"2023-12-06T18:27:06.147130+0000","last_fullsized":"2023-12-06T18:27:06.147130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:44:02.916244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.704793+0000","last_change":"2023-12-06T18:27:04.336778+0000","last_active":"2023-12-06T18:27:07.704793+0000","last_peered":"2023-12-06T18:27:07.704793+0000","last_clean":"2023-12-06T18:27:07.704793+0000","last_became_active":"2023-12-06T18:27:04.321269+0000","last_became_peered":"2023-12-06T18:27:04.321269+0000","last_unstale":"2023-12-06T18:27:07.704793+0000","last_undegraded":"2023-12-06T18:27:07.704793+0000","last_fullsized":"2023-12-06T18:27:07.704793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T22:25:15.703579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.175299+0000","last_change":"2023-12-06T18:27:04.453102+0000","last_active":"2023-12-06T18:27:07.175299+0000","last_peered":"2023-12-06T18:27:07.175299+0000","last_clean":"2023-12-06T18:27:07.175299+0000","last_became_active":"2023-12-06T18:27:04.420809+0000","last_became_peered":"2023-12-06T18:27:04.420809+0000","last_unstale":"2023-12-06T18:27:07.175299+0000","last_undegraded":"2023-12-06T18:27:07.175299+0000","last_fullsized":"2023-12-06T18:27:07.175299+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T19:05:57.586399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.688352+0000","last_change":"2023-12-06T18:27:04.408081+0000","last_active":"2023-12-06T18:27:07.688352+0000","last_peered":"2023-12-06T18:27:07.688352+0000","last_clean":"2023-12-06T18:27:07.688352+0000","last_became_active":"2023-12-06T18:27:04.399765+0000","last_became_peered":"2023-12-06T18:27:04.399765+0000","last_unstale":"2023-12-06T18:27:07.688352+0000","last_undegraded":"2023-12-06T18:27:07.688352+0000","last_fullsized":"2023-12-06T18:27:07.688352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:40:33.570114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:06.178486+0000","last_change":"2023-12-06T18:27:04.308250+0000","last_active":"2023-12-06T18:27:06.178486+0000","last_peered":"2023-12-06T18:27:06.178486+0000","last_clean":"2023-12-06T18:27:06.178486+0000","last_became_active":"2023-12-06T18:27:04.273881+0000","last_became_peered":"2023-12-06T18:27:04.273881+0000","last_unstale":"2023-12-06T18:27:06.178486+0000","last_undegraded":"2023-12-06T18:27:06.178486+0000","last_fullsized":"2023-12-06T18:27:06.178486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T00:55:59.429987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.679250+0000","last_change":"2023-12-06T18:27:04.274499+0000","last_active":"2023-12-06T18:27:07.679250+0000","last_peered":"2023-12-06T18:27:07.679250+0000","last_clean":"2023-12-06T18:27:07.679250+0000","last_became_active":"2023-12-06T18:27:04.243006+0000","last_became_peered":"2023-12-06T18:27:04.243006+0000","last_unstale":"2023-12-06T18:27:07.679250+0000","last_undegraded":"2023-12-06T18:27:07.679250+0000","last_fullsized":"2023-12-06T18:27:07.679250+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T06:17:30.361169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.699366+0000","last_change":"2023-12-06T18:27:04.609381+0000","last_active":"2023-12-06T18:27:07.699366+0000","last_peered":"2023-12-06T18:27:07.699366+0000","last_clean":"2023-12-06T18:27:07.699366+0000","last_became_active":"2023-12-06T18:27:04.598701+0000","last_became_peered":"2023-12-06T18:27:04.598701+0000","last_unstale":"2023-12-06T18:27:07.699366+0000","last_undegraded":"2023-12-06T18:27:07.699366+0000","last_fullsized":"2023-12-06T18:27:07.699366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:29:58.962692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":120,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-06T18:27:05.549823+0000","last_change":"2023-12-06T18:26:56.215945+0000","last_active":"2023-12-06T18:27:05.549823+0000","last_peered":"2023-12-06T18:27:05.549823+0000","last_clean":"2023-12-06T18:27:05.549823+0000","last_became_active":"2023-12-06T18:26:56.180726+0000","last_became_peered":"2023-12-06T18:26:56.180726+0000","last_unstale":"2023-12-06T18:27:05.549823+0000","last_undegraded":"2023-12-06T18:27:05.549823+0000","last_fullsized":"2023-12-06T18:27:05.549823+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T03:39:20.100471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411334,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.7289999999999992}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.074}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.620999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.311}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":24.875}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":20.170000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":33.475000000000001}]}]},{"osd":2,"up_from":16,"seq":68719476741,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.7789999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8919999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.519}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.6200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.132}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.618}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.271999999999998}]}]},{"osd":1,"up_from":16,"seq":68719476742,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":606208,"data_stored":469894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.458}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.513}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.858000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.967000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.698}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8339999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.693}]}]},{"osd":0,"up_from":16,"seq":68719476743,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.446999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.629}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.646000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":39.851999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.324}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":20.968}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":46.796999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":606208,"data_stored":469894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.731999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.27}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.356999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.945}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.318999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.741}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.762}]}]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.867000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":49.183}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.045999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.23}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.2880000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.84}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.809999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":198.46700000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.6470000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":69.350999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.891}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.516}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.401999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.484999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2850000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.170999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.5039999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.115}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.721}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.224}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.018000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:27:12.972 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-06T18:27:12.972 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-06T18:27:12.972 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-12-06T18:27:12.972 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-12-06T18:27:13.654 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:27:13.654 INFO:teuthology.orchestra.run.smithi061.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-06T18:27:13.668 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-12-06T18:27:13.668 INFO:teuthology.run_tasks:Running task thrashosds... 2023-12-06T18:27:13.679 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2023-12-06T18:27:13.679 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-sseshasa-testing-2023-12-04-1848'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd heartbeat grace': 80}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osd crush smoke test': False, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 1, 'debug bluestore': 1, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(OSDMAP_FLAGS\\)', '\\(SMALLER_PGP_NUM\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW OPS\\)', 'slow request', 'overall HEALTH_', '\\(PG_', '\\(MON_DOWN\\)', '\\(SLOW_OPS\\)', 'slow request'], 'sha1': '96fa2578c622ca9c169af6f426ca8b1616130e58', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': '96fa2578c622ca9c169af6f426ca8b1616130e58'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-sseshasa-testing-2023-12-04-1848', 'sha1': '96fa2578c622ca9c169af6f426ca8b1616130e58'}} 2023-12-06T18:27:13.680 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2023-12-06T18:27:13.680 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2023-12-06T18:27:14.117 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:25.456 86952== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:14.203 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:01:25.523 86580== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:14.225 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:25.564 86952== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:14.226 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:25.565 86953== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:14.253 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:01:25.573 86580== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:14.283 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:25.622 86953== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:14.502 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:01:25.823 86580== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:14.542 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:25.882 86953== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:14.543 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:25.883 86952== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:14.967 INFO:tasks.thrashosds:Beginning thrashosds... 2023-12-06T18:27:14.967 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-06T18:27:14.968 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:27:15.550 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:27:15.551 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:27:15.562 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-12-06T18:27:14.216782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218948,"kb_used_data":2384,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755772,"statfs":{"total":773094113280,"available":772869910528,"internally_reserved":0,"allocated":2441216,"data_stored":1372112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.089217"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.712401+0000","last_change":"2023-12-06T18:27:04.849231+0000","last_active":"2023-12-06T18:27:07.712401+0000","last_peered":"2023-12-06T18:27:07.712401+0000","last_clean":"2023-12-06T18:27:07.712401+0000","last_became_active":"2023-12-06T18:27:04.837862+0000","last_became_peered":"2023-12-06T18:27:04.837862+0000","last_unstale":"2023-12-06T18:27:07.712401+0000","last_undegraded":"2023-12-06T18:27:07.712401+0000","last_fullsized":"2023-12-06T18:27:07.712401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:03:55.366586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:06.147130+0000","last_change":"2023-12-06T18:27:04.883840+0000","last_active":"2023-12-06T18:27:06.147130+0000","last_peered":"2023-12-06T18:27:06.147130+0000","last_clean":"2023-12-06T18:27:06.147130+0000","last_became_active":"2023-12-06T18:27:04.858001+0000","last_became_peered":"2023-12-06T18:27:04.858001+0000","last_unstale":"2023-12-06T18:27:06.147130+0000","last_undegraded":"2023-12-06T18:27:06.147130+0000","last_fullsized":"2023-12-06T18:27:06.147130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:44:02.916244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.704793+0000","last_change":"2023-12-06T18:27:04.336778+0000","last_active":"2023-12-06T18:27:07.704793+0000","last_peered":"2023-12-06T18:27:07.704793+0000","last_clean":"2023-12-06T18:27:07.704793+0000","last_became_active":"2023-12-06T18:27:04.321269+0000","last_became_peered":"2023-12-06T18:27:04.321269+0000","last_unstale":"2023-12-06T18:27:07.704793+0000","last_undegraded":"2023-12-06T18:27:07.704793+0000","last_fullsized":"2023-12-06T18:27:07.704793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T22:25:15.703579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.175299+0000","last_change":"2023-12-06T18:27:04.453102+0000","last_active":"2023-12-06T18:27:07.175299+0000","last_peered":"2023-12-06T18:27:07.175299+0000","last_clean":"2023-12-06T18:27:07.175299+0000","last_became_active":"2023-12-06T18:27:04.420809+0000","last_became_peered":"2023-12-06T18:27:04.420809+0000","last_unstale":"2023-12-06T18:27:07.175299+0000","last_undegraded":"2023-12-06T18:27:07.175299+0000","last_fullsized":"2023-12-06T18:27:07.175299+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T19:05:57.586399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.688352+0000","last_change":"2023-12-06T18:27:04.408081+0000","last_active":"2023-12-06T18:27:07.688352+0000","last_peered":"2023-12-06T18:27:07.688352+0000","last_clean":"2023-12-06T18:27:07.688352+0000","last_became_active":"2023-12-06T18:27:04.399765+0000","last_became_peered":"2023-12-06T18:27:04.399765+0000","last_unstale":"2023-12-06T18:27:07.688352+0000","last_undegraded":"2023-12-06T18:27:07.688352+0000","last_fullsized":"2023-12-06T18:27:07.688352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:40:33.570114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:06.178486+0000","last_change":"2023-12-06T18:27:04.308250+0000","last_active":"2023-12-06T18:27:06.178486+0000","last_peered":"2023-12-06T18:27:06.178486+0000","last_clean":"2023-12-06T18:27:06.178486+0000","last_became_active":"2023-12-06T18:27:04.273881+0000","last_became_peered":"2023-12-06T18:27:04.273881+0000","last_unstale":"2023-12-06T18:27:06.178486+0000","last_undegraded":"2023-12-06T18:27:06.178486+0000","last_fullsized":"2023-12-06T18:27:06.178486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T00:55:59.429987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.679250+0000","last_change":"2023-12-06T18:27:04.274499+0000","last_active":"2023-12-06T18:27:07.679250+0000","last_peered":"2023-12-06T18:27:07.679250+0000","last_clean":"2023-12-06T18:27:07.679250+0000","last_became_active":"2023-12-06T18:27:04.243006+0000","last_became_peered":"2023-12-06T18:27:04.243006+0000","last_unstale":"2023-12-06T18:27:07.679250+0000","last_undegraded":"2023-12-06T18:27:07.679250+0000","last_fullsized":"2023-12-06T18:27:07.679250+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T06:17:30.361169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.699366+0000","last_change":"2023-12-06T18:27:04.609381+0000","last_active":"2023-12-06T18:27:07.699366+0000","last_peered":"2023-12-06T18:27:07.699366+0000","last_clean":"2023-12-06T18:27:07.699366+0000","last_became_active":"2023-12-06T18:27:04.598701+0000","last_became_peered":"2023-12-06T18:27:04.598701+0000","last_unstale":"2023-12-06T18:27:07.699366+0000","last_undegraded":"2023-12-06T18:27:07.699366+0000","last_fullsized":"2023-12-06T18:27:07.699366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:29:58.962692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":120,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-06T18:27:05.549823+0000","last_change":"2023-12-06T18:26:56.215945+0000","last_active":"2023-12-06T18:27:05.549823+0000","last_peered":"2023-12-06T18:27:05.549823+0000","last_clean":"2023-12-06T18:27:05.549823+0000","last_became_active":"2023-12-06T18:26:56.180726+0000","last_became_peered":"2023-12-06T18:26:56.180726+0000","last_unstale":"2023-12-06T18:27:05.549823+0000","last_undegraded":"2023-12-06T18:27:05.549823+0000","last_fullsized":"2023-12-06T18:27:05.549823+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T03:39:20.100471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411334,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.7289999999999992}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.074}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.620999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.311}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":24.875}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":20.170000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":33.475000000000001}]}]},{"osd":2,"up_from":16,"seq":68719476741,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.7789999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8919999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.519}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.6200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.132}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.618}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.271999999999998}]}]},{"osd":1,"up_from":16,"seq":68719476742,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":606208,"data_stored":469894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.458}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.513}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.858000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.967000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.698}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8339999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.693}]}]},{"osd":0,"up_from":16,"seq":68719476743,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.446999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.629}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.646000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":39.851999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.324}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":20.968}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":46.796999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":606208,"data_stored":469894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.731999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.27}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.356999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.945}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.318999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.741}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.762}]}]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.867000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":49.183}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.045999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.23}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.2880000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.84}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.809999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6179999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.715999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.711}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.0250000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.282}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.433}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.311}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.1269999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.244}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.475}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":25.190999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":26.405999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":24.146000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.449999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:27:15.564 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:27:16.163 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:27:16.163 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:27:16.175 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-12-06T18:27:14.216782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218948,"kb_used_data":2384,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755772,"statfs":{"total":773094113280,"available":772869910528,"internally_reserved":0,"allocated":2441216,"data_stored":1372112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.089217"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.712401+0000","last_change":"2023-12-06T18:27:04.849231+0000","last_active":"2023-12-06T18:27:07.712401+0000","last_peered":"2023-12-06T18:27:07.712401+0000","last_clean":"2023-12-06T18:27:07.712401+0000","last_became_active":"2023-12-06T18:27:04.837862+0000","last_became_peered":"2023-12-06T18:27:04.837862+0000","last_unstale":"2023-12-06T18:27:07.712401+0000","last_undegraded":"2023-12-06T18:27:07.712401+0000","last_fullsized":"2023-12-06T18:27:07.712401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:03:55.366586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:06.147130+0000","last_change":"2023-12-06T18:27:04.883840+0000","last_active":"2023-12-06T18:27:06.147130+0000","last_peered":"2023-12-06T18:27:06.147130+0000","last_clean":"2023-12-06T18:27:06.147130+0000","last_became_active":"2023-12-06T18:27:04.858001+0000","last_became_peered":"2023-12-06T18:27:04.858001+0000","last_unstale":"2023-12-06T18:27:06.147130+0000","last_undegraded":"2023-12-06T18:27:06.147130+0000","last_fullsized":"2023-12-06T18:27:06.147130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:44:02.916244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.704793+0000","last_change":"2023-12-06T18:27:04.336778+0000","last_active":"2023-12-06T18:27:07.704793+0000","last_peered":"2023-12-06T18:27:07.704793+0000","last_clean":"2023-12-06T18:27:07.704793+0000","last_became_active":"2023-12-06T18:27:04.321269+0000","last_became_peered":"2023-12-06T18:27:04.321269+0000","last_unstale":"2023-12-06T18:27:07.704793+0000","last_undegraded":"2023-12-06T18:27:07.704793+0000","last_fullsized":"2023-12-06T18:27:07.704793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T22:25:15.703579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.175299+0000","last_change":"2023-12-06T18:27:04.453102+0000","last_active":"2023-12-06T18:27:07.175299+0000","last_peered":"2023-12-06T18:27:07.175299+0000","last_clean":"2023-12-06T18:27:07.175299+0000","last_became_active":"2023-12-06T18:27:04.420809+0000","last_became_peered":"2023-12-06T18:27:04.420809+0000","last_unstale":"2023-12-06T18:27:07.175299+0000","last_undegraded":"2023-12-06T18:27:07.175299+0000","last_fullsized":"2023-12-06T18:27:07.175299+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T19:05:57.586399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.688352+0000","last_change":"2023-12-06T18:27:04.408081+0000","last_active":"2023-12-06T18:27:07.688352+0000","last_peered":"2023-12-06T18:27:07.688352+0000","last_clean":"2023-12-06T18:27:07.688352+0000","last_became_active":"2023-12-06T18:27:04.399765+0000","last_became_peered":"2023-12-06T18:27:04.399765+0000","last_unstale":"2023-12-06T18:27:07.688352+0000","last_undegraded":"2023-12-06T18:27:07.688352+0000","last_fullsized":"2023-12-06T18:27:07.688352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:40:33.570114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:06.178486+0000","last_change":"2023-12-06T18:27:04.308250+0000","last_active":"2023-12-06T18:27:06.178486+0000","last_peered":"2023-12-06T18:27:06.178486+0000","last_clean":"2023-12-06T18:27:06.178486+0000","last_became_active":"2023-12-06T18:27:04.273881+0000","last_became_peered":"2023-12-06T18:27:04.273881+0000","last_unstale":"2023-12-06T18:27:06.178486+0000","last_undegraded":"2023-12-06T18:27:06.178486+0000","last_fullsized":"2023-12-06T18:27:06.178486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T00:55:59.429987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.679250+0000","last_change":"2023-12-06T18:27:04.274499+0000","last_active":"2023-12-06T18:27:07.679250+0000","last_peered":"2023-12-06T18:27:07.679250+0000","last_clean":"2023-12-06T18:27:07.679250+0000","last_became_active":"2023-12-06T18:27:04.243006+0000","last_became_peered":"2023-12-06T18:27:04.243006+0000","last_unstale":"2023-12-06T18:27:07.679250+0000","last_undegraded":"2023-12-06T18:27:07.679250+0000","last_fullsized":"2023-12-06T18:27:07.679250+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T06:17:30.361169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-06T18:27:07.699366+0000","last_change":"2023-12-06T18:27:04.609381+0000","last_active":"2023-12-06T18:27:07.699366+0000","last_peered":"2023-12-06T18:27:07.699366+0000","last_clean":"2023-12-06T18:27:07.699366+0000","last_became_active":"2023-12-06T18:27:04.598701+0000","last_became_peered":"2023-12-06T18:27:04.598701+0000","last_unstale":"2023-12-06T18:27:07.699366+0000","last_undegraded":"2023-12-06T18:27:07.699366+0000","last_fullsized":"2023-12-06T18:27:07.699366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:29:58.962692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":120,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-06T18:27:05.549823+0000","last_change":"2023-12-06T18:26:56.215945+0000","last_active":"2023-12-06T18:27:05.549823+0000","last_peered":"2023-12-06T18:27:05.549823+0000","last_clean":"2023-12-06T18:27:05.549823+0000","last_became_active":"2023-12-06T18:26:56.180726+0000","last_became_peered":"2023-12-06T18:26:56.180726+0000","last_unstale":"2023-12-06T18:27:05.549823+0000","last_undegraded":"2023-12-06T18:27:05.549823+0000","last_fullsized":"2023-12-06T18:27:05.549823+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T03:39:20.100471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411334,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.7289999999999992}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.074}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.620999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.311}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":24.875}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":20.170000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":33.475000000000001}]}]},{"osd":2,"up_from":16,"seq":68719476741,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.7789999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8919999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.519}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.6200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.132}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.618}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.271999999999998}]}]},{"osd":1,"up_from":16,"seq":68719476742,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":606208,"data_stored":469894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.458}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.513}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.858000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.967000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.698}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8339999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.693}]}]},{"osd":0,"up_from":16,"seq":68719476743,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.446999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.629}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.646000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":39.851999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.324}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":20.968}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":46.796999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":606208,"data_stored":469894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.731999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.27}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.356999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.945}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.318999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.741}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.762}]}]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.867000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":49.183}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.045999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.23}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.2880000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.84}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.809999999999999}]}]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6179999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.715999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.711}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.0250000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.282}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.433}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.311}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":204800,"data_stored":72054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.1269999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.244}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.475}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":25.190999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":26.405999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":24.146000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.449999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:27:16.176 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-06T18:27:16.176 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2023-12-06T18:27:16.773 INFO:teuthology.orchestra.run.smithi061.stdout:epoch 24 2023-12-06T18:27:16.773 INFO:teuthology.orchestra.run.smithi061.stdout:fsid 497e3283-cc16-40ff-a72c-a7487474fb4b 2023-12-06T18:27:16.773 INFO:teuthology.orchestra.run.smithi061.stdout:created 2023-12-06T18:26:07.122862+0000 2023-12-06T18:27:16.773 INFO:teuthology.orchestra.run.smithi061.stdout:modified 2023-12-06T18:27:06.066062+0000 2023-12-06T18:27:16.773 INFO:teuthology.orchestra.run.smithi061.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-12-06T18:27:16.774 INFO:teuthology.orchestra.run.smithi061.stdout:crush_version 7 2023-12-06T18:27:16.774 INFO:teuthology.orchestra.run.smithi061.stdout:full_ratio 0.95 2023-12-06T18:27:16.774 INFO:teuthology.orchestra.run.smithi061.stdout:backfillfull_ratio 0.9 2023-12-06T18:27:16.774 INFO:teuthology.orchestra.run.smithi061.stdout:nearfull_ratio 0.85 2023-12-06T18:27:16.774 INFO:teuthology.orchestra.run.smithi061.stdout:require_min_compat_client luminous 2023-12-06T18:27:16.774 INFO:teuthology.orchestra.run.smithi061.stdout:min_compat_client jewel 2023-12-06T18:27:16.774 INFO:teuthology.orchestra.run.smithi061.stdout:require_osd_release reef 2023-12-06T18:27:16.774 INFO:teuthology.orchestra.run.smithi061.stdout:stretch_mode_enabled false 2023-12-06T18:27:16.774 INFO:teuthology.orchestra.run.smithi061.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 19 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2023-12-06T18:27:16.774 INFO:teuthology.orchestra.run.smithi061.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 24 flags hashpspool stripe_width 0 application rbd read_balance_score 4.98 2023-12-06T18:27:16.774 INFO:teuthology.orchestra.run.smithi061.stdout:max_osd 8 2023-12-06T18:27:16.775 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0 up in weight 1 up_from 16 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.61:6805/281956997 v2:172.21.15.61:6806/281956997 exists,up 5e6e17b9-e72e-4e34-a531-0830d659bfe0 2023-12-06T18:27:16.775 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1 up in weight 1 up_from 16 up_thru 16 down_at 0 last_clean_interval [0,0) v2:172.21.15.61:6809/3306625622 v2:172.21.15.61:6810/3306625622 exists,up b78c49b0-360d-4351-b44e-65ccaaeb768f 2023-12-06T18:27:16.775 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2 up in weight 1 up_from 16 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.61:6801/3589270120 v2:172.21.15.61:6802/3589270120 exists,up a2af3151-a4da-4c34-8fb6-78abf035785b 2023-12-06T18:27:16.775 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3 up in weight 1 up_from 18 up_thru 21 down_at 0 last_clean_interval [0,0) v2:172.21.15.61:6813/2849920842 v2:172.21.15.61:6814/2849920842 exists,up 1c754ed6-c535-454f-b35b-00f62cd6ce5a 2023-12-06T18:27:16.775 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4 up in weight 1 up_from 15 up_thru 21 down_at 0 last_clean_interval [0,0) v2:172.21.15.152:6800/2778770069 v2:172.21.15.152:6801/2778770069 exists,up 766572af-b9ca-4f38-927d-36f26101ba51 2023-12-06T18:27:16.775 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5 up in weight 1 up_from 15 up_thru 21 down_at 0 last_clean_interval [0,0) v2:172.21.15.152:6809/263671335 v2:172.21.15.152:6810/263671335 exists,up 7a566b60-1543-41d7-894c-e2fbb63e48dc 2023-12-06T18:27:16.775 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.152:6808/4245639079 v2:172.21.15.152:6811/4245639079 exists,up 28cc2358-4ac4-40b2-91a4-20edfe625a73 2023-12-06T18:27:16.775 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7 up in weight 1 up_from 14 up_thru 21 down_at 0 last_clean_interval [0,0) v2:172.21.15.152:6804/3940055127 v2:172.21.15.152:6805/3940055127 exists,up 8f79279b-d2d0-438e-b3c2-88df1241ff30 2023-12-06T18:27:16.787 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 16 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.61:6805/281956997 v2:172.21.15.61:6806/281956997 exists,up 5e6e17b9-e72e-4e34-a531-0830d659bfe0', 'osd.1 up in weight 1 up_from 16 up_thru 16 down_at 0 last_clean_interval [0,0) v2:172.21.15.61:6809/3306625622 v2:172.21.15.61:6810/3306625622 exists,up b78c49b0-360d-4351-b44e-65ccaaeb768f', 'osd.2 up in weight 1 up_from 16 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.61:6801/3589270120 v2:172.21.15.61:6802/3589270120 exists,up a2af3151-a4da-4c34-8fb6-78abf035785b', 'osd.3 up in weight 1 up_from 18 up_thru 21 down_at 0 last_clean_interval [0,0) v2:172.21.15.61:6813/2849920842 v2:172.21.15.61:6814/2849920842 exists,up 1c754ed6-c535-454f-b35b-00f62cd6ce5a', 'osd.4 up in weight 1 up_from 15 up_thru 21 down_at 0 last_clean_interval [0,0) v2:172.21.15.152:6800/2778770069 v2:172.21.15.152:6801/2778770069 exists,up 766572af-b9ca-4f38-927d-36f26101ba51', 'osd.5 up in weight 1 up_from 15 up_thru 21 down_at 0 last_clean_interval [0,0) v2:172.21.15.152:6809/263671335 v2:172.21.15.152:6810/263671335 exists,up 7a566b60-1543-41d7-894c-e2fbb63e48dc', 'osd.6 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.152:6808/4245639079 v2:172.21.15.152:6811/4245639079 exists,up 28cc2358-4ac4-40b2-91a4-20edfe625a73', 'osd.7 up in weight 1 up_from 14 up_thru 21 down_at 0 last_clean_interval [0,0) v2:172.21.15.152:6804/3940055127 v2:172.21.15.152:6805/3940055127 exists,up 8f79279b-d2d0-438e-b3c2-88df1241ff30'] 2023-12-06T18:27:16.787 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2023-12-06T18:27:16.981 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:16.981 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "mon.a", 2023-12-06T18:27:16.981 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster": "ceph", 2023-12-06T18:27:16.981 INFO:teuthology.orchestra.run.smithi061.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2023-12-06T18:27:16.981 INFO:teuthology.orchestra.run.smithi061.stdout: "admin_socket_mode": "", 2023-12-06T18:27:16.981 INFO:teuthology.orchestra.run.smithi061.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2023-12-06T18:27:16.981 INFO:teuthology.orchestra.run.smithi061.stdout: "auth_client_required": "cephx, none", 2023-12-06T18:27:16.981 INFO:teuthology.orchestra.run.smithi061.stdout: "auth_cluster_required": "cephx", 2023-12-06T18:27:16.981 INFO:teuthology.orchestra.run.smithi061.stdout: "auth_debug": "true", 2023-12-06T18:27:16.981 INFO:teuthology.orchestra.run.smithi061.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2023-12-06T18:27:16.981 INFO:teuthology.orchestra.run.smithi061.stdout: "auth_mon_ticket_ttl": "660.000000", 2023-12-06T18:27:16.981 INFO:teuthology.orchestra.run.smithi061.stdout: "auth_service_required": "cephx", 2023-12-06T18:27:16.982 INFO:teuthology.orchestra.run.smithi061.stdout: "auth_service_ticket_ttl": "240.000000", 2023-12-06T18:27:16.982 INFO:teuthology.orchestra.run.smithi061.stdout: "auth_supported": "cephx", 2023-12-06T18:27:16.982 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_aio": "true", 2023-12-06T18:27:16.982 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_aio_max_queue_depth": "1024", 2023-12-06T18:27:16.982 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_aio_poll_ms": "250", 2023-12-06T18:27:16.982 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_aio_reap_max": "16", 2023-12-06T18:27:16.982 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_async_discard": "false", 2023-12-06T18:27:16.982 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_block_size": "4096", 2023-12-06T18:27:16.982 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_debug_aio": "false", 2023-12-06T18:27:16.982 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_debug_aio_log_age": "5.000000", 2023-12-06T18:27:16.982 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2023-12-06T18:27:16.982 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_debug_inflight_ios": "false", 2023-12-06T18:27:16.982 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_enable_discard": "false", 2023-12-06T18:27:16.982 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_flock_retry": "3", 2023-12-06T18:27:16.982 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_flock_retry_interval": "0.100000", 2023-12-06T18:27:16.983 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_inject_crash": "0", 2023-12-06T18:27:16.983 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_inject_crash_flush_delay": "2", 2023-12-06T18:27:16.983 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_ioring": "false", 2023-12-06T18:27:16.983 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_ioring_hipri": "false", 2023-12-06T18:27:16.983 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_ioring_sqthread_poll": "false", 2023-12-06T18:27:16.983 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_nvme_unbind_from_kernel": "false", 2023-12-06T18:27:16.983 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_read_buffer_alignment": "4096", 2023-12-06T18:27:16.983 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_read_preallocated_huge_buffers": "", 2023-12-06T18:27:16.983 INFO:teuthology.orchestra.run.smithi061.stdout: "bdev_type": "", 2023-12-06T18:27:16.983 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_alloc_size": "1048576", 2023-12-06T18:27:16.983 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_allocator": "hybrid", 2023-12-06T18:27:16.983 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_buffered_io": "true", 2023-12-06T18:27:16.983 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_check_for_zeros": "false", 2023-12-06T18:27:16.983 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_check_volume_selector_often": "false", 2023-12-06T18:27:16.983 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_check_volume_selector_on_umount": "false", 2023-12-06T18:27:16.984 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_compact_log_sync": "false", 2023-12-06T18:27:16.984 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2023-12-06T18:27:16.984 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2023-12-06T18:27:16.984 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_log_compact_min_size": "16777216", 2023-12-06T18:27:16.984 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_log_replay_check_allocations": "true", 2023-12-06T18:27:16.984 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_max_log_runway": "4194304", 2023-12-06T18:27:16.984 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_max_prefetch": "1048576", 2023-12-06T18:27:16.984 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_min_flush_size": "524288", 2023-12-06T18:27:16.984 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_min_log_runway": "1048576", 2023-12-06T18:27:16.984 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_replay_recovery": "false", 2023-12-06T18:27:16.984 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_replay_recovery_disable_compact": "false", 2023-12-06T18:27:16.984 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_shared_alloc_size": "65536", 2023-12-06T18:27:16.985 INFO:teuthology.orchestra.run.smithi061.stdout: "bluefs_sync_write": "false", 2023-12-06T18:27:16.985 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2023-12-06T18:27:16.985 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2023-12-06T18:27:16.985 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2023-12-06T18:27:16.985 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_allocation_from_file": "true", 2023-12-06T18:27:16.985 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_allocator": "hybrid", 2023-12-06T18:27:16.985 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2023-12-06T18:27:16.985 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2023-12-06T18:27:16.985 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2023-12-06T18:27:16.985 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2023-12-06T18:27:16.985 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2023-12-06T18:27:16.985 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_bitmapallocator_span_size": "1024", 2023-12-06T18:27:16.985 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_blobid_prealloc": "10240", 2023-12-06T18:27:16.985 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_block_create": "true", 2023-12-06T18:27:16.985 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_block_db_create": "false", 2023-12-06T18:27:16.986 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_block_db_path": "", 2023-12-06T18:27:16.986 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_block_db_size": "0", 2023-12-06T18:27:16.986 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_block_path": "", 2023-12-06T18:27:16.986 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_block_preallocate_file": "false", 2023-12-06T18:27:16.986 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_block_size": "107374182400", 2023-12-06T18:27:16.986 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_block_wal_create": "false", 2023-12-06T18:27:16.986 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_block_wal_path": "", 2023-12-06T18:27:16.986 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_block_wal_size": "100663296", 2023-12-06T18:27:16.986 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_bluefs": "true", 2023-12-06T18:27:16.986 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2023-12-06T18:27:16.986 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_bluefs_env_mirror": "false", 2023-12-06T18:27:16.986 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_bluefs_max_free": "10737418240", 2023-12-06T18:27:16.986 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2023-12-06T18:27:16.986 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2023-12-06T18:27:16.987 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2023-12-06T18:27:16.987 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2023-12-06T18:27:16.987 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2023-12-06T18:27:16.987 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_autotune": "true", 2023-12-06T18:27:16.987 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_autotune_interval": "5.000000", 2023-12-06T18:27:16.987 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2023-12-06T18:27:16.987 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_kv_ratio": "0.450000", 2023-12-06T18:27:16.987 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_meta_ratio": "0.450000", 2023-12-06T18:27:16.987 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_size": "0", 2023-12-06T18:27:16.987 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_size_hdd": "1073741824", 2023-12-06T18:27:16.987 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_size_ssd": "3221225472", 2023-12-06T18:27:16.987 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_trim_interval": "0.050000", 2023-12-06T18:27:16.987 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2023-12-06T18:27:16.987 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cache_type": "2q", 2023-12-06T18:27:16.987 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2023-12-06T18:27:16.987 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_clone_cow": "true", 2023-12-06T18:27:16.987 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_compression_algorithm": "snappy", 2023-12-06T18:27:16.988 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_compression_max_blob_size": "0", 2023-12-06T18:27:16.988 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2023-12-06T18:27:16.988 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2023-12-06T18:27:16.988 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_compression_min_blob_size": "0", 2023-12-06T18:27:16.988 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2023-12-06T18:27:16.988 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2023-12-06T18:27:16.988 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_compression_mode": "none", 2023-12-06T18:27:16.988 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_compression_required_ratio": "0.875000", 2023-12-06T18:27:16.988 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_csum_type": "crc32c", 2023-12-06T18:27:16.988 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_debug_enforce_settings": "default", 2023-12-06T18:27:16.988 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_debug_freelist": "false", 2023-12-06T18:27:16.988 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_debug_fsck_abort": "false", 2023-12-06T18:27:16.988 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2023-12-06T18:27:16.988 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2023-12-06T18:27:16.988 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_debug_inject_read_err": "false", 2023-12-06T18:27:16.988 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_debug_legacy_omap": "false", 2023-12-06T18:27:16.988 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_debug_no_reuse_blocks": "false", 2023-12-06T18:27:16.989 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_debug_omit_block_device_write": "false", 2023-12-06T18:27:16.989 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_debug_omit_kv_commit": "false", 2023-12-06T18:27:16.989 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2023-12-06T18:27:16.989 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_debug_prefill": "0.000000", 2023-12-06T18:27:16.989 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_debug_prefragment_max": "1048576", 2023-12-06T18:27:16.989 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_debug_random_read_err": "0.000000", 2023-12-06T18:27:16.989 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2023-12-06T18:27:16.989 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_debug_small_allocations": "0", 2023-12-06T18:27:16.989 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2023-12-06T18:27:16.989 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_default_buffered_read": "true", 2023-12-06T18:27:16.989 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_default_buffered_write": "false", 2023-12-06T18:27:16.989 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_deferred_batch_ops": "0", 2023-12-06T18:27:16.989 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2023-12-06T18:27:16.989 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2023-12-06T18:27:16.990 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_elastic_shared_blobs": "true", 2023-12-06T18:27:16.990 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2023-12-06T18:27:16.990 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_extent_map_shard_max_size": "1200", 2023-12-06T18:27:16.990 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_extent_map_shard_min_size": "150", 2023-12-06T18:27:16.990 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_extent_map_shard_target_size": "500", 2023-12-06T18:27:16.990 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2023-12-06T18:27:16.990 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fail_eio": "false", 2023-12-06T18:27:16.990 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_freelist_blocks_per_key": "128", 2023-12-06T18:27:16.990 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2023-12-06T18:27:16.990 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2023-12-06T18:27:16.990 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2023-12-06T18:27:16.990 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_on_mkfs": "true", 2023-12-06T18:27:16.990 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2023-12-06T18:27:16.990 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_on_mount": "false", 2023-12-06T18:27:16.990 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_on_mount_deep": "false", 2023-12-06T18:27:16.991 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_on_umount": "false", 2023-12-06T18:27:16.991 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_on_umount_deep": "false", 2023-12-06T18:27:16.991 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2023-12-06T18:27:16.991 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_quick_fix_threads": "2", 2023-12-06T18:27:16.991 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2023-12-06T18:27:16.991 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2023-12-06T18:27:16.991 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_gc_enable_blob_threshold": "0", 2023-12-06T18:27:16.991 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_gc_enable_total_threshold": "0", 2023-12-06T18:27:16.991 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2023-12-06T18:27:16.991 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_ignore_data_csum": "false", 2023-12-06T18:27:16.991 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2023-12-06T18:27:16.991 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_kvbackend": "rocksdb", 2023-12-06T18:27:16.991 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_log_collection_list_age": "60.000000", 2023-12-06T18:27:16.991 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2023-12-06T18:27:16.991 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_log_op_age": "5.000000", 2023-12-06T18:27:16.991 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_max_alloc_size": "0", 2023-12-06T18:27:16.992 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_max_blob_size": "0", 2023-12-06T18:27:16.992 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_max_blob_size_hdd": "65536", 2023-12-06T18:27:16.992 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_max_blob_size_ssd": "65536", 2023-12-06T18:27:16.992 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_max_defer_interval": "3.000000", 2023-12-06T18:27:16.992 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_max_deferred_txc": "32", 2023-12-06T18:27:16.992 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_min_alloc_size": "0", 2023-12-06T18:27:16.992 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_min_alloc_size_hdd": "4096", 2023-12-06T18:27:16.992 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_min_alloc_size_ssd": "4096", 2023-12-06T18:27:16.992 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_nid_prealloc": "1024", 2023-12-06T18:27:16.992 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_prefer_deferred_size": "0", 2023-12-06T18:27:16.992 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2023-12-06T18:27:16.992 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2023-12-06T18:27:16.992 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_qfsck_on_mount": "true", 2023-12-06T18:27:16.992 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_retry_disk_reads": "3", 2023-12-06T18:27:16.992 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_rocksdb_cf": "true", 2023-12-06T18:27:16.993 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2023-12-06T18:27:16.993 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2023-12-06T18:27:16.993 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_rocksdb_options_annex": "", 2023-12-06T18:27:16.993 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_spdk_coremask": "0x1", 2023-12-06T18:27:16.993 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_spdk_io_sleep": "5", 2023-12-06T18:27:16.993 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_spdk_max_io_completion": "0", 2023-12-06T18:27:16.993 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_spdk_mem": "512", 2023-12-06T18:27:16.993 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_sync_submit_transaction": "false", 2023-12-06T18:27:16.993 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_throttle_bytes": "67108864", 2023-12-06T18:27:16.993 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_throttle_cost_per_io": "0", 2023-12-06T18:27:16.993 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2023-12-06T18:27:16.993 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2023-12-06T18:27:16.993 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2023-12-06T18:27:16.994 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_throttle_trace_rate": "0.000000", 2023-12-06T18:27:16.994 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_tracing": "false", 2023-12-06T18:27:16.994 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2023-12-06T18:27:16.994 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2023-12-06T18:27:16.994 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_volume_selection_reserved": "0", 2023-12-06T18:27:16.994 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2023-12-06T18:27:16.994 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2023-12-06T18:27:16.994 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_warn_on_legacy_statfs": "true", 2023-12-06T18:27:16.994 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2023-12-06T18:27:16.994 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2023-12-06T18:27:16.994 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2023-12-06T18:27:16.994 INFO:teuthology.orchestra.run.smithi061.stdout: "bluestore_zero_block_detection": "false", 2023-12-06T18:27:16.994 INFO:teuthology.orchestra.run.smithi061.stdout: "cephadm_path": "/usr/sbin/cephadm", 2023-12-06T18:27:16.994 INFO:teuthology.orchestra.run.smithi061.stdout: "cephfs_mirror_action_update_interval": "2", 2023-12-06T18:27:16.994 INFO:teuthology.orchestra.run.smithi061.stdout: "cephfs_mirror_directory_scan_interval": "10", 2023-12-06T18:27:16.995 INFO:teuthology.orchestra.run.smithi061.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2023-12-06T18:27:16.995 INFO:teuthology.orchestra.run.smithi061.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2023-12-06T18:27:16.995 INFO:teuthology.orchestra.run.smithi061.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2023-12-06T18:27:16.995 INFO:teuthology.orchestra.run.smithi061.stdout: "cephfs_mirror_mount_timeout": "10", 2023-12-06T18:27:16.995 INFO:teuthology.orchestra.run.smithi061.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2023-12-06T18:27:16.995 INFO:teuthology.orchestra.run.smithi061.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2023-12-06T18:27:16.995 INFO:teuthology.orchestra.run.smithi061.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2023-12-06T18:27:16.995 INFO:teuthology.orchestra.run.smithi061.stdout: "cephsqlite_blocklist_dead_locker": "true", 2023-12-06T18:27:16.995 INFO:teuthology.orchestra.run.smithi061.stdout: "cephsqlite_lock_renewal_interval": "2000", 2023-12-06T18:27:16.995 INFO:teuthology.orchestra.run.smithi061.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2023-12-06T18:27:16.995 INFO:teuthology.orchestra.run.smithi061.stdout: "cephx_cluster_require_signatures": "false", 2023-12-06T18:27:16.995 INFO:teuthology.orchestra.run.smithi061.stdout: "cephx_cluster_require_version": "2", 2023-12-06T18:27:16.995 INFO:teuthology.orchestra.run.smithi061.stdout: "cephx_require_signatures": "false", 2023-12-06T18:27:16.996 INFO:teuthology.orchestra.run.smithi061.stdout: "cephx_require_version": "2", 2023-12-06T18:27:16.996 INFO:teuthology.orchestra.run.smithi061.stdout: "cephx_service_require_signatures": "false", 2023-12-06T18:27:16.996 INFO:teuthology.orchestra.run.smithi061.stdout: "cephx_service_require_version": "2", 2023-12-06T18:27:16.996 INFO:teuthology.orchestra.run.smithi061.stdout: "cephx_sign_messages": "true", 2023-12-06T18:27:16.996 INFO:teuthology.orchestra.run.smithi061.stdout: "chdir": "", 2023-12-06T18:27:16.996 INFO:teuthology.orchestra.run.smithi061.stdout: "client_acl_type": "", 2023-12-06T18:27:16.996 INFO:teuthology.orchestra.run.smithi061.stdout: "client_asio_thread_count": "2", 2023-12-06T18:27:16.996 INFO:teuthology.orchestra.run.smithi061.stdout: "client_cache_mid": "0.750000", 2023-12-06T18:27:16.996 INFO:teuthology.orchestra.run.smithi061.stdout: "client_cache_size": "16384", 2023-12-06T18:27:16.996 INFO:teuthology.orchestra.run.smithi061.stdout: "client_caps_release_delay": "5", 2023-12-06T18:27:16.996 INFO:teuthology.orchestra.run.smithi061.stdout: "client_check_pool_perm": "true", 2023-12-06T18:27:16.996 INFO:teuthology.orchestra.run.smithi061.stdout: "client_collect_and_send_global_metrics": "false", 2023-12-06T18:27:16.996 INFO:teuthology.orchestra.run.smithi061.stdout: "client_debug_force_sync_read": "false", 2023-12-06T18:27:16.996 INFO:teuthology.orchestra.run.smithi061.stdout: "client_debug_getattr_caps": "false", 2023-12-06T18:27:16.996 INFO:teuthology.orchestra.run.smithi061.stdout: "client_debug_inject_tick_delay": "0", 2023-12-06T18:27:16.996 INFO:teuthology.orchestra.run.smithi061.stdout: "client_die_on_failed_dentry_invalidate": "true", 2023-12-06T18:27:16.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_die_on_failed_remount": "false", 2023-12-06T18:27:16.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_dirsize_rbytes": "true", 2023-12-06T18:27:16.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_force_lazyio": "false", 2023-12-06T18:27:16.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_fs": "", 2023-12-06T18:27:16.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_inject_fixed_oldest_tid": "false", 2023-12-06T18:27:16.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_inject_release_failure": "false", 2023-12-06T18:27:16.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_max_inline_size": "4096", 2023-12-06T18:27:16.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_max_retries_on_remount_failure": "5", 2023-12-06T18:27:16.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_mds_namespace": "", 2023-12-06T18:27:16.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_metadata": "", 2023-12-06T18:27:16.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_mount_gid": "-1", 2023-12-06T18:27:16.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_mount_timeout": "300", 2023-12-06T18:27:16.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_mount_uid": "-1", 2023-12-06T18:27:16.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_mountpoint": "/", 2023-12-06T18:27:16.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_notify_timeout": "10", 2023-12-06T18:27:16.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_oc": "true", 2023-12-06T18:27:16.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_oc_max_dirty": "104857600", 2023-12-06T18:27:16.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_oc_max_dirty_age": "5.000000", 2023-12-06T18:27:16.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_oc_max_objects": "1000", 2023-12-06T18:27:16.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_oc_size": "209715200", 2023-12-06T18:27:16.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_oc_target_dirty": "8388608", 2023-12-06T18:27:16.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_permissions": "true", 2023-12-06T18:27:16.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_quota": "true", 2023-12-06T18:27:16.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_quota_df": "true", 2023-12-06T18:27:16.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_readahead_max_bytes": "0", 2023-12-06T18:27:16.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_readahead_max_periods": "4", 2023-12-06T18:27:16.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_readahead_min": "131072", 2023-12-06T18:27:16.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_reconnect_stale": "false", 2023-12-06T18:27:16.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_shutdown_timeout": "30", 2023-12-06T18:27:16.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_snapdir": ".snap", 2023-12-06T18:27:16.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_tick_interval": "1", 2023-12-06T18:27:16.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_trace": "", 2023-12-06T18:27:16.999 INFO:teuthology.orchestra.run.smithi061.stdout: "client_try_dentry_invalidate": "false", 2023-12-06T18:27:16.999 INFO:teuthology.orchestra.run.smithi061.stdout: "client_use_faked_inos": "false", 2023-12-06T18:27:16.999 INFO:teuthology.orchestra.run.smithi061.stdout: "client_use_random_mds": "false", 2023-12-06T18:27:16.999 INFO:teuthology.orchestra.run.smithi061.stdout: "clog_to_graylog": "false", 2023-12-06T18:27:16.999 INFO:teuthology.orchestra.run.smithi061.stdout: "clog_to_graylog_host": "127.0.0.1", 2023-12-06T18:27:16.999 INFO:teuthology.orchestra.run.smithi061.stdout: "clog_to_graylog_port": "12201", 2023-12-06T18:27:16.999 INFO:teuthology.orchestra.run.smithi061.stdout: "clog_to_monitors": "default=true", 2023-12-06T18:27:16.999 INFO:teuthology.orchestra.run.smithi061.stdout: "clog_to_syslog": "false", 2023-12-06T18:27:16.999 INFO:teuthology.orchestra.run.smithi061.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2023-12-06T18:27:16.999 INFO:teuthology.orchestra.run.smithi061.stdout: "clog_to_syslog_level": "info", 2023-12-06T18:27:16.999 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "-", 2023-12-06T18:27:16.999 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_network": "", 2023-12-06T18:27:16.999 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_network_interface": "", 2023-12-06T18:27:16.999 INFO:teuthology.orchestra.run.smithi061.stdout: "compressor_zlib_isal": "false", 2023-12-06T18:27:16.999 INFO:teuthology.orchestra.run.smithi061.stdout: "compressor_zlib_level": "5", 2023-12-06T18:27:17.000 INFO:teuthology.orchestra.run.smithi061.stdout: "compressor_zlib_winsize": "-15", 2023-12-06T18:27:17.000 INFO:teuthology.orchestra.run.smithi061.stdout: "compressor_zstd_level": "1", 2023-12-06T18:27:17.000 INFO:teuthology.orchestra.run.smithi061.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2023-12-06T18:27:17.000 INFO:teuthology.orchestra.run.smithi061.stdout: "crash_dir": "/var/lib/ceph/crash", 2023-12-06T18:27:17.000 INFO:teuthology.orchestra.run.smithi061.stdout: "crimson_alien_op_num_threads": "6", 2023-12-06T18:27:17.000 INFO:teuthology.orchestra.run.smithi061.stdout: "crimson_alien_thread_cpu_cores": "", 2023-12-06T18:27:17.000 INFO:teuthology.orchestra.run.smithi061.stdout: "crimson_osd_obc_lru_size": "10", 2023-12-06T18:27:17.000 INFO:teuthology.orchestra.run.smithi061.stdout: "crimson_osd_scheduler_concurrency": "0", 2023-12-06T18:27:17.000 INFO:teuthology.orchestra.run.smithi061.stdout: "crimson_seastar_smp": "1", 2023-12-06T18:27:17.000 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_location": "", 2023-12-06T18:27:17.000 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_location_hook": "", 2023-12-06T18:27:17.000 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_location_hook_timeout": "10", 2023-12-06T18:27:17.000 INFO:teuthology.orchestra.run.smithi061.stdout: "daemonize": "false", 2023-12-06T18:27:17.000 INFO:teuthology.orchestra.run.smithi061.stdout: "daos_pool": "tank", 2023-12-06T18:27:17.001 INFO:teuthology.orchestra.run.smithi061.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2023-12-06T18:27:17.001 INFO:teuthology.orchestra.run.smithi061.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2023-12-06T18:27:17.001 INFO:teuthology.orchestra.run.smithi061.stdout: "dbstore_db_name_prefix": "dbstore", 2023-12-06T18:27:17.001 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_alienstore": "0/5", 2023-12-06T18:27:17.001 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_allow_any_pool_priority": "false", 2023-12-06T18:27:17.001 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_asok": "1/5", 2023-12-06T18:27:17.001 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_asok_assert_abort": "false", 2023-12-06T18:27:17.001 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_asserts_on_shutdown": "true", 2023-12-06T18:27:17.001 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_auth": "20/20", 2023-12-06T18:27:17.001 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_bdev": "1/3", 2023-12-06T18:27:17.001 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_bluefs": "1/5", 2023-12-06T18:27:17.001 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_bluestore": "1/5", 2023-12-06T18:27:17.001 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_buffer": "0/1", 2023-12-06T18:27:17.001 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_ceph_exporter": "1/5", 2023-12-06T18:27:17.001 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_cephfs_mirror": "0/5", 2023-12-06T18:27:17.001 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_cephsqlite": "0/5", 2023-12-06T18:27:17.002 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_client": "0/5", 2023-12-06T18:27:17.002 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_compressor": "1/5", 2023-12-06T18:27:17.002 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_context": "0/1", 2023-12-06T18:27:17.002 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_crimson_interrupt": "0/5", 2023-12-06T18:27:17.002 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_crush": "1/1", 2023-12-06T18:27:17.002 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_crypto": "1/5", 2023-12-06T18:27:17.002 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_cyanstore": "0/5", 2023-12-06T18:27:17.002 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_deliberately_leak_memory": "false", 2023-12-06T18:27:17.002 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_disable_randomized_ping": "false", 2023-12-06T18:27:17.002 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_dpdk": "1/5", 2023-12-06T18:27:17.002 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_eventtrace": "1/5", 2023-12-06T18:27:17.002 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_filer": "0/1", 2023-12-06T18:27:17.002 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_filestore": "1/3", 2023-12-06T18:27:17.002 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_finisher": "1/1", 2023-12-06T18:27:17.002 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_fuse": "1/5", 2023-12-06T18:27:17.002 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_heartbeat_testing_span": "0", 2023-12-06T18:27:17.003 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_heartbeatmap": "1/5", 2023-12-06T18:27:17.003 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_immutable_obj_cache": "0/5", 2023-12-06T18:27:17.003 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_javaclient": "1/5", 2023-12-06T18:27:17.003 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_journal": "1/3", 2023-12-06T18:27:17.003 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_journaler": "0/5", 2023-12-06T18:27:17.003 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_kstore": "1/5", 2023-12-06T18:27:17.003 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_lockdep": "0/1", 2023-12-06T18:27:17.003 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_mclock": "1/5", 2023-12-06T18:27:17.003 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_mds": "1/5", 2023-12-06T18:27:17.003 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_mds_balancer": "1/5", 2023-12-06T18:27:17.003 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_mds_locker": "1/5", 2023-12-06T18:27:17.003 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_mds_log": "1/5", 2023-12-06T18:27:17.003 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_mds_log_expire": "1/5", 2023-12-06T18:27:17.003 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_mds_migrator": "1/5", 2023-12-06T18:27:17.003 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_memstore": "1/5", 2023-12-06T18:27:17.004 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_mgr": "2/5", 2023-12-06T18:27:17.004 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_mgrc": "1/5", 2023-12-06T18:27:17.004 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_mon": "20/20", 2023-12-06T18:27:17.004 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_monc": "0/10", 2023-12-06T18:27:17.004 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_ms": "1/1", 2023-12-06T18:27:17.004 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_none": "0/5", 2023-12-06T18:27:17.004 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_objclass": "0/5", 2023-12-06T18:27:17.004 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_objectcacher": "0/5", 2023-12-06T18:27:17.004 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_objecter": "0/1", 2023-12-06T18:27:17.004 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_optracker": "0/5", 2023-12-06T18:27:17.004 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_osd": "1/5", 2023-12-06T18:27:17.004 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_paxos": "20/20", 2023-12-06T18:27:17.004 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_perfcounter": "1/5", 2023-12-06T18:27:17.004 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_prioritycache": "1/5", 2023-12-06T18:27:17.005 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_rados": "0/5", 2023-12-06T18:27:17.005 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_rbd": "0/5", 2023-12-06T18:27:17.005 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_rbd_mirror": "0/5", 2023-12-06T18:27:17.005 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_rbd_pwl": "0/5", 2023-12-06T18:27:17.005 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_rbd_replay": "0/5", 2023-12-06T18:27:17.005 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_refs": "0/0", 2023-12-06T18:27:17.005 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_reserver": "1/1", 2023-12-06T18:27:17.005 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_rgw": "1/5", 2023-12-06T18:27:17.005 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_rgw_access": "1/5", 2023-12-06T18:27:17.005 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_rgw_datacache": "1/5", 2023-12-06T18:27:17.005 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_rgw_dbstore": "1/5", 2023-12-06T18:27:17.005 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_rgw_flight": "1/5", 2023-12-06T18:27:17.005 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_rgw_sync": "1/5", 2023-12-06T18:27:17.005 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_rocksdb": "4/5", 2023-12-06T18:27:17.005 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_seastore": "0/5", 2023-12-06T18:27:17.005 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_seastore_backref": "0/5", 2023-12-06T18:27:17.005 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_seastore_cache": "0/5", 2023-12-06T18:27:17.006 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_seastore_cleaner": "0/5", 2023-12-06T18:27:17.006 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_seastore_device": "0/5", 2023-12-06T18:27:17.006 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_seastore_epm": "0/5", 2023-12-06T18:27:17.006 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_seastore_fixedkv_tree": "0/5", 2023-12-06T18:27:17.006 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_seastore_journal": "0/5", 2023-12-06T18:27:17.006 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_seastore_lba": "0/5", 2023-12-06T18:27:17.006 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_seastore_odata": "0/5", 2023-12-06T18:27:17.006 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_seastore_omap": "0/5", 2023-12-06T18:27:17.006 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_seastore_onode": "0/5", 2023-12-06T18:27:17.006 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_seastore_t": "0/5", 2023-12-06T18:27:17.006 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_seastore_tm": "0/5", 2023-12-06T18:27:17.006 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_striper": "0/1", 2023-12-06T18:27:17.006 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_test": "0/5", 2023-12-06T18:27:17.006 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_throttle": "1/1", 2023-12-06T18:27:17.006 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_timer": "0/1", 2023-12-06T18:27:17.007 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_tp": "0/5", 2023-12-06T18:27:17.007 INFO:teuthology.orchestra.run.smithi061.stdout: "debug_trace": "1/5", 2023-12-06T18:27:17.007 INFO:teuthology.orchestra.run.smithi061.stdout: "defer_client_eviction_on_laggy_osds": "true", 2023-12-06T18:27:17.007 INFO:teuthology.orchestra.run.smithi061.stdout: "device_failure_prediction_mode": "none", 2023-12-06T18:27:17.007 INFO:teuthology.orchestra.run.smithi061.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2023-12-06T18:27:17.007 INFO:teuthology.orchestra.run.smithi061.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2023-12-06T18:27:17.007 INFO:teuthology.orchestra.run.smithi061.stdout: "err_to_graylog": "false", 2023-12-06T18:27:17.007 INFO:teuthology.orchestra.run.smithi061.stdout: "err_to_journald": "false", 2023-12-06T18:27:17.007 INFO:teuthology.orchestra.run.smithi061.stdout: "err_to_stderr": "true", 2023-12-06T18:27:17.007 INFO:teuthology.orchestra.run.smithi061.stdout: "err_to_syslog": "false", 2023-12-06T18:27:17.007 INFO:teuthology.orchestra.run.smithi061.stdout: "event_tracing": "false", 2023-12-06T18:27:17.007 INFO:teuthology.orchestra.run.smithi061.stdout: "exporter_addr": "0.0.0.0", 2023-12-06T18:27:17.007 INFO:teuthology.orchestra.run.smithi061.stdout: "exporter_http_port": "9926", 2023-12-06T18:27:17.007 INFO:teuthology.orchestra.run.smithi061.stdout: "exporter_prio_limit": "5", 2023-12-06T18:27:17.007 INFO:teuthology.orchestra.run.smithi061.stdout: "exporter_sock_dir": "/var/run/ceph/", 2023-12-06T18:27:17.007 INFO:teuthology.orchestra.run.smithi061.stdout: "exporter_sort_metrics": "true", 2023-12-06T18:27:17.008 INFO:teuthology.orchestra.run.smithi061.stdout: "exporter_stats_period": "5", 2023-12-06T18:27:17.008 INFO:teuthology.orchestra.run.smithi061.stdout: "fake_statfs_for_testing": "0", 2023-12-06T18:27:17.008 INFO:teuthology.orchestra.run.smithi061.stdout: "fatal_signal_handlers": "true", 2023-12-06T18:27:17.008 INFO:teuthology.orchestra.run.smithi061.stdout: "filer_max_purge_ops": "10", 2023-12-06T18:27:17.008 INFO:teuthology.orchestra.run.smithi061.stdout: "filer_max_truncate_ops": "128", 2023-12-06T18:27:17.008 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_apply_finisher_threads": "1", 2023-12-06T18:27:17.008 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_blackhole": "false", 2023-12-06T18:27:17.008 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_btrfs_clone_range": "true", 2023-12-06T18:27:17.008 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_btrfs_snap": "true", 2023-12-06T18:27:17.008 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_caller_concurrency": "10", 2023-12-06T18:27:17.008 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_collect_device_partition_information": "true", 2023-12-06T18:27:17.008 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_commit_timeout": "600.000000", 2023-12-06T18:27:17.008 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_debug_inject_read_err": "false", 2023-12-06T18:27:17.008 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_debug_omap_check": "false", 2023-12-06T18:27:17.008 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_debug_random_read_err": "0.000000", 2023-12-06T18:27:17.008 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_debug_verify_split": "false", 2023-12-06T18:27:17.008 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_dump_file": "", 2023-12-06T18:27:17.009 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2023-12-06T18:27:17.009 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_expected_throughput_ops": "200.000000", 2023-12-06T18:27:17.009 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_fadvise": "true", 2023-12-06T18:27:17.009 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_fail_eio": "true", 2023-12-06T18:27:17.009 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_fd_cache_shards": "16", 2023-12-06T18:27:17.009 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_fd_cache_size": "128", 2023-12-06T18:27:17.009 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_fiemap": "false", 2023-12-06T18:27:17.009 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_fiemap_threshold": "4096", 2023-12-06T18:27:17.009 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_fsync_flushes_journal_data": "false", 2023-12-06T18:27:17.009 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_index_retry_probability": "0.000000", 2023-12-06T18:27:17.009 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_inject_stall": "0", 2023-12-06T18:27:17.009 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_journal_parallel": "false", 2023-12-06T18:27:17.009 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_journal_trailing": "false", 2023-12-06T18:27:17.009 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_journal_writeahead": "false", 2023-12-06T18:27:17.009 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_kill_at": "0", 2023-12-06T18:27:17.010 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_max_alloc_hint_size": "1048576", 2023-12-06T18:27:17.010 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_max_inline_xattr_size": "0", 2023-12-06T18:27:17.010 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2023-12-06T18:27:17.010 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_max_inline_xattr_size_other": "512", 2023-12-06T18:27:17.010 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2023-12-06T18:27:17.010 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_max_inline_xattrs": "0", 2023-12-06T18:27:17.010 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2023-12-06T18:27:17.010 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_max_inline_xattrs_other": "2", 2023-12-06T18:27:17.010 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_max_inline_xattrs_xfs": "10", 2023-12-06T18:27:17.010 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_max_sync_interval": "5.000000", 2023-12-06T18:27:17.010 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_max_xattr_value_size": "0", 2023-12-06T18:27:17.010 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2023-12-06T18:27:17.010 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_max_xattr_value_size_other": "1024", 2023-12-06T18:27:17.010 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2023-12-06T18:27:17.010 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_merge_threshold": "-10", 2023-12-06T18:27:17.011 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_min_sync_interval": "0.010000", 2023-12-06T18:27:17.011 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_odsync_write": "false", 2023-12-06T18:27:17.011 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_omap_backend": "rocksdb", 2023-12-06T18:27:17.011 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_omap_backend_path": "", 2023-12-06T18:27:17.011 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_omap_header_cache_size": "1024", 2023-12-06T18:27:17.011 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_ondisk_finisher_threads": "1", 2023-12-06T18:27:17.011 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_op_thread_suicide_timeout": "180", 2023-12-06T18:27:17.011 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_op_thread_timeout": "60", 2023-12-06T18:27:17.011 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_op_threads": "2", 2023-12-06T18:27:17.011 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_punch_hole": "false", 2023-12-06T18:27:17.011 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2023-12-06T18:27:17.011 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2023-12-06T18:27:17.011 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2023-12-06T18:27:17.011 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_queue_high_threshhold": "0.900000", 2023-12-06T18:27:17.011 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_queue_low_threshhold": "0.300000", 2023-12-06T18:27:17.011 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_queue_max_bytes": "104857600", 2023-12-06T18:27:17.011 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2023-12-06T18:27:17.012 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2023-12-06T18:27:17.012 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2023-12-06T18:27:17.012 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_queue_max_ops": "50", 2023-12-06T18:27:17.012 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2023-12-06T18:27:17.012 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_seek_data_hole": "false", 2023-12-06T18:27:17.012 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_sloppy_crc": "false", 2023-12-06T18:27:17.012 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_sloppy_crc_block_size": "65536", 2023-12-06T18:27:17.012 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_splice": "false", 2023-12-06T18:27:17.012 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_split_multiple": "2", 2023-12-06T18:27:17.012 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_split_rand_factor": "20", 2023-12-06T18:27:17.012 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_update_to": "1000", 2023-12-06T18:27:17.012 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2023-12-06T18:27:17.012 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2023-12-06T18:27:17.012 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2023-12-06T18:27:17.012 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2023-12-06T18:27:17.012 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2023-12-06T18:27:17.012 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2023-12-06T18:27:17.013 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_wbthrottle_enable": "true", 2023-12-06T18:27:17.013 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2023-12-06T18:27:17.013 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2023-12-06T18:27:17.013 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2023-12-06T18:27:17.013 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2023-12-06T18:27:17.013 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2023-12-06T18:27:17.013 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2023-12-06T18:27:17.013 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_xfs_extsize": "false", 2023-12-06T18:27:17.013 INFO:teuthology.orchestra.run.smithi061.stdout: "filestore_zfs_snap": "false", 2023-12-06T18:27:17.013 INFO:teuthology.orchestra.run.smithi061.stdout: "fio_dir": "/tmp/fio", 2023-12-06T18:27:17.013 INFO:teuthology.orchestra.run.smithi061.stdout: "fsid": "497e3283-cc16-40ff-a72c-a7487474fb4b", 2023-12-06T18:27:17.013 INFO:teuthology.orchestra.run.smithi061.stdout: "fuse_allow_other": "true", 2023-12-06T18:27:17.013 INFO:teuthology.orchestra.run.smithi061.stdout: "fuse_atomic_o_trunc": "true", 2023-12-06T18:27:17.013 INFO:teuthology.orchestra.run.smithi061.stdout: "fuse_big_writes": "true", 2023-12-06T18:27:17.013 INFO:teuthology.orchestra.run.smithi061.stdout: "fuse_debug": "false", 2023-12-06T18:27:17.014 INFO:teuthology.orchestra.run.smithi061.stdout: "fuse_default_permissions": "false", 2023-12-06T18:27:17.014 INFO:teuthology.orchestra.run.smithi061.stdout: "fuse_disable_pagecache": "false", 2023-12-06T18:27:17.014 INFO:teuthology.orchestra.run.smithi061.stdout: "fuse_max_write": "0", 2023-12-06T18:27:17.014 INFO:teuthology.orchestra.run.smithi061.stdout: "fuse_multithreaded": "true", 2023-12-06T18:27:17.014 INFO:teuthology.orchestra.run.smithi061.stdout: "fuse_require_active_mds": "true", 2023-12-06T18:27:17.014 INFO:teuthology.orchestra.run.smithi061.stdout: "fuse_set_user_groups": "true", 2023-12-06T18:27:17.014 INFO:teuthology.orchestra.run.smithi061.stdout: "fuse_splice_move": "true", 2023-12-06T18:27:17.014 INFO:teuthology.orchestra.run.smithi061.stdout: "fuse_splice_read": "true", 2023-12-06T18:27:17.014 INFO:teuthology.orchestra.run.smithi061.stdout: "fuse_splice_write": "true", 2023-12-06T18:27:17.014 INFO:teuthology.orchestra.run.smithi061.stdout: "fuse_syncfs_on_mksnap": "true", 2023-12-06T18:27:17.014 INFO:teuthology.orchestra.run.smithi061.stdout: "fuse_use_invalidate_cb": "true", 2023-12-06T18:27:17.014 INFO:teuthology.orchestra.run.smithi061.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2023-12-06T18:27:17.014 INFO:teuthology.orchestra.run.smithi061.stdout: "gss_target_name": "ceph", 2023-12-06T18:27:17.014 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_file": "", 2023-12-06T18:27:17.015 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_inject_failure": "0", 2023-12-06T18:27:17.015 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_interval": "5", 2023-12-06T18:27:17.015 INFO:teuthology.orchestra.run.smithi061.stdout: "host": "smithi061", 2023-12-06T18:27:17.015 INFO:teuthology.orchestra.run.smithi061.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2023-12-06T18:27:17.015 INFO:teuthology.orchestra.run.smithi061.stdout: "immutable_object_cache_max_inflight_ops": "128", 2023-12-06T18:27:17.015 INFO:teuthology.orchestra.run.smithi061.stdout: "immutable_object_cache_max_size": "1073741824", 2023-12-06T18:27:17.015 INFO:teuthology.orchestra.run.smithi061.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2023-12-06T18:27:17.015 INFO:teuthology.orchestra.run.smithi061.stdout: "immutable_object_cache_qos_bps_burst": "0", 2023-12-06T18:27:17.015 INFO:teuthology.orchestra.run.smithi061.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2023-12-06T18:27:17.015 INFO:teuthology.orchestra.run.smithi061.stdout: "immutable_object_cache_qos_bps_limit": "0", 2023-12-06T18:27:17.015 INFO:teuthology.orchestra.run.smithi061.stdout: "immutable_object_cache_qos_iops_burst": "0", 2023-12-06T18:27:17.015 INFO:teuthology.orchestra.run.smithi061.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2023-12-06T18:27:17.015 INFO:teuthology.orchestra.run.smithi061.stdout: "immutable_object_cache_qos_iops_limit": "0", 2023-12-06T18:27:17.015 INFO:teuthology.orchestra.run.smithi061.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2023-12-06T18:27:17.015 INFO:teuthology.orchestra.run.smithi061.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2023-12-06T18:27:17.015 INFO:teuthology.orchestra.run.smithi061.stdout: "immutable_object_cache_watermark": "0.900000", 2023-12-06T18:27:17.015 INFO:teuthology.orchestra.run.smithi061.stdout: "inject_early_sigterm": "false", 2023-12-06T18:27:17.016 INFO:teuthology.orchestra.run.smithi061.stdout: "jaeger_agent_port": "6799", 2023-12-06T18:27:17.016 INFO:teuthology.orchestra.run.smithi061.stdout: "jaeger_tracing_enable": "false", 2023-12-06T18:27:17.016 INFO:teuthology.orchestra.run.smithi061.stdout: "journal_aio": "true", 2023-12-06T18:27:17.016 INFO:teuthology.orchestra.run.smithi061.stdout: "journal_align_min_size": "65536", 2023-12-06T18:27:17.016 INFO:teuthology.orchestra.run.smithi061.stdout: "journal_block_align": "true", 2023-12-06T18:27:17.016 INFO:teuthology.orchestra.run.smithi061.stdout: "journal_block_size": "4096", 2023-12-06T18:27:17.016 INFO:teuthology.orchestra.run.smithi061.stdout: "journal_dio": "true", 2023-12-06T18:27:17.016 INFO:teuthology.orchestra.run.smithi061.stdout: "journal_discard": "false", 2023-12-06T18:27:17.016 INFO:teuthology.orchestra.run.smithi061.stdout: "journal_force_aio": "false", 2023-12-06T18:27:17.016 INFO:teuthology.orchestra.run.smithi061.stdout: "journal_ignore_corruption": "false", 2023-12-06T18:27:17.016 INFO:teuthology.orchestra.run.smithi061.stdout: "journal_max_write_bytes": "10485760", 2023-12-06T18:27:17.016 INFO:teuthology.orchestra.run.smithi061.stdout: "journal_max_write_entries": "100", 2023-12-06T18:27:17.016 INFO:teuthology.orchestra.run.smithi061.stdout: "journal_replay_from": "0", 2023-12-06T18:27:17.016 INFO:teuthology.orchestra.run.smithi061.stdout: "journal_throttle_high_multiple": "0.000000", 2023-12-06T18:27:17.016 INFO:teuthology.orchestra.run.smithi061.stdout: "journal_throttle_high_threshhold": "0.900000", 2023-12-06T18:27:17.016 INFO:teuthology.orchestra.run.smithi061.stdout: "journal_throttle_low_threshhold": "0.600000", 2023-12-06T18:27:17.017 INFO:teuthology.orchestra.run.smithi061.stdout: "journal_throttle_max_multiple": "0.000000", 2023-12-06T18:27:17.017 INFO:teuthology.orchestra.run.smithi061.stdout: "journal_write_header_frequency": "0", 2023-12-06T18:27:17.017 INFO:teuthology.orchestra.run.smithi061.stdout: "journal_zero_on_create": "false", 2023-12-06T18:27:17.017 INFO:teuthology.orchestra.run.smithi061.stdout: "journaler_prefetch_periods": "10", 2023-12-06T18:27:17.017 INFO:teuthology.orchestra.run.smithi061.stdout: "journaler_prezero_periods": "5", 2023-12-06T18:27:17.017 INFO:teuthology.orchestra.run.smithi061.stdout: "journaler_write_head_interval": "15", 2023-12-06T18:27:17.017 INFO:teuthology.orchestra.run.smithi061.stdout: "key": "", 2023-12-06T18:27:17.017 INFO:teuthology.orchestra.run.smithi061.stdout: "keyfile": "", 2023-12-06T18:27:17.017 INFO:teuthology.orchestra.run.smithi061.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2023-12-06T18:27:17.017 INFO:teuthology.orchestra.run.smithi061.stdout: "kstore_backend": "rocksdb", 2023-12-06T18:27:17.017 INFO:teuthology.orchestra.run.smithi061.stdout: "kstore_default_stripe_size": "65536", 2023-12-06T18:27:17.017 INFO:teuthology.orchestra.run.smithi061.stdout: "kstore_fsck_on_mount": "false", 2023-12-06T18:27:17.017 INFO:teuthology.orchestra.run.smithi061.stdout: "kstore_fsck_on_mount_deep": "true", 2023-12-06T18:27:17.018 INFO:teuthology.orchestra.run.smithi061.stdout: "kstore_max_bytes": "67108864", 2023-12-06T18:27:17.018 INFO:teuthology.orchestra.run.smithi061.stdout: "kstore_max_ops": "512", 2023-12-06T18:27:17.018 INFO:teuthology.orchestra.run.smithi061.stdout: "kstore_nid_prealloc": "1024", 2023-12-06T18:27:17.018 INFO:teuthology.orchestra.run.smithi061.stdout: "kstore_onode_map_size": "1024", 2023-12-06T18:27:17.018 INFO:teuthology.orchestra.run.smithi061.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2023-12-06T18:27:17.018 INFO:teuthology.orchestra.run.smithi061.stdout: "kstore_sync_submit_transaction": "false", 2023-12-06T18:27:17.018 INFO:teuthology.orchestra.run.smithi061.stdout: "kstore_sync_transaction": "false", 2023-12-06T18:27:17.018 INFO:teuthology.orchestra.run.smithi061.stdout: "librados_thread_count": "2", 2023-12-06T18:27:17.018 INFO:teuthology.orchestra.run.smithi061.stdout: "lockdep": "false", 2023-12-06T18:27:17.018 INFO:teuthology.orchestra.run.smithi061.stdout: "lockdep_force_backtrace": "false", 2023-12-06T18:27:17.018 INFO:teuthology.orchestra.run.smithi061.stdout: "log_coarse_timestamps": "true", 2023-12-06T18:27:17.018 INFO:teuthology.orchestra.run.smithi061.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2023-12-06T18:27:17.018 INFO:teuthology.orchestra.run.smithi061.stdout: "log_flush_on_exit": "false", 2023-12-06T18:27:17.018 INFO:teuthology.orchestra.run.smithi061.stdout: "log_graylog_host": "127.0.0.1", 2023-12-06T18:27:17.019 INFO:teuthology.orchestra.run.smithi061.stdout: "log_graylog_port": "12201", 2023-12-06T18:27:17.019 INFO:teuthology.orchestra.run.smithi061.stdout: "log_max_new": "1000", 2023-12-06T18:27:17.019 INFO:teuthology.orchestra.run.smithi061.stdout: "log_max_recent": "10000", 2023-12-06T18:27:17.019 INFO:teuthology.orchestra.run.smithi061.stdout: "log_stderr_prefix": "", 2023-12-06T18:27:17.019 INFO:teuthology.orchestra.run.smithi061.stdout: "log_stop_at_utilization": "0.970000", 2023-12-06T18:27:17.019 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_file": "true", 2023-12-06T18:27:17.019 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_graylog": "false", 2023-12-06T18:27:17.019 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_journald": "false", 2023-12-06T18:27:17.019 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_stderr": "false", 2023-12-06T18:27:17.019 INFO:teuthology.orchestra.run.smithi061.stdout: "log_to_syslog": "false", 2023-12-06T18:27:17.019 INFO:teuthology.orchestra.run.smithi061.stdout: "mandatory_topic_permissions": "false", 2023-12-06T18:27:17.019 INFO:teuthology.orchestra.run.smithi061.stdout: "max_rotating_auth_attempts": "10", 2023-12-06T18:27:17.019 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2023-12-06T18:27:17.020 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_action_on_write_error": "1", 2023-12-06T18:27:17.020 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_alternate_name_max": "8192", 2023-12-06T18:27:17.020 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_asio_thread_count": "2", 2023-12-06T18:27:17.020 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_export_pin": "true", 2023-12-06T18:27:17.020 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_fragment_dirs": "true", 2023-12-06T18:27:17.020 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2023-12-06T18:27:17.020 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_fragment_interval": "5", 2023-12-06T18:27:17.020 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_fragment_size_max": "100000", 2023-12-06T18:27:17.020 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_idle_threshold": "0.000000", 2023-12-06T18:27:17.020 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_interval": "10", 2023-12-06T18:27:17.020 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_max": "-1", 2023-12-06T18:27:17.020 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_max_until": "-1", 2023-12-06T18:27:17.020 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_merge_size": "50", 2023-12-06T18:27:17.020 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_midchunk": "0.300000", 2023-12-06T18:27:17.020 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_min_rebalance": "0.100000", 2023-12-06T18:27:17.021 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_min_start": "0.200000", 2023-12-06T18:27:17.021 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_minchunk": "0.001000", 2023-12-06T18:27:17.021 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_mode": "0", 2023-12-06T18:27:17.021 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_need_max": "1.200000", 2023-12-06T18:27:17.021 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_need_min": "0.800000", 2023-12-06T18:27:17.021 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_overload_epochs": "2", 2023-12-06T18:27:17.021 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_replicate_threshold": "8000.000000", 2023-12-06T18:27:17.021 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_sample_interval": "3.000000", 2023-12-06T18:27:17.021 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_split_bits": "3", 2023-12-06T18:27:17.021 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_split_rd": "25000.000000", 2023-12-06T18:27:17.021 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_split_size": "10000", 2023-12-06T18:27:17.021 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_split_wr": "10000.000000", 2023-12-06T18:27:17.021 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_target_decay": "10.000000", 2023-12-06T18:27:17.021 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2023-12-06T18:27:17.021 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_beacon_grace": "15.000000", 2023-12-06T18:27:17.021 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_beacon_interval": "4.000000", 2023-12-06T18:27:17.021 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_beacon_mon_down_grace": "60", 2023-12-06T18:27:17.022 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_cache_memory_limit": "4294967296", 2023-12-06T18:27:17.022 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_cache_mid": "0.700000", 2023-12-06T18:27:17.022 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_cache_release_free_interval": "10", 2023-12-06T18:27:17.022 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_cache_reservation": "0.050000", 2023-12-06T18:27:17.022 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_cache_trim_decay_rate": "1.000000", 2023-12-06T18:27:17.022 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_cache_trim_interval": "1", 2023-12-06T18:27:17.022 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_cache_trim_threshold": "262144", 2023-12-06T18:27:17.022 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2023-12-06T18:27:17.022 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2023-12-06T18:27:17.022 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_client_delegate_inos_pct": "50", 2023-12-06T18:27:17.022 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_client_prealloc_inos": "1000", 2023-12-06T18:27:17.022 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2023-12-06T18:27:17.022 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_connect_bootstrapping": "false", 2023-12-06T18:27:17.022 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_damage_table_max_entries": "10000", 2023-12-06T18:27:17.022 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2023-12-06T18:27:17.023 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_debug_auth_pins": "false", 2023-12-06T18:27:17.023 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_debug_frag": "false", 2023-12-06T18:27:17.023 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_debug_scatterstat": "false", 2023-12-06T18:27:17.023 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_debug_subtrees": "false", 2023-12-06T18:27:17.023 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_decay_halflife": "5.000000", 2023-12-06T18:27:17.023 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_default_dir_hash": "2", 2023-12-06T18:27:17.023 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_defer_session_stale": "true", 2023-12-06T18:27:17.023 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_deny_all_reconnect": "false", 2023-12-06T18:27:17.023 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_dir_keys_per_op": "16384", 2023-12-06T18:27:17.023 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_dir_max_commit_size": "10", 2023-12-06T18:27:17.023 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_dir_max_entries": "0", 2023-12-06T18:27:17.023 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_dir_prefetch": "true", 2023-12-06T18:27:17.023 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_dirstat_min_interval": "1.000000", 2023-12-06T18:27:17.023 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_dump_cache_after_rejoin": "false", 2023-12-06T18:27:17.023 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_dump_cache_on_map": "false", 2023-12-06T18:27:17.024 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_dump_cache_threshold_file": "0", 2023-12-06T18:27:17.024 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2023-12-06T18:27:17.024 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_early_reply": "true", 2023-12-06T18:27:17.024 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_enable_op_tracker": "true", 2023-12-06T18:27:17.024 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_enforce_unique_name": "true", 2023-12-06T18:27:17.024 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_export_ephemeral_distributed": "true", 2023-12-06T18:27:17.024 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2023-12-06T18:27:17.024 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_export_ephemeral_random": "true", 2023-12-06T18:27:17.024 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_export_ephemeral_random_max": "0.010000", 2023-12-06T18:27:17.024 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_extraordinary_events_dump_interval": "0", 2023-12-06T18:27:17.024 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_forward_all_requests_to_auth": "false", 2023-12-06T18:27:17.024 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_freeze_tree_timeout": "30.000000", 2023-12-06T18:27:17.024 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_fscrypt_last_block_max_size": "4096", 2023-12-06T18:27:17.024 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_go_bad_corrupt_dentry": "true", 2023-12-06T18:27:17.025 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2023-12-06T18:27:17.025 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_health_cache_threshold": "1.500000", 2023-12-06T18:27:17.025 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_health_summarize_threshold": "10", 2023-12-06T18:27:17.025 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_heartbeat_grace": "15.000000", 2023-12-06T18:27:17.025 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_heartbeat_reset_grace": "1000", 2023-12-06T18:27:17.025 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_inject_health_dummy": "false", 2023-12-06T18:27:17.025 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2023-12-06T18:27:17.025 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_inject_migrator_session_race": "false", 2023-12-06T18:27:17.025 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2023-12-06T18:27:17.025 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_inject_skip_replaying_inotable": "false", 2023-12-06T18:27:17.025 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2023-12-06T18:27:17.025 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_join_fs": "", 2023-12-06T18:27:17.025 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_journal_format": "1", 2023-12-06T18:27:17.025 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_kill_create_at": "0", 2023-12-06T18:27:17.025 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_kill_export_at": "0", 2023-12-06T18:27:17.025 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_kill_import_at": "0", 2023-12-06T18:27:17.025 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_kill_journal_at": "0", 2023-12-06T18:27:17.026 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_kill_journal_expire_at": "0", 2023-12-06T18:27:17.026 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_kill_journal_replay_at": "0", 2023-12-06T18:27:17.026 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_kill_link_at": "0", 2023-12-06T18:27:17.026 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_kill_mdstable_at": "0", 2023-12-06T18:27:17.026 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_kill_openc_at": "0", 2023-12-06T18:27:17.026 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_kill_rename_at": "0", 2023-12-06T18:27:17.026 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_kill_shutdown_at": "0", 2023-12-06T18:27:17.026 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_kill_skip_replaying_inotable": "false", 2023-12-06T18:27:17.026 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_log_event_large_threshold": "524288", 2023-12-06T18:27:17.026 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_log_events_per_segment": "1024", 2023-12-06T18:27:17.026 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_log_major_segment_event_ratio": "12", 2023-12-06T18:27:17.026 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_log_max_events": "-1", 2023-12-06T18:27:17.026 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_log_max_segments": "128", 2023-12-06T18:27:17.026 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_log_pause": "false", 2023-12-06T18:27:17.027 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_log_segment_size": "0", 2023-12-06T18:27:17.027 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_log_skip_corrupt_events": "false", 2023-12-06T18:27:17.027 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_log_skip_unbounded_events": "false", 2023-12-06T18:27:17.027 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_log_warn_factor": "2.000000", 2023-12-06T18:27:17.027 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_max_caps_per_client": "1048576", 2023-12-06T18:27:17.027 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_max_completed_flushes": "100000", 2023-12-06T18:27:17.027 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_max_completed_requests": "100000", 2023-12-06T18:27:17.027 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_max_export_size": "20971520", 2023-12-06T18:27:17.027 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_max_file_recover": "32", 2023-12-06T18:27:17.027 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_max_purge_files": "64", 2023-12-06T18:27:17.027 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_max_purge_ops": "8192", 2023-12-06T18:27:17.027 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2023-12-06T18:27:17.027 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_max_scrub_ops_in_progress": "5", 2023-12-06T18:27:17.027 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_max_snaps_per_dir": "100", 2023-12-06T18:27:17.027 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_metrics_update_interval": "2", 2023-12-06T18:27:17.027 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_min_caps_per_client": "100", 2023-12-06T18:27:17.027 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_min_caps_working_set": "10000", 2023-12-06T18:27:17.028 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_mon_shutdown_timeout": "5.000000", 2023-12-06T18:27:17.028 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_numa_node": "-1", 2023-12-06T18:27:17.028 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_oft_prefetch_dirfrags": "false", 2023-12-06T18:27:17.028 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_op_complaint_time": "30.000000", 2023-12-06T18:27:17.028 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_op_history_duration": "600", 2023-12-06T18:27:17.028 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_op_history_size": "20", 2023-12-06T18:27:17.028 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_op_history_slow_op_size": "20", 2023-12-06T18:27:17.028 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_op_history_slow_op_threshold": "10", 2023-12-06T18:27:17.028 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_op_log_threshold": "5", 2023-12-06T18:27:17.028 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_ping_grace": "15", 2023-12-06T18:27:17.028 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_ping_interval": "5", 2023-12-06T18:27:17.028 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2023-12-06T18:27:17.028 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_recall_global_max_decay_threshold": "131072", 2023-12-06T18:27:17.028 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_recall_max_caps": "30000", 2023-12-06T18:27:17.028 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_recall_max_decay_rate": "1.500000", 2023-12-06T18:27:17.029 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_recall_max_decay_threshold": "131072", 2023-12-06T18:27:17.029 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_recall_warning_decay_rate": "60.000000", 2023-12-06T18:27:17.029 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_recall_warning_threshold": "262144", 2023-12-06T18:27:17.029 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_reconnect_timeout": "45.000000", 2023-12-06T18:27:17.029 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_replay_interval": "1.000000", 2023-12-06T18:27:17.029 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_replay_unsafe_with_closed_session": "false", 2023-12-06T18:27:17.029 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_request_load_average_decay_rate": "60.000000", 2023-12-06T18:27:17.029 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_root_ino_gid": "0", 2023-12-06T18:27:17.029 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_root_ino_uid": "0", 2023-12-06T18:27:17.029 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_scatter_nudge_interval": "5.000000", 2023-12-06T18:27:17.029 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_session_blocklist_on_evict": "true", 2023-12-06T18:27:17.029 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_session_blocklist_on_timeout": "true", 2023-12-06T18:27:17.029 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2023-12-06T18:27:17.029 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_session_cache_liveness_magnitude": "10", 2023-12-06T18:27:17.029 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2023-12-06T18:27:17.029 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_session_cap_acquisition_throttle": "100000", 2023-12-06T18:27:17.029 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2023-12-06T18:27:17.030 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_session_metadata_threshold": "16777216", 2023-12-06T18:27:17.030 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_sessionmap_keys_per_op": "1024", 2023-12-06T18:27:17.030 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_shutdown_check": "0", 2023-12-06T18:27:17.030 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_skip_ino": "0", 2023-12-06T18:27:17.030 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_sleep_rank_change": "0.000000", 2023-12-06T18:27:17.030 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_snap_max_uid": "4294967294", 2023-12-06T18:27:17.030 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_snap_min_uid": "0", 2023-12-06T18:27:17.030 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_snap_rstat": "false", 2023-12-06T18:27:17.030 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_standby_replay_damaged": "false", 2023-12-06T18:27:17.030 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_symlink_recovery": "true", 2023-12-06T18:27:17.030 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_task_status_update_interval": "2.000000", 2023-12-06T18:27:17.030 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_thrash_exports": "0", 2023-12-06T18:27:17.030 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_thrash_fragments": "0", 2023-12-06T18:27:17.031 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_tick_interval": "5.000000", 2023-12-06T18:27:17.031 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_valgrind_exit": "false", 2023-12-06T18:27:17.031 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_verify_backtrace": "1", 2023-12-06T18:27:17.031 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_verify_scatter": "false", 2023-12-06T18:27:17.031 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_wipe_ino_prealloc": "false", 2023-12-06T18:27:17.031 INFO:teuthology.orchestra.run.smithi061.stdout: "mds_wipe_sessions": "false", 2023-12-06T18:27:17.031 INFO:teuthology.orchestra.run.smithi061.stdout: "mempool_debug": "false", 2023-12-06T18:27:17.031 INFO:teuthology.orchestra.run.smithi061.stdout: "memstore_debug_omit_block_device_write": "false", 2023-12-06T18:27:17.031 INFO:teuthology.orchestra.run.smithi061.stdout: "memstore_device_bytes": "1073741824", 2023-12-06T18:27:17.031 INFO:teuthology.orchestra.run.smithi061.stdout: "memstore_page_set": "false", 2023-12-06T18:27:17.031 INFO:teuthology.orchestra.run.smithi061.stdout: "memstore_page_size": "65536", 2023-12-06T18:27:17.031 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_client_bytes": "134217728", 2023-12-06T18:27:17.031 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_client_messages": "512", 2023-12-06T18:27:17.032 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2023-12-06T18:27:17.032 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_connect_retry_interval": "1.000000", 2023-12-06T18:27:17.032 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2023-12-06T18:27:17.032 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2023-12-06T18:27:17.032 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_disabled_modules": "", 2023-12-06T18:27:17.032 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_initial_modules": "restful iostat nfs", 2023-12-06T18:27:17.032 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_max_pg_num_change": "128", 2023-12-06T18:27:17.032 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_mds_bytes": "134217728", 2023-12-06T18:27:17.032 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_mds_messages": "128", 2023-12-06T18:27:17.032 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2023-12-06T18:27:17.032 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_mon_bytes": "134217728", 2023-12-06T18:27:17.032 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_mon_messages": "128", 2023-12-06T18:27:17.032 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_osd_bytes": "536870912", 2023-12-06T18:27:17.032 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_osd_messages": "8192", 2023-12-06T18:27:17.032 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_pool": "true", 2023-12-06T18:27:17.032 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_service_beacon_grace": "60.000000", 2023-12-06T18:27:17.033 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_standby_modules": "true", 2023-12-06T18:27:17.033 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_stats_period": "5", 2023-12-06T18:27:17.033 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_stats_threshold": "5", 2023-12-06T18:27:17.033 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_tick_period": "2", 2023-12-06T18:27:17.033 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr_ttl_cache_expire_seconds": "0", 2023-12-06T18:27:17.033 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_accept_timeout_factor": "2.000000", 2023-12-06T18:27:17.033 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_allow_pool_delete": "true", 2023-12-06T18:27:17.033 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_allow_pool_size_one": "true", 2023-12-06T18:27:17.033 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_auth_validate_all_caps": "true", 2023-12-06T18:27:17.033 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2023-12-06T18:27:17.033 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2023-12-06T18:27:17.033 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_client_bytes": "104857600", 2023-12-06T18:27:17.033 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_client_directed_command_retry": "5", 2023-12-06T18:27:17.033 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_client_hunt_interval": "3.000000", 2023-12-06T18:27:17.033 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2023-12-06T18:27:17.033 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2023-12-06T18:27:17.034 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2023-12-06T18:27:17.034 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_client_hunt_parallel": "3", 2023-12-06T18:27:17.034 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_client_log_interval": "1.000000", 2023-12-06T18:27:17.034 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_client_max_log_entries_per_message": "1000", 2023-12-06T18:27:17.034 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_client_ping_interval": "10.000000", 2023-12-06T18:27:17.034 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_client_ping_timeout": "30.000000", 2023-12-06T18:27:17.034 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_client_target_rank": "-1", 2023-12-06T18:27:17.034 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_clock_drift_allowed": "1.000000", 2023-12-06T18:27:17.034 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2023-12-06T18:27:17.034 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2023-12-06T18:27:17.034 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_cluster_log_file_level": "debug", 2023-12-06T18:27:17.034 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_cluster_log_to_file": "true", 2023-12-06T18:27:17.034 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_cluster_log_to_graylog": "false", 2023-12-06T18:27:17.034 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2023-12-06T18:27:17.035 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_cluster_log_to_graylog_port": "12201", 2023-12-06T18:27:17.035 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_cluster_log_to_journald": "false", 2023-12-06T18:27:17.035 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_cluster_log_to_stderr": "false", 2023-12-06T18:27:17.035 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_cluster_log_to_syslog": "default=false", 2023-12-06T18:27:17.035 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2023-12-06T18:27:17.035 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_cluster_log_to_syslog_level": "info", 2023-12-06T18:27:17.035 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_compact_on_bootstrap": "false", 2023-12-06T18:27:17.035 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_compact_on_start": "false", 2023-12-06T18:27:17.035 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_compact_on_trim": "true", 2023-12-06T18:27:17.035 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_con_tracker_persist_interval": "10", 2023-12-06T18:27:17.035 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_con_tracker_score_halflife": "43200", 2023-12-06T18:27:17.035 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_config_key_max_entry_size": "65536", 2023-12-06T18:27:17.035 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_cpu_threads": "4", 2023-12-06T18:27:17.036 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_crush_min_required_version": "hammer", 2023-12-06T18:27:17.036 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_daemon_bytes": "419430400", 2023-12-06T18:27:17.036 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2023-12-06T18:27:17.036 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_data_avail_crit": "5", 2023-12-06T18:27:17.036 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_data_avail_warn": "5", 2023-12-06T18:27:17.036 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_data_size_warn": "16106127360", 2023-12-06T18:27:17.036 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_debug_block_osdmap_trim": "false", 2023-12-06T18:27:17.036 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_debug_deprecated_as_obsolete": "false", 2023-12-06T18:27:17.036 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_debug_dump_json": "false", 2023-12-06T18:27:17.036 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2023-12-06T18:27:17.036 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_debug_dump_transactions": "false", 2023-12-06T18:27:17.036 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_debug_extra_checks": "false", 2023-12-06T18:27:17.036 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_debug_no_initial_persistent_features": "false", 2023-12-06T18:27:17.036 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2023-12-06T18:27:17.036 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_debug_no_require_quincy": "false", 2023-12-06T18:27:17.037 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_debug_no_require_reef": "false", 2023-12-06T18:27:17.037 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2023-12-06T18:27:17.037 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_delta_reset_interval": "10.000000", 2023-12-06T18:27:17.037 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_dns_srv_name": "ceph-mon", 2023-12-06T18:27:17.037 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_down_mkfs_grace": "120", 2023-12-06T18:27:17.037 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_election_default_strategy": "3", 2023-12-06T18:27:17.037 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_election_timeout": "5.000000", 2023-12-06T18:27:17.037 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2023-12-06T18:27:17.037 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_elector_ping_divisor": "2", 2023-12-06T18:27:17.037 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_elector_ping_timeout": "2.000000", 2023-12-06T18:27:17.037 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_enable_op_tracker": "true", 2023-12-06T18:27:17.037 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_fake_pool_delete": "false", 2023-12-06T18:27:17.037 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_force_quorum_join": "false", 2023-12-06T18:27:17.037 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_globalid_prealloc": "10000", 2023-12-06T18:27:17.038 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_health_detail_to_clog": "false", 2023-12-06T18:27:17.038 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_health_log_update_period": "5", 2023-12-06T18:27:17.038 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_health_max_detail": "50", 2023-12-06T18:27:17.038 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_health_to_clog": "true", 2023-12-06T18:27:17.038 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_health_to_clog_interval": "600", 2023-12-06T18:27:17.038 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2023-12-06T18:27:17.038 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_host": "172.21.15.61,[v2:172.21.15.61:3301,v1:172.21.15.61:6790],172.21.15.152", 2023-12-06T18:27:17.038 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_host_override": "", 2023-12-06T18:27:17.038 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_initial_members": "", 2023-12-06T18:27:17.038 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2023-12-06T18:27:17.038 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2023-12-06T18:27:17.038 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_inject_transaction_delay_max": "10.000000", 2023-12-06T18:27:17.038 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2023-12-06T18:27:17.038 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_keyvaluedb": "rocksdb", 2023-12-06T18:27:17.039 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_lease": "5.000000", 2023-12-06T18:27:17.039 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2023-12-06T18:27:17.039 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_lease_renew_interval_factor": "0.600000", 2023-12-06T18:27:17.039 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_log_full_interval": "50", 2023-12-06T18:27:17.039 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_log_max": "10000", 2023-12-06T18:27:17.039 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_log_max_summary": "50", 2023-12-06T18:27:17.039 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_max_log_entries_per_event": "4096", 2023-12-06T18:27:17.039 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_max_log_epochs": "500", 2023-12-06T18:27:17.039 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_max_mdsmap_epochs": "500", 2023-12-06T18:27:17.039 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_max_mgrmap_epochs": "500", 2023-12-06T18:27:17.039 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_max_osd": "10000", 2023-12-06T18:27:17.039 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_max_pg_per_osd": "10000", 2023-12-06T18:27:17.039 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_max_pool_pg_num": "65536", 2023-12-06T18:27:17.039 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_max_snap_prune_per_epoch": "100", 2023-12-06T18:27:17.039 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_mds_blocklist_interval": "86400.000000", 2023-12-06T18:27:17.040 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_mds_force_trim_to": "0", 2023-12-06T18:27:17.040 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_mds_skip_sanity": "false", 2023-12-06T18:27:17.040 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_memory_autotune": "true", 2023-12-06T18:27:17.040 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_memory_target": "2147483648", 2023-12-06T18:27:17.040 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_mgr_beacon_grace": "30", 2023-12-06T18:27:17.040 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2023-12-06T18:27:17.040 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_mgr_digest_period": "5", 2023-12-06T18:27:17.040 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_mgr_inactive_grace": "60", 2023-12-06T18:27:17.040 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_mgr_mkfs_grace": "240", 2023-12-06T18:27:17.040 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2023-12-06T18:27:17.040 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_min_osdmap_epochs": "50", 2023-12-06T18:27:17.040 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_op_complaint_time": "30", 2023-12-06T18:27:17.040 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_op_history_duration": "600", 2023-12-06T18:27:17.041 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_op_history_size": "20", 2023-12-06T18:27:17.041 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_op_history_slow_op_size": "20", 2023-12-06T18:27:17.041 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_op_history_slow_op_threshold": "10", 2023-12-06T18:27:17.041 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_op_log_threshold": "5", 2023-12-06T18:27:17.041 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_adjust_down_out_interval": "true", 2023-12-06T18:27:17.041 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2023-12-06T18:27:17.041 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2023-12-06T18:27:17.041 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_auto_mark_in": "false", 2023-12-06T18:27:17.041 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_auto_mark_new_in": "true", 2023-12-06T18:27:17.041 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2023-12-06T18:27:17.041 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2023-12-06T18:27:17.041 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_cache_size": "500", 2023-12-06T18:27:17.041 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_cache_size_min": "134217728", 2023-12-06T18:27:17.041 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_crush_smoke_test": "false", 2023-12-06T18:27:17.042 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_destroyed_out_interval": "600", 2023-12-06T18:27:17.042 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_down_out_interval": "600", 2023-12-06T18:27:17.042 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_down_out_subtree_limit": "rack", 2023-12-06T18:27:17.042 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2023-12-06T18:27:17.042 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_force_trim_to": "0", 2023-12-06T18:27:17.042 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_full_ratio": "0.950000", 2023-12-06T18:27:17.042 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2023-12-06T18:27:17.042 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_laggy_halflife": "3600", 2023-12-06T18:27:17.042 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_laggy_max_interval": "300", 2023-12-06T18:27:17.042 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_laggy_weight": "0.300000", 2023-12-06T18:27:17.042 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2023-12-06T18:27:17.042 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_max_creating_pgs": "1024", 2023-12-06T18:27:17.042 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_max_initial_pgs": "1024", 2023-12-06T18:27:17.043 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_min_down_reporters": "2", 2023-12-06T18:27:17.043 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_min_in_ratio": "0.750000", 2023-12-06T18:27:17.043 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_min_up_ratio": "0.300000", 2023-12-06T18:27:17.043 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_nearfull_ratio": "0.850000", 2023-12-06T18:27:17.043 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_prime_pg_temp": "true", 2023-12-06T18:27:17.043 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2023-12-06T18:27:17.043 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2023-12-06T18:27:17.043 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_report_timeout": "900", 2023-12-06T18:27:17.043 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_reporter_subtree_level": "osd", 2023-12-06T18:27:17.043 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2023-12-06T18:27:17.043 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_warn_num_repaired": "10", 2023-12-06T18:27:17.043 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osd_warn_op_age": "32.000000", 2023-12-06T18:27:17.043 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osdmap_full_prune_enabled": "true", 2023-12-06T18:27:17.044 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osdmap_full_prune_interval": "2", 2023-12-06T18:27:17.044 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osdmap_full_prune_min": "15", 2023-12-06T18:27:17.044 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_osdmap_full_prune_txsize": "2", 2023-12-06T18:27:17.044 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2023-12-06T18:27:17.044 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_pg_stuck_threshold": "60", 2023-12-06T18:27:17.044 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2023-12-06T18:27:17.044 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_pg_warn_min_objects": "10000", 2023-12-06T18:27:17.044 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_pg_warn_min_per_osd": "0", 2023-12-06T18:27:17.044 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_pg_warn_min_pool_objects": "1000", 2023-12-06T18:27:17.044 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_pool_quota_crit_threshold": "0", 2023-12-06T18:27:17.044 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_pool_quota_warn_threshold": "0", 2023-12-06T18:27:17.044 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_probe_timeout": "2.000000", 2023-12-06T18:27:17.044 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_reweight_max_change": "0.050000", 2023-12-06T18:27:17.045 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_reweight_max_osds": "4", 2023-12-06T18:27:17.045 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_reweight_min_bytes_per_osd": "10", 2023-12-06T18:27:17.045 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_reweight_min_pgs_per_osd": "4", 2023-12-06T18:27:17.045 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2023-12-06T18:27:17.045 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2023-12-06T18:27:17.045 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2023-12-06T18:27:17.045 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_scrub_interval": "300", 2023-12-06T18:27:17.045 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_scrub_max_keys": "100", 2023-12-06T18:27:17.045 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_scrub_timeout": "300", 2023-12-06T18:27:17.045 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_session_timeout": "300", 2023-12-06T18:27:17.045 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_smart_report_timeout": "5", 2023-12-06T18:27:17.045 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_stat_smooth_intervals": "6", 2023-12-06T18:27:17.045 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2023-12-06T18:27:17.045 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_stretch_pool_min_size": "2", 2023-12-06T18:27:17.046 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_stretch_pool_size": "4", 2023-12-06T18:27:17.046 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2023-12-06T18:27:17.046 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_subscribe_interval": "86400.000000", 2023-12-06T18:27:17.046 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_sync_debug": "false", 2023-12-06T18:27:17.046 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_sync_max_payload_keys": "2000", 2023-12-06T18:27:17.046 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_sync_max_payload_size": "1048576", 2023-12-06T18:27:17.046 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_sync_provider_kill_at": "0", 2023-12-06T18:27:17.046 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_sync_requester_kill_at": "0", 2023-12-06T18:27:17.046 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_sync_timeout": "60.000000", 2023-12-06T18:27:17.046 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_target_pg_per_osd": "100", 2023-12-06T18:27:17.047 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_tick_interval": "5", 2023-12-06T18:27:17.047 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_timecheck_interval": "300.000000", 2023-12-06T18:27:17.047 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_timecheck_skew_interval": "30.000000", 2023-12-06T18:27:17.047 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_use_min_delay_socket": "false", 2023-12-06T18:27:17.047 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_older_version_delay": "604800", 2023-12-06T18:27:17.047 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2023-12-06T18:27:17.047 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2023-12-06T18:27:17.047 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2023-12-06T18:27:17.048 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_on_filestore_osds": "false", 2023-12-06T18:27:17.048 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2023-12-06T18:27:17.048 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2023-12-06T18:27:17.048 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2023-12-06T18:27:17.048 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_on_misplaced": "false", 2023-12-06T18:27:17.048 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2023-12-06T18:27:17.048 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_on_older_version": "true", 2023-12-06T18:27:17.048 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2023-12-06T18:27:17.049 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_on_pool_no_app": "true", 2023-12-06T18:27:17.049 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_on_pool_no_redundancy": "false", 2023-12-06T18:27:17.049 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2023-12-06T18:27:17.049 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2023-12-06T18:27:17.049 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2023-12-06T18:27:17.049 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_on_too_few_osds": "false", 2023-12-06T18:27:17.049 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2023-12-06T18:27:17.049 INFO:teuthology.orchestra.run.smithi061.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2023-12-06T18:27:17.050 INFO:teuthology.orchestra.run.smithi061.stdout: "monmap": "", 2023-12-06T18:27:17.050 INFO:teuthology.orchestra.run.smithi061.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2023-12-06T18:27:17.050 INFO:teuthology.orchestra.run.smithi061.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2023-12-06T18:27:17.050 INFO:teuthology.orchestra.run.smithi061.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2023-12-06T18:27:17.050 INFO:teuthology.orchestra.run.smithi061.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2023-12-06T18:27:17.050 INFO:teuthology.orchestra.run.smithi061.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2023-12-06T18:27:17.050 INFO:teuthology.orchestra.run.smithi061.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2023-12-06T18:27:17.050 INFO:teuthology.orchestra.run.smithi061.stdout: "motr_tracing_enabled": "false", 2023-12-06T18:27:17.051 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_async_op_threads": "3", 2023-12-06T18:27:17.051 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_async_rdma_buffer_size": "131072", 2023-12-06T18:27:17.051 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_async_rdma_cm": "false", 2023-12-06T18:27:17.051 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_async_rdma_device_name": "", 2023-12-06T18:27:17.051 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_async_rdma_dscp": "96", 2023-12-06T18:27:17.051 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_async_rdma_enable_hugepage": "false", 2023-12-06T18:27:17.051 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_async_rdma_gid_idx": "0", 2023-12-06T18:27:17.051 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_async_rdma_local_gid": "", 2023-12-06T18:27:17.052 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_async_rdma_polling_us": "1000", 2023-12-06T18:27:17.052 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_async_rdma_port_num": "1", 2023-12-06T18:27:17.052 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_async_rdma_receive_buffers": "32768", 2023-12-06T18:27:17.052 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_async_rdma_receive_queue_len": "4096", 2023-12-06T18:27:17.052 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_async_rdma_roce_ver": "1", 2023-12-06T18:27:17.052 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_async_rdma_send_buffers": "1024", 2023-12-06T18:27:17.052 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_async_rdma_sl": "3", 2023-12-06T18:27:17.052 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_async_rdma_support_srq": "true", 2023-12-06T18:27:17.053 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_async_rdma_type": "ib", 2023-12-06T18:27:17.053 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_async_reap_threshold": "5", 2023-12-06T18:27:17.053 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_bind_before_connect": "false", 2023-12-06T18:27:17.053 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_bind_ipv4": "true", 2023-12-06T18:27:17.053 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_bind_ipv6": "false", 2023-12-06T18:27:17.053 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_bind_msgr1": "false", 2023-12-06T18:27:17.053 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_bind_msgr2": "true", 2023-12-06T18:27:17.053 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_bind_port_max": "7568", 2023-12-06T18:27:17.054 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_bind_port_min": "6800", 2023-12-06T18:27:17.054 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_bind_prefer_ipv4": "false", 2023-12-06T18:27:17.054 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_bind_retry_count": "3", 2023-12-06T18:27:17.054 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_bind_retry_delay": "5", 2023-12-06T18:27:17.054 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_blackhole_client": "false", 2023-12-06T18:27:17.054 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_blackhole_mds": "false", 2023-12-06T18:27:17.054 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_blackhole_mgr": "false", 2023-12-06T18:27:17.054 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_blackhole_mon": "false", 2023-12-06T18:27:17.055 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_blackhole_osd": "false", 2023-12-06T18:27:17.055 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_client_mode": "crc secure", 2023-12-06T18:27:17.055 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_cluster_mode": "crc secure", 2023-12-06T18:27:17.055 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_cluster_type": "", 2023-12-06T18:27:17.055 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_compress_secure": "false", 2023-12-06T18:27:17.055 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_connection_idle_timeout": "900", 2023-12-06T18:27:17.055 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_connection_ready_timeout": "10", 2023-12-06T18:27:17.055 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_crc_data": "true", 2023-12-06T18:27:17.055 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_crc_header": "true", 2023-12-06T18:27:17.055 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_die_on_bad_msg": "false", 2023-12-06T18:27:17.055 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_die_on_bug": "true", 2023-12-06T18:27:17.055 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_die_on_old_message": "true", 2023-12-06T18:27:17.055 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_die_on_skipped_message": "false", 2023-12-06T18:27:17.056 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_die_on_unhandled_msg": "false", 2023-12-06T18:27:17.056 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_dispatch_throttle_bytes": "104857600", 2023-12-06T18:27:17.056 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_dpdk_coremask": "0xF", 2023-12-06T18:27:17.056 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_dpdk_debug_allow_loopback": "false", 2023-12-06T18:27:17.056 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_dpdk_devs_allowlist": "", 2023-12-06T18:27:17.056 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_dpdk_enable_tso": "true", 2023-12-06T18:27:17.056 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2023-12-06T18:27:17.056 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_dpdk_host_ipv4_addr": "", 2023-12-06T18:27:17.056 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_dpdk_hugepages": "", 2023-12-06T18:27:17.056 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_dpdk_hw_flow_control": "true", 2023-12-06T18:27:17.056 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2023-12-06T18:27:17.056 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_dpdk_lro": "true", 2023-12-06T18:27:17.056 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_dpdk_memory_channel": "4", 2023-12-06T18:27:17.056 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2023-12-06T18:27:17.056 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_dpdk_pmd": "", 2023-12-06T18:27:17.057 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_dpdk_port_id": "0", 2023-12-06T18:27:17.057 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2023-12-06T18:27:17.057 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_dump_corrupt_message_level": "1", 2023-12-06T18:27:17.057 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_dump_on_send": "false", 2023-12-06T18:27:17.057 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_initial_backoff": "0.200000", 2023-12-06T18:27:17.057 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_inject_delay_max": "1.000000", 2023-12-06T18:27:17.057 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_inject_delay_probability": "0.000000", 2023-12-06T18:27:17.057 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_inject_delay_type": "", 2023-12-06T18:27:17.057 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_inject_internal_delays": "0.000000", 2023-12-06T18:27:17.057 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_inject_network_congestion": "0", 2023-12-06T18:27:17.057 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_inject_socket_failures": "5000", 2023-12-06T18:27:17.057 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_learn_addr_from_peer": "true", 2023-12-06T18:27:17.057 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_max_accept_failures": "4", 2023-12-06T18:27:17.057 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_max_backoff": "15.000000", 2023-12-06T18:27:17.057 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_mon_client_mode": "secure crc", 2023-12-06T18:27:17.058 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_mon_cluster_mode": "secure crc", 2023-12-06T18:27:17.058 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_mon_service_mode": "secure crc", 2023-12-06T18:27:17.058 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_osd_compress_min_size": "1024", 2023-12-06T18:27:17.058 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_osd_compress_mode": "none", 2023-12-06T18:27:17.058 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_osd_compression_algorithm": "snappy", 2023-12-06T18:27:17.058 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2023-12-06T18:27:17.058 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_pq_min_cost": "65536", 2023-12-06T18:27:17.058 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_public_type": "", 2023-12-06T18:27:17.058 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_service_mode": "crc secure", 2023-12-06T18:27:17.058 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_tcp_listen_backlog": "512", 2023-12-06T18:27:17.058 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_tcp_nodelay": "true", 2023-12-06T18:27:17.058 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_tcp_prefetch_max_size": "4096", 2023-12-06T18:27:17.058 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_tcp_rcvbuf": "0", 2023-12-06T18:27:17.058 INFO:teuthology.orchestra.run.smithi061.stdout: "ms_type": "async", 2023-12-06T18:27:17.059 INFO:teuthology.orchestra.run.smithi061.stdout: "no_config_file": "false", 2023-12-06T18:27:17.059 INFO:teuthology.orchestra.run.smithi061.stdout: "objecter_completion_locks_per_session": "32", 2023-12-06T18:27:17.059 INFO:teuthology.orchestra.run.smithi061.stdout: "objecter_debug_inject_relock_delay": "false", 2023-12-06T18:27:17.059 INFO:teuthology.orchestra.run.smithi061.stdout: "objecter_inflight_op_bytes": "104857600", 2023-12-06T18:27:17.059 INFO:teuthology.orchestra.run.smithi061.stdout: "objecter_inflight_ops": "1024", 2023-12-06T18:27:17.059 INFO:teuthology.orchestra.run.smithi061.stdout: "objecter_inject_no_watch_ping": "false", 2023-12-06T18:27:17.059 INFO:teuthology.orchestra.run.smithi061.stdout: "objecter_retry_writes_after_first_reply": "false", 2023-12-06T18:27:17.059 INFO:teuthology.orchestra.run.smithi061.stdout: "objecter_tick_interval": "5.000000", 2023-12-06T18:27:17.059 INFO:teuthology.orchestra.run.smithi061.stdout: "objecter_timeout": "10.000000", 2023-12-06T18:27:17.059 INFO:teuthology.orchestra.run.smithi061.stdout: "objectstore_blackhole": "false", 2023-12-06T18:27:17.059 INFO:teuthology.orchestra.run.smithi061.stdout: "openssl_engine_opts": "", 2023-12-06T18:27:17.059 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_agent_delay_time": "5.000000", 2023-12-06T18:27:17.059 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_agent_hist_halflife": "1000", 2023-12-06T18:27:17.059 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_agent_max_low_ops": "2", 2023-12-06T18:27:17.060 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_agent_max_ops": "4", 2023-12-06T18:27:17.060 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_agent_min_evict_effort": "0.100000", 2023-12-06T18:27:17.060 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_agent_quantize_effort": "0.100000", 2023-12-06T18:27:17.060 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_agent_slop": "0.020000", 2023-12-06T18:27:17.060 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_aggregated_slow_ops_logging": "true", 2023-12-06T18:27:17.060 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_allow_recovery_below_min_size": "true", 2023-12-06T18:27:17.060 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_asio_thread_count": "2", 2023-12-06T18:27:17.060 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_async_recovery_min_cost": "100", 2023-12-06T18:27:17.060 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_auto_mark_unfound_lost": "false", 2023-12-06T18:27:17.060 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_backfill_retry_interval": "30.000000", 2023-12-06T18:27:17.060 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_backfill_scan_max": "512", 2023-12-06T18:27:17.060 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_backfill_scan_min": "64", 2023-12-06T18:27:17.060 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_backoff_on_degraded": "false", 2023-12-06T18:27:17.060 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_backoff_on_peering": "false", 2023-12-06T18:27:17.060 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_backoff_on_unfound": "true", 2023-12-06T18:27:17.061 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_beacon_report_interval": "300", 2023-12-06T18:27:17.061 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_bench_duration": "30", 2023-12-06T18:27:17.061 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_bench_large_size_max_throughput": "104857600", 2023-12-06T18:27:17.061 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_bench_max_block_size": "67108864", 2023-12-06T18:27:17.061 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_bench_small_size_max_iops": "100", 2023-12-06T18:27:17.061 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_blkin_trace_all": "false", 2023-12-06T18:27:17.061 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_blocked_scrub_grace_period": "120", 2023-12-06T18:27:17.061 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2023-12-06T18:27:17.061 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2023-12-06T18:27:17.061 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2023-12-06T18:27:17.061 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_check_for_log_corruption": "false", 2023-12-06T18:27:17.061 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_check_max_object_name_len_on_startup": "true", 2023-12-06T18:27:17.061 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2023-12-06T18:27:17.061 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2023-12-06T18:27:17.061 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2023-12-06T18:27:17.061 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_class_update_on_start": "true", 2023-12-06T18:27:17.061 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_client_message_cap": "256", 2023-12-06T18:27:17.062 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_client_message_size_cap": "524288000", 2023-12-06T18:27:17.062 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_client_op_priority": "63", 2023-12-06T18:27:17.062 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_client_watch_timeout": "30", 2023-12-06T18:27:17.062 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_command_max_records": "256", 2023-12-06T18:27:17.062 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_compact_on_start": "false", 2023-12-06T18:27:17.062 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_copyfrom_max_chunk": "8388608", 2023-12-06T18:27:17.062 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_crush_chooseleaf_type": "0", 2023-12-06T18:27:17.062 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_crush_initial_weight": "-1.000000", 2023-12-06T18:27:17.062 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_crush_update_on_start": "true", 2023-12-06T18:27:17.062 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_crush_update_weight_set": "true", 2023-12-06T18:27:17.062 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2023-12-06T18:27:17.062 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2023-12-06T18:27:17.062 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2023-12-06T18:27:17.062 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_drop_ping_duration": "0", 2023-12-06T18:27:17.062 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_drop_ping_probability": "0.000000", 2023-12-06T18:27:17.063 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_feed_pullee": "-1", 2023-12-06T18:27:17.063 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_inject_copyfrom_error": "false", 2023-12-06T18:27:17.063 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2023-12-06T18:27:17.063 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2023-12-06T18:27:17.063 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_misdirected_ops": "false", 2023-12-06T18:27:17.063 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_no_acting_change": "false", 2023-12-06T18:27:17.063 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_no_purge_strays": "false", 2023-12-06T18:27:17.063 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_op_order": "false", 2023-12-06T18:27:17.063 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_pg_log_writeout": "false", 2023-12-06T18:27:17.063 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_pretend_recovery_active": "false", 2023-12-06T18:27:17.063 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_random_push_read_error": "0.000000", 2023-12-06T18:27:17.063 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2023-12-06T18:27:17.063 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_shutdown": "false", 2023-12-06T18:27:17.063 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2023-12-06T18:27:17.063 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2023-12-06T18:27:17.063 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_trim_objects": "false", 2023-12-06T18:27:17.064 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_verify_cached_snaps": "false", 2023-12-06T18:27:17.064 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_verify_missing_on_start": "false", 2023-12-06T18:27:17.064 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_verify_snaps": "false", 2023-12-06T18:27:17.064 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_debug_verify_stray_on_activate": "false", 2023-12-06T18:27:17.064 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_deep_scrub_interval": "604800.000000", 2023-12-06T18:27:17.064 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_deep_scrub_keys": "1024", 2023-12-06T18:27:17.064 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2023-12-06T18:27:17.064 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2023-12-06T18:27:17.064 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2023-12-06T18:27:17.064 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_deep_scrub_stride": "524288", 2023-12-06T18:27:17.064 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2023-12-06T18:27:17.064 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_default_data_pool_replay_window": "5", 2023-12-06T18:27:17.064 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_default_notify_timeout": "30", 2023-12-06T18:27:17.064 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_delete_sleep": "0.000000", 2023-12-06T18:27:17.065 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_delete_sleep_hdd": "5.000000", 2023-12-06T18:27:17.065 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_delete_sleep_hybrid": "1.000000", 2023-12-06T18:27:17.065 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_delete_sleep_ssd": "1.000000", 2023-12-06T18:27:17.065 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_discard_disconnected_ops": "true", 2023-12-06T18:27:17.065 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_enable_op_tracker": "true", 2023-12-06T18:27:17.065 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2023-12-06T18:27:17.065 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_extblkdev_plugins": "vdo", 2023-12-06T18:27:17.065 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_failsafe_full_ratio": "0.970000", 2023-12-06T18:27:17.065 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_fast_fail_on_connection_refused": "true", 2023-12-06T18:27:17.065 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_fast_info": "true", 2023-12-06T18:27:17.065 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_fast_shutdown": "true", 2023-12-06T18:27:17.065 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_fast_shutdown_notify_mon": "true", 2023-12-06T18:27:17.065 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_fast_shutdown_timeout": "15", 2023-12-06T18:27:17.065 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_find_best_info_ignore_history_les": "false", 2023-12-06T18:27:17.065 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_force_auth_primary_missing_objects": "100", 2023-12-06T18:27:17.065 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2023-12-06T18:27:17.066 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_function_tracing": "false", 2023-12-06T18:27:17.066 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_heartbeat_grace": "80", 2023-12-06T18:27:17.066 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_heartbeat_interval": "6", 2023-12-06T18:27:17.066 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2023-12-06T18:27:17.066 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_heartbeat_min_peers": "10", 2023-12-06T18:27:17.066 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_heartbeat_min_size": "2000", 2023-12-06T18:27:17.066 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_heartbeat_stale": "600", 2023-12-06T18:27:17.066 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2023-12-06T18:27:17.066 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_hit_set_max_size": "100000", 2023-12-06T18:27:17.066 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_hit_set_min_size": "1000", 2023-12-06T18:27:17.066 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_hit_set_namespace": ".ceph-internal", 2023-12-06T18:27:17.066 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_ignore_stale_divergent_priors": "false", 2023-12-06T18:27:17.066 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2023-12-06T18:27:17.066 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_inject_failure_on_pg_removal": "false", 2023-12-06T18:27:17.067 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2023-12-06T18:27:17.067 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_journal_flush_on_shutdown": "true", 2023-12-06T18:27:17.067 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_journal_size": "5120", 2023-12-06T18:27:17.067 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_kill_backfill_at": "0", 2023-12-06T18:27:17.067 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_loop_before_reset_tphandle": "64", 2023-12-06T18:27:17.067 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_map_cache_size": "50", 2023-12-06T18:27:17.067 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_map_dedup": "true", 2023-12-06T18:27:17.067 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_map_message_max": "40", 2023-12-06T18:27:17.067 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_map_message_max_bytes": "10485760", 2023-12-06T18:27:17.067 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_map_share_max_epochs": "40", 2023-12-06T18:27:17.067 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_attr_name_len": "100", 2023-12-06T18:27:17.067 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_attr_size": "0", 2023-12-06T18:27:17.067 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_backfills": "1", 2023-12-06T18:27:17.068 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_markdown_count": "5", 2023-12-06T18:27:17.068 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_markdown_period": "600", 2023-12-06T18:27:17.068 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_object_name_len": "2048", 2023-12-06T18:27:17.068 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_object_namespace_len": "256", 2023-12-06T18:27:17.068 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_object_size": "134217728", 2023-12-06T18:27:17.068 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2023-12-06T18:27:17.068 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_omap_entries_per_request": "1024", 2023-12-06T18:27:17.068 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_pg_blocked_by": "16", 2023-12-06T18:27:17.068 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_pg_log_entries": "10000", 2023-12-06T18:27:17.068 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2023-12-06T18:27:17.068 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_pgls": "1024", 2023-12-06T18:27:17.068 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_push_cost": "8388608", 2023-12-06T18:27:17.068 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_push_objects": "10", 2023-12-06T18:27:17.068 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_scrubs": "3", 2023-12-06T18:27:17.068 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2023-12-06T18:27:17.069 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_trimming_pgs": "2", 2023-12-06T18:27:17.069 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_write_op_reply_len": "64", 2023-12-06T18:27:17.069 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_max_write_size": "90", 2023-12-06T18:27:17.069 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2023-12-06T18:27:17.069 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2023-12-06T18:27:17.069 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2023-12-06T18:27:17.069 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2023-12-06T18:27:17.069 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2023-12-06T18:27:17.069 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2023-12-06T18:27:17.069 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2023-12-06T18:27:17.069 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_override_recovery_settings": "false", 2023-12-06T18:27:17.069 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_profile": "balanced", 2023-12-06T18:27:17.069 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2023-12-06T18:27:17.069 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2023-12-06T18:27:17.070 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2023-12-06T18:27:17.070 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2023-12-06T18:27:17.070 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2023-12-06T18:27:17.070 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2023-12-06T18:27:17.070 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2023-12-06T18:27:17.070 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2023-12-06T18:27:17.070 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2023-12-06T18:27:17.070 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_scheduler_client_wgt": "1", 2023-12-06T18:27:17.070 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mclock_skip_benchmark": "false", 2023-12-06T18:27:17.070 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_memory_base": "805306368", 2023-12-06T18:27:17.070 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_memory_cache_min": "134217728", 2023-12-06T18:27:17.070 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_memory_cache_resize_interval": "1.000000", 2023-12-06T18:27:17.071 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_memory_expected_fragmentation": "0.150000", 2023-12-06T18:27:17.071 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_memory_target": "4294967296", 2023-12-06T18:27:17.071 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_memory_target_autotune": "false", 2023-12-06T18:27:17.071 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2023-12-06T18:27:17.071 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_min_pg_log_entries": "250", 2023-12-06T18:27:17.071 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_min_recovery_priority": "0", 2023-12-06T18:27:17.071 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mon_heartbeat_interval": "30", 2023-12-06T18:27:17.071 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2023-12-06T18:27:17.071 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mon_report_interval": "5", 2023-12-06T18:27:17.071 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mon_report_max_in_flight": "2", 2023-12-06T18:27:17.071 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_mon_shutdown_timeout": "5.000000", 2023-12-06T18:27:17.071 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_num_cache_shards": "32", 2023-12-06T18:27:17.071 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_num_op_tracker_shard": "32", 2023-12-06T18:27:17.071 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_numa_auto_affinity": "true", 2023-12-06T18:27:17.071 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_numa_node": "-1", 2023-12-06T18:27:17.071 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_numa_prefer_iface": "true", 2023-12-06T18:27:17.071 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_object_clean_region_max_num_intervals": "10", 2023-12-06T18:27:17.072 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_objecter_finishers": "1", 2023-12-06T18:27:17.072 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_objectstore": "bluestore", 2023-12-06T18:27:17.072 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_objectstore_fuse": "false", 2023-12-06T18:27:17.072 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_objectstore_tracing": "false", 2023-12-06T18:27:17.072 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_op_complaint_time": "30.000000", 2023-12-06T18:27:17.072 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_op_history_duration": "600", 2023-12-06T18:27:17.072 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_op_history_size": "20", 2023-12-06T18:27:17.072 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_op_history_slow_op_size": "20", 2023-12-06T18:27:17.072 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2023-12-06T18:27:17.072 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_op_log_threshold": "5", 2023-12-06T18:27:17.072 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_op_num_shards": "0", 2023-12-06T18:27:17.072 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_op_num_shards_hdd": "5", 2023-12-06T18:27:17.072 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_op_num_shards_ssd": "8", 2023-12-06T18:27:17.072 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_op_num_threads_per_shard": "0", 2023-12-06T18:27:17.072 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2023-12-06T18:27:17.072 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2023-12-06T18:27:17.073 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2023-12-06T18:27:17.073 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_op_pq_min_cost": "65536", 2023-12-06T18:27:17.073 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_op_queue": "mclock_scheduler", 2023-12-06T18:27:17.073 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_op_queue_cut_off": "high", 2023-12-06T18:27:17.073 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_op_thread_suicide_timeout": "150", 2023-12-06T18:27:17.073 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_op_thread_timeout": "15", 2023-12-06T18:27:17.073 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_open_classes_on_start": "true", 2023-12-06T18:27:17.073 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_os_flags": "0", 2023-12-06T18:27:17.073 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_peering_op_priority": "255", 2023-12-06T18:27:17.073 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pg_delete_cost": "1048576", 2023-12-06T18:27:17.073 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pg_delete_priority": "5", 2023-12-06T18:27:17.073 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2023-12-06T18:27:17.073 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2023-12-06T18:27:17.073 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pg_log_dups_tracked": "3000", 2023-12-06T18:27:17.073 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pg_log_trim_max": "10000", 2023-12-06T18:27:17.074 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pg_log_trim_min": "100", 2023-12-06T18:27:17.074 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2023-12-06T18:27:17.074 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pg_object_context_cache_count": "64", 2023-12-06T18:27:17.074 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pg_stat_report_interval_max": "500", 2023-12-06T18:27:17.074 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2023-12-06T18:27:17.074 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_cache_min_evict_age": "0", 2023-12-06T18:27:17.074 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_cache_min_flush_age": "0", 2023-12-06T18:27:17.074 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2023-12-06T18:27:17.074 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2023-12-06T18:27:17.074 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2023-12-06T18:27:17.074 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_crimson": "false", 2023-12-06T18:27:17.074 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_crush_rule": "-1", 2023-12-06T18:27:17.074 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_ec_fast_read": "false", 2023-12-06T18:27:17.074 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2023-12-06T18:27:17.074 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_flag_bulk": "false", 2023-12-06T18:27:17.074 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_flag_hashpspool": "true", 2023-12-06T18:27:17.075 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_flag_nodelete": "false", 2023-12-06T18:27:17.075 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_flag_nopgchange": "false", 2023-12-06T18:27:17.075 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_flag_nosizechange": "false", 2023-12-06T18:27:17.075 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_flags": "0", 2023-12-06T18:27:17.075 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2023-12-06T18:27:17.075 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_min_size": "0", 2023-12-06T18:27:17.075 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2023-12-06T18:27:17.075 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_pg_num": "32", 2023-12-06T18:27:17.075 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_pgp_num": "0", 2023-12-06T18:27:17.075 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2023-12-06T18:27:17.075 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_size": "2", 2023-12-06T18:27:17.075 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_default_type": "replicated", 2023-12-06T18:27:17.075 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2023-12-06T18:27:17.075 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_pool_use_gmt_hitset": "true", 2023-12-06T18:27:17.075 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_push_per_object_cost": "1000", 2023-12-06T18:27:17.075 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_read_ec_check_for_errors": "false", 2023-12-06T18:27:17.075 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_recover_clone_overlap": "true", 2023-12-06T18:27:17.076 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_recover_clone_overlap_limit": "10", 2023-12-06T18:27:17.076 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_recovery_cost": "20971520", 2023-12-06T18:27:17.076 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_recovery_delay_start": "0.000000", 2023-12-06T18:27:17.076 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_recovery_max_active": "0", 2023-12-06T18:27:17.076 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_recovery_max_active_hdd": "3", 2023-12-06T18:27:17.076 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_recovery_max_active_ssd": "10", 2023-12-06T18:27:17.076 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_recovery_max_chunk": "8388608", 2023-12-06T18:27:17.076 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2023-12-06T18:27:17.076 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_recovery_max_single_start": "1", 2023-12-06T18:27:17.076 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_recovery_op_priority": "3", 2023-12-06T18:27:17.076 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_recovery_op_warn_multiple": "16", 2023-12-06T18:27:17.076 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_recovery_priority": "5", 2023-12-06T18:27:17.076 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_recovery_retry_interval": "30.000000", 2023-12-06T18:27:17.076 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_recovery_sleep": "0.000000", 2023-12-06T18:27:17.076 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_recovery_sleep_hdd": "0.100000", 2023-12-06T18:27:17.076 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2023-12-06T18:27:17.077 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_recovery_sleep_ssd": "0.000000", 2023-12-06T18:27:17.077 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_repair_during_recovery": "false", 2023-12-06T18:27:17.077 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_requested_scrub_priority": "120", 2023-12-06T18:27:17.077 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2023-12-06T18:27:17.077 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_rollback_to_cluster_snap": "", 2023-12-06T18:27:17.077 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_auto_repair": "false", 2023-12-06T18:27:17.077 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_auto_repair_num_errors": "5", 2023-12-06T18:27:17.077 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_backoff_ratio": "0.660000", 2023-12-06T18:27:17.077 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_begin_hour": "0", 2023-12-06T18:27:17.077 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_begin_week_day": "0", 2023-12-06T18:27:17.077 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_chunk_max": "25", 2023-12-06T18:27:17.077 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_chunk_min": "5", 2023-12-06T18:27:17.077 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_cost": "52428800", 2023-12-06T18:27:17.077 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_during_recovery": "false", 2023-12-06T18:27:17.078 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_end_hour": "0", 2023-12-06T18:27:17.078 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_end_week_day": "0", 2023-12-06T18:27:17.078 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_event_cost": "4096", 2023-12-06T18:27:17.078 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_extended_sleep": "0.000000", 2023-12-06T18:27:17.078 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2023-12-06T18:27:17.078 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_invalid_stats": "true", 2023-12-06T18:27:17.078 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_load_threshold": "0.500000", 2023-12-06T18:27:17.078 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_max_interval": "604800.000000", 2023-12-06T18:27:17.078 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_max_preemptions": "5", 2023-12-06T18:27:17.078 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_min_interval": "86400.000000", 2023-12-06T18:27:17.078 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_priority": "5", 2023-12-06T18:27:17.078 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_reservation_timeout": "300000", 2023-12-06T18:27:17.078 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_sleep": "0.000000", 2023-12-06T18:27:17.078 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_scrub_slow_reservation_response": "30000", 2023-12-06T18:27:17.079 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_shallow_scrub_chunk_max": "100", 2023-12-06T18:27:17.079 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_shallow_scrub_chunk_min": "50", 2023-12-06T18:27:17.079 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_shutdown_pgref_assert": "false", 2023-12-06T18:27:17.079 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_skip_data_digest": "false", 2023-12-06T18:27:17.079 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_smart_report_timeout": "5", 2023-12-06T18:27:17.079 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_snap_trim_cost": "1048576", 2023-12-06T18:27:17.079 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_snap_trim_priority": "5", 2023-12-06T18:27:17.079 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_snap_trim_sleep": "0.000000", 2023-12-06T18:27:17.079 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2023-12-06T18:27:17.079 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2023-12-06T18:27:17.079 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2023-12-06T18:27:17.079 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_stats_update_period_not_scrubbing": "120", 2023-12-06T18:27:17.079 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_stats_update_period_scrubbing": "15", 2023-12-06T18:27:17.080 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2023-12-06T18:27:17.080 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_target_transaction_size": "30", 2023-12-06T18:27:17.080 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_tier_default_cache_hit_set_count": "4", 2023-12-06T18:27:17.080 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2023-12-06T18:27:17.080 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2023-12-06T18:27:17.080 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2023-12-06T18:27:17.080 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2023-12-06T18:27:17.080 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2023-12-06T18:27:17.080 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2023-12-06T18:27:17.080 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_tier_default_cache_mode": "writeback", 2023-12-06T18:27:17.080 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2023-12-06T18:27:17.080 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_tier_promote_max_objects_sec": "25", 2023-12-06T18:27:17.080 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_tracing": "false", 2023-12-06T18:27:17.080 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_use_stale_snap": "false", 2023-12-06T18:27:17.081 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2023-12-06T18:27:17.081 INFO:teuthology.orchestra.run.smithi061.stdout: "osdc_blkin_trace_all": "false", 2023-12-06T18:27:17.081 INFO:teuthology.orchestra.run.smithi061.stdout: "paxos_kill_at": "0", 2023-12-06T18:27:17.081 INFO:teuthology.orchestra.run.smithi061.stdout: "paxos_max_join_drift": "10", 2023-12-06T18:27:17.081 INFO:teuthology.orchestra.run.smithi061.stdout: "paxos_min": "500", 2023-12-06T18:27:17.081 INFO:teuthology.orchestra.run.smithi061.stdout: "paxos_min_wait": "0.050000", 2023-12-06T18:27:17.081 INFO:teuthology.orchestra.run.smithi061.stdout: "paxos_propose_interval": "1.000000", 2023-12-06T18:27:17.081 INFO:teuthology.orchestra.run.smithi061.stdout: "paxos_service_trim_max": "500", 2023-12-06T18:27:17.081 INFO:teuthology.orchestra.run.smithi061.stdout: "paxos_service_trim_max_multiplier": "20", 2023-12-06T18:27:17.081 INFO:teuthology.orchestra.run.smithi061.stdout: "paxos_service_trim_min": "10", 2023-12-06T18:27:17.081 INFO:teuthology.orchestra.run.smithi061.stdout: "paxos_stash_full_interval": "25", 2023-12-06T18:27:17.081 INFO:teuthology.orchestra.run.smithi061.stdout: "paxos_trim_max": "500", 2023-12-06T18:27:17.081 INFO:teuthology.orchestra.run.smithi061.stdout: "paxos_trim_min": "250", 2023-12-06T18:27:17.081 INFO:teuthology.orchestra.run.smithi061.stdout: "perf": "true", 2023-12-06T18:27:17.081 INFO:teuthology.orchestra.run.smithi061.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2023-12-06T18:27:17.082 INFO:teuthology.orchestra.run.smithi061.stdout: "plugin_crypto_accelerator": "crypto_isal", 2023-12-06T18:27:17.082 INFO:teuthology.orchestra.run.smithi061.stdout: "plugin_dir": "/usr/lib64/ceph", 2023-12-06T18:27:17.082 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "-", 2023-12-06T18:27:17.082 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrv": "", 2023-12-06T18:27:17.082 INFO:teuthology.orchestra.run.smithi061.stdout: "public_bind_addr": "-", 2023-12-06T18:27:17.082 INFO:teuthology.orchestra.run.smithi061.stdout: "public_network": "", 2023-12-06T18:27:17.082 INFO:teuthology.orchestra.run.smithi061.stdout: "public_network_interface": "", 2023-12-06T18:27:17.082 INFO:teuthology.orchestra.run.smithi061.stdout: "qat_compressor_enabled": "false", 2023-12-06T18:27:17.082 INFO:teuthology.orchestra.run.smithi061.stdout: "qat_compressor_session_max_number": "256", 2023-12-06T18:27:17.082 INFO:teuthology.orchestra.run.smithi061.stdout: "rados_mon_op_timeout": "0", 2023-12-06T18:27:17.082 INFO:teuthology.orchestra.run.smithi061.stdout: "rados_osd_op_timeout": "0", 2023-12-06T18:27:17.082 INFO:teuthology.orchestra.run.smithi061.stdout: "rados_tracing": "false", 2023-12-06T18:27:17.082 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_atime_update_interval": "60", 2023-12-06T18:27:17.082 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2023-12-06T18:27:17.083 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_balance_parent_reads": "false", 2023-12-06T18:27:17.083 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_balance_snap_reads": "false", 2023-12-06T18:27:17.083 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_blkin_trace_all": "false", 2023-12-06T18:27:17.083 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_blocklist_expire_seconds": "0", 2023-12-06T18:27:17.083 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_blocklist_on_break_lock": "true", 2023-12-06T18:27:17.083 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_cache": "true", 2023-12-06T18:27:17.083 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_cache_block_writes_upfront": "false", 2023-12-06T18:27:17.083 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_cache_max_dirty": "25165824", 2023-12-06T18:27:17.083 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_cache_max_dirty_age": "1.000000", 2023-12-06T18:27:17.083 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_cache_max_dirty_object": "0", 2023-12-06T18:27:17.083 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_cache_policy": "writearound", 2023-12-06T18:27:17.083 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_cache_size": "33554432", 2023-12-06T18:27:17.083 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_cache_target_dirty": "16777216", 2023-12-06T18:27:17.083 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_cache_writethrough_until_flush": "true", 2023-12-06T18:27:17.083 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_clone_copy_on_read": "false", 2023-12-06T18:27:17.083 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_compression_hint": "none", 2023-12-06T18:27:17.083 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_concurrent_management_ops": "10", 2023-12-06T18:27:17.084 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_config_pool_override_update_timestamp": "0", 2023-12-06T18:27:17.084 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_default_clone_format": "auto", 2023-12-06T18:27:17.084 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_default_data_pool": "", 2023-12-06T18:27:17.084 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_default_features": "61", 2023-12-06T18:27:17.084 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_default_format": "2", 2023-12-06T18:27:17.084 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_default_map_options": "", 2023-12-06T18:27:17.084 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_default_order": "22", 2023-12-06T18:27:17.084 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_default_pool": "rbd", 2023-12-06T18:27:17.084 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2023-12-06T18:27:17.084 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_default_stripe_count": "0", 2023-12-06T18:27:17.084 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_default_stripe_unit": "0", 2023-12-06T18:27:17.084 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_disable_zero_copy_writes": "true", 2023-12-06T18:27:17.084 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_discard_granularity_bytes": "65536", 2023-12-06T18:27:17.084 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_discard_on_zeroed_write_same": "true", 2023-12-06T18:27:17.085 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_enable_alloc_hint": "true", 2023-12-06T18:27:17.085 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2023-12-06T18:27:17.085 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_io_scheduler": "simple", 2023-12-06T18:27:17.085 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2023-12-06T18:27:17.085 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_journal_commit_age": "5.000000", 2023-12-06T18:27:17.085 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2023-12-06T18:27:17.085 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_journal_max_payload_bytes": "16384", 2023-12-06T18:27:17.085 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_journal_object_flush_age": "0.000000", 2023-12-06T18:27:17.085 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_journal_object_flush_bytes": "1048576", 2023-12-06T18:27:17.085 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_journal_object_flush_interval": "0", 2023-12-06T18:27:17.085 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2023-12-06T18:27:17.085 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2023-12-06T18:27:17.085 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_journal_order": "24", 2023-12-06T18:27:17.085 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_journal_pool": "", 2023-12-06T18:27:17.086 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_journal_splay_width": "4", 2023-12-06T18:27:17.086 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_localize_parent_reads": "false", 2023-12-06T18:27:17.086 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_localize_snap_reads": "false", 2023-12-06T18:27:17.086 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2023-12-06T18:27:17.086 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2023-12-06T18:27:17.086 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2023-12-06T18:27:17.086 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2023-12-06T18:27:17.086 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2023-12-06T18:27:17.086 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2023-12-06T18:27:17.086 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_image_policy_type": "simple", 2023-12-06T18:27:17.086 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2023-12-06T18:27:17.086 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_image_state_check_interval": "30", 2023-12-06T18:27:17.086 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2023-12-06T18:27:17.086 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2023-12-06T18:27:17.086 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2023-12-06T18:27:17.087 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2023-12-06T18:27:17.087 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2023-12-06T18:27:17.087 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_memory_autotune": "true", 2023-12-06T18:27:17.087 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_memory_base": "805306368", 2023-12-06T18:27:17.087 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2023-12-06T18:27:17.087 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_memory_cache_min": "134217728", 2023-12-06T18:27:17.087 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2023-12-06T18:27:17.087 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2023-12-06T18:27:17.087 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_memory_target": "4294967296", 2023-12-06T18:27:17.087 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_perf_stats_prio": "5", 2023-12-06T18:27:17.087 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2023-12-06T18:27:17.087 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2023-12-06T18:27:17.087 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirroring_delete_delay": "0", 2023-12-06T18:27:17.087 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2023-12-06T18:27:17.087 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirroring_replay_delay": "0", 2023-12-06T18:27:17.087 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2023-12-06T18:27:17.088 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2023-12-06T18:27:17.088 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_move_to_trash_on_remove": "false", 2023-12-06T18:27:17.088 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2023-12-06T18:27:17.088 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_mtime_update_interval": "60", 2023-12-06T18:27:17.088 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_non_blocking_aio": "true", 2023-12-06T18:27:17.088 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_op_thread_timeout": "60", 2023-12-06T18:27:17.088 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_op_threads": "1", 2023-12-06T18:27:17.088 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_parent_cache_enabled": "false", 2023-12-06T18:27:17.088 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_persistent_cache_mode": "disabled", 2023-12-06T18:27:17.088 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_persistent_cache_path": "/tmp", 2023-12-06T18:27:17.088 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_persistent_cache_size": "1073741824", 2023-12-06T18:27:17.088 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_plugins": "", 2023-12-06T18:27:17.088 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_bps_burst": "0", 2023-12-06T18:27:17.088 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_bps_burst_seconds": "1", 2023-12-06T18:27:17.088 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_bps_limit": "0", 2023-12-06T18:27:17.089 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_exclude_ops": "0", 2023-12-06T18:27:17.089 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_iops_burst": "0", 2023-12-06T18:27:17.089 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_iops_burst_seconds": "1", 2023-12-06T18:27:17.089 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_iops_limit": "0", 2023-12-06T18:27:17.089 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_read_bps_burst": "0", 2023-12-06T18:27:17.089 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2023-12-06T18:27:17.089 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_read_bps_limit": "0", 2023-12-06T18:27:17.089 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_read_iops_burst": "0", 2023-12-06T18:27:17.089 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2023-12-06T18:27:17.089 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_read_iops_limit": "0", 2023-12-06T18:27:17.089 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_schedule_tick_min": "50", 2023-12-06T18:27:17.089 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_write_bps_burst": "0", 2023-12-06T18:27:17.089 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2023-12-06T18:27:17.089 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_write_bps_limit": "0", 2023-12-06T18:27:17.090 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_write_iops_burst": "0", 2023-12-06T18:27:17.090 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2023-12-06T18:27:17.090 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_qos_write_iops_limit": "0", 2023-12-06T18:27:17.090 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_quiesce_notification_attempts": "10", 2023-12-06T18:27:17.090 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_read_from_replica_policy": "default", 2023-12-06T18:27:17.090 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2023-12-06T18:27:17.090 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_readahead_max_bytes": "524288", 2023-12-06T18:27:17.090 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_readahead_trigger_requests": "10", 2023-12-06T18:27:17.090 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_request_timed_out_seconds": "30", 2023-12-06T18:27:17.090 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_skip_partial_discard": "true", 2023-12-06T18:27:17.090 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2023-12-06T18:27:17.090 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_tracing": "false", 2023-12-06T18:27:17.090 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_validate_names": "true", 2023-12-06T18:27:17.090 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd_validate_pool": "true", 2023-12-06T18:27:17.090 INFO:teuthology.orchestra.run.smithi061.stdout: "restapi_base_url": "", 2023-12-06T18:27:17.091 INFO:teuthology.orchestra.run.smithi061.stdout: "restapi_log_level": "", 2023-12-06T18:27:17.091 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_acl_grants_max_num": "100", 2023-12-06T18:27:17.091 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_admin_entry": "admin", 2023-12-06T18:27:17.091 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2023-12-06T18:27:17.091 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_backend_store": "rados", 2023-12-06T18:27:17.091 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_barbican_url": "", 2023-12-06T18:27:17.091 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_beast_enable_async": "true", 2023-12-06T18:27:17.091 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_bucket_counters_cache": "false", 2023-12-06T18:27:17.091 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_bucket_counters_cache_size": "10000", 2023-12-06T18:27:17.091 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2023-12-06T18:27:17.091 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_bucket_default_quota_max_size": "-1", 2023-12-06T18:27:17.091 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_bucket_index_max_aio": "128", 2023-12-06T18:27:17.091 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2023-12-06T18:27:17.091 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_bucket_quota_cache_size": "10000", 2023-12-06T18:27:17.091 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_bucket_quota_ttl": "600", 2023-12-06T18:27:17.092 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_bucket_sync_spawn_window": "20", 2023-12-06T18:27:17.092 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_cache_enabled": "true", 2023-12-06T18:27:17.092 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_cache_expiry_interval": "900", 2023-12-06T18:27:17.092 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_cache_lru_size": "10000", 2023-12-06T18:27:17.092 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_config_store": "rados", 2023-12-06T18:27:17.092 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_content_length_compat": "false", 2023-12-06T18:27:17.092 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_copy_obj_progress": "true", 2023-12-06T18:27:17.092 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2023-12-06T18:27:17.092 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_cors_rules_max_num": "100", 2023-12-06T18:27:17.092 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_cross_domain_policy": "", 2023-12-06T18:27:17.092 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_default_encryption_key": "", 2023-12-06T18:27:17.092 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_kmip_addr": "", 2023-12-06T18:27:17.092 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_kmip_ca_path": "", 2023-12-06T18:27:17.092 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_kmip_client_cert": "", 2023-12-06T18:27:17.092 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_kmip_client_key": "", 2023-12-06T18:27:17.092 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_kmip_kms_key_template": "", 2023-12-06T18:27:17.093 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_kmip_password": "", 2023-12-06T18:27:17.093 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2023-12-06T18:27:17.093 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_kmip_username": "", 2023-12-06T18:27:17.093 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_require_ssl": "true", 2023-12-06T18:27:17.093 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2023-12-06T18:27:17.093 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2023-12-06T18:27:17.093 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_sse_s3_backend": "vault", 2023-12-06T18:27:17.093 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2023-12-06T18:27:17.093 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2023-12-06T18:27:17.093 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2023-12-06T18:27:17.093 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2023-12-06T18:27:17.093 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2023-12-06T18:27:17.093 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2023-12-06T18:27:17.093 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2023-12-06T18:27:17.093 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2023-12-06T18:27:17.093 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2023-12-06T18:27:17.094 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2023-12-06T18:27:17.094 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2023-12-06T18:27:17.094 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_suppress_logs": "true", 2023-12-06T18:27:17.094 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_vault_addr": "", 2023-12-06T18:27:17.094 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_vault_auth": "token", 2023-12-06T18:27:17.094 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_vault_namespace": "", 2023-12-06T18:27:17.094 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_vault_prefix": "", 2023-12-06T18:27:17.094 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_vault_secret_engine": "transit", 2023-12-06T18:27:17.094 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_vault_ssl_cacert": "", 2023-12-06T18:27:17.094 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2023-12-06T18:27:17.094 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2023-12-06T18:27:17.094 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_vault_token_file": "", 2023-12-06T18:27:17.094 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_crypt_vault_verify_ssl": "true", 2023-12-06T18:27:17.094 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_curl_buffersize": "524288", 2023-12-06T18:27:17.094 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_curl_low_speed_limit": "1024", 2023-12-06T18:27:17.095 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_curl_low_speed_time": "300", 2023-12-06T18:27:17.095 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_curl_tcp_keepalive": "0", 2023-12-06T18:27:17.095 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_curl_wait_timeout_ms": "1000", 2023-12-06T18:27:17.095 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2023-12-06T18:27:17.095 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2023-12-06T18:27:17.095 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2023-12-06T18:27:17.095 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2023-12-06T18:27:17.095 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_d3n_l1_fadvise": "4", 2023-12-06T18:27:17.095 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2023-12-06T18:27:17.095 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_d3n_libaio_aio_num": "64", 2023-12-06T18:27:17.095 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_d3n_libaio_aio_threads": "20", 2023-12-06T18:27:17.095 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_d4n_host": "127.0.0.1", 2023-12-06T18:27:17.095 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_d4n_port": "6379", 2023-12-06T18:27:17.095 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2023-12-06T18:27:17.095 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_data_log_changes_size": "1000", 2023-12-06T18:27:17.095 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_data_log_num_shards": "128", 2023-12-06T18:27:17.096 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_data_log_obj_prefix": "data_log", 2023-12-06T18:27:17.096 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_data_log_window": "30", 2023-12-06T18:27:17.096 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_data_notify_interval_msec": "0", 2023-12-06T18:27:17.096 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_data_sync_poll_interval": "20", 2023-12-06T18:27:17.096 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_data_sync_spawn_window": "20", 2023-12-06T18:27:17.096 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2023-12-06T18:27:17.096 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_debug_inject_set_olh_err": "0", 2023-12-06T18:27:17.096 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_default_data_log_backing": "fifo", 2023-12-06T18:27:17.096 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_default_realm_info_oid": "default.realm", 2023-12-06T18:27:17.096 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_default_region_info_oid": "default.region", 2023-12-06T18:27:17.096 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_default_zone_info_oid": "default.zone", 2023-12-06T18:27:17.096 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2023-12-06T18:27:17.096 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_defer_to_bucket_acls": "", 2023-12-06T18:27:17.096 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_delete_multi_obj_max_num": "1000", 2023-12-06T18:27:17.096 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_dmclock_admin_lim": "0.000000", 2023-12-06T18:27:17.097 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_dmclock_admin_res": "100.000000", 2023-12-06T18:27:17.097 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2023-12-06T18:27:17.097 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_dmclock_auth_lim": "0.000000", 2023-12-06T18:27:17.097 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_dmclock_auth_res": "200.000000", 2023-12-06T18:27:17.097 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2023-12-06T18:27:17.097 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_dmclock_data_lim": "0.000000", 2023-12-06T18:27:17.097 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_dmclock_data_res": "500.000000", 2023-12-06T18:27:17.097 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_dmclock_data_wgt": "500.000000", 2023-12-06T18:27:17.097 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2023-12-06T18:27:17.097 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_dmclock_metadata_res": "500.000000", 2023-12-06T18:27:17.097 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2023-12-06T18:27:17.097 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_dns_name": "", 2023-12-06T18:27:17.097 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_dns_s3website_name": "", 2023-12-06T18:27:17.098 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_dynamic_resharding": "true", 2023-12-06T18:27:17.098 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2023-12-06T18:27:17.098 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_enable_gc_threads": "true", 2023-12-06T18:27:17.098 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_enable_lc_threads": "true", 2023-12-06T18:27:17.098 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_enable_ops_log": "false", 2023-12-06T18:27:17.098 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_enable_quota_threads": "true", 2023-12-06T18:27:17.098 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_enable_static_website": "false", 2023-12-06T18:27:17.098 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_enable_usage_log": "false", 2023-12-06T18:27:17.098 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_enforce_swift_acls": "true", 2023-12-06T18:27:17.098 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_exit_timeout_secs": "120", 2023-12-06T18:27:17.098 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_expose_bucket": "false", 2023-12-06T18:27:17.098 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_extended_http_attrs": "", 2023-12-06T18:27:17.098 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_filter": "none", 2023-12-06T18:27:17.098 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2023-12-06T18:27:17.099 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_frontends": "beast port=7480", 2023-12-06T18:27:17.099 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_gc_max_concurrent_io": "10", 2023-12-06T18:27:17.099 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_gc_max_deferred": "50", 2023-12-06T18:27:17.099 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2023-12-06T18:27:17.099 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_gc_max_objs": "32", 2023-12-06T18:27:17.099 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_gc_max_queue_size": "134213632", 2023-12-06T18:27:17.099 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_gc_max_trim_chunk": "16", 2023-12-06T18:27:17.099 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_gc_obj_min_wait": "7200", 2023-12-06T18:27:17.099 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_gc_processor_max_time": "3600", 2023-12-06T18:27:17.099 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_gc_processor_period": "3600", 2023-12-06T18:27:17.099 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_get_obj_max_req_size": "4194304", 2023-12-06T18:27:17.099 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_get_obj_window_size": "16777216", 2023-12-06T18:27:17.099 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_healthcheck_disabling_path": "", 2023-12-06T18:27:17.100 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_ignore_get_invalid_range": "false", 2023-12-06T18:27:17.100 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_init_timeout": "300", 2023-12-06T18:27:17.100 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2023-12-06T18:27:17.100 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2023-12-06T18:27:17.100 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_accepted_admin_roles": "", 2023-12-06T18:27:17.100 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_accepted_reader_roles": "", 2023-12-06T18:27:17.100 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2023-12-06T18:27:17.100 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_admin_domain": "", 2023-12-06T18:27:17.100 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_admin_password": "", 2023-12-06T18:27:17.100 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_admin_password_path": "", 2023-12-06T18:27:17.100 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_admin_project": "", 2023-12-06T18:27:17.100 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_admin_tenant": "", 2023-12-06T18:27:17.100 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_admin_token": "", 2023-12-06T18:27:17.100 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_admin_token_path": "", 2023-12-06T18:27:17.100 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_admin_user": "", 2023-12-06T18:27:17.101 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_api_version": "2", 2023-12-06T18:27:17.101 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_barbican_domain": "", 2023-12-06T18:27:17.101 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_barbican_password": "", 2023-12-06T18:27:17.101 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_barbican_project": "", 2023-12-06T18:27:17.101 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_barbican_tenant": "", 2023-12-06T18:27:17.101 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_barbican_user": "", 2023-12-06T18:27:17.101 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2023-12-06T18:27:17.101 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_implicit_tenants": "false", 2023-12-06T18:27:17.101 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2023-12-06T18:27:17.101 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_service_token_enabled": "false", 2023-12-06T18:27:17.101 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_token_cache_size": "10000", 2023-12-06T18:27:17.101 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_url": "", 2023-12-06T18:27:17.101 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_keystone_verify_ssl": "true", 2023-12-06T18:27:17.101 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_lc_debug_interval": "-1", 2023-12-06T18:27:17.101 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_lc_lock_max_time": "90", 2023-12-06T18:27:17.102 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_lc_max_objs": "32", 2023-12-06T18:27:17.102 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_lc_max_rules": "1000", 2023-12-06T18:27:17.102 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_lc_max_worker": "3", 2023-12-06T18:27:17.102 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_lc_max_wp_worker": "3", 2023-12-06T18:27:17.102 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_lc_thread_delay": "0", 2023-12-06T18:27:17.102 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2023-12-06T18:27:17.102 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_ldap_dnattr": "uid", 2023-12-06T18:27:17.102 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2023-12-06T18:27:17.102 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_ldap_searchfilter": "", 2023-12-06T18:27:17.102 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2023-12-06T18:27:17.102 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_ldap_uri": "ldaps://", 2023-12-06T18:27:17.102 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2023-12-06T18:27:17.102 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_list_bucket_min_readahead": "1000", 2023-12-06T18:27:17.102 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_list_buckets_max_chunk": "1000", 2023-12-06T18:27:17.102 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_log_http_headers": "", 2023-12-06T18:27:17.103 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_log_nonexistent_bucket": "false", 2023-12-06T18:27:17.103 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2023-12-06T18:27:17.103 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_log_object_name_utc": "false", 2023-12-06T18:27:17.103 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_lua_max_memory_per_state": "128000", 2023-12-06T18:27:17.103 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2023-12-06T18:27:17.103 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_max_attr_name_len": "0", 2023-12-06T18:27:17.103 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_max_attr_size": "0", 2023-12-06T18:27:17.103 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_max_attrs_num_in_req": "0", 2023-12-06T18:27:17.103 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_max_chunk_size": "4194304", 2023-12-06T18:27:17.103 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_max_concurrent_requests": "1024", 2023-12-06T18:27:17.103 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2023-12-06T18:27:17.103 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_max_dynamic_shards": "1999", 2023-12-06T18:27:17.103 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_max_listing_results": "1000", 2023-12-06T18:27:17.103 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_max_notify_retries": "10", 2023-12-06T18:27:17.103 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_max_objs_per_shard": "100000", 2023-12-06T18:27:17.103 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_max_put_param_size": "1048576", 2023-12-06T18:27:17.104 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_max_put_size": "5368709120", 2023-12-06T18:27:17.104 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_max_slo_entries": "1000", 2023-12-06T18:27:17.104 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_md_log_max_shards": "64", 2023-12-06T18:27:17.104 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_md_notify_interval_msec": "200", 2023-12-06T18:27:17.104 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_meta_sync_poll_interval": "20", 2023-12-06T18:27:17.104 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_meta_sync_spawn_window": "20", 2023-12-06T18:27:17.104 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_mime_types_file": "/etc/mime.types", 2023-12-06T18:27:17.104 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_mp_lock_max_time": "600", 2023-12-06T18:27:17.104 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_multi_obj_del_max_aio": "16", 2023-12-06T18:27:17.104 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_multipart_min_part_size": "5242880", 2023-12-06T18:27:17.104 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_multipart_part_upload_limit": "10000", 2023-12-06T18:27:17.104 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_nfs_fhcache_partitions": "3", 2023-12-06T18:27:17.104 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_nfs_fhcache_size": "2017", 2023-12-06T18:27:17.104 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_nfs_frontends": "rgw-nfs", 2023-12-06T18:27:17.104 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2023-12-06T18:27:17.105 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_nfs_lru_lanes": "5", 2023-12-06T18:27:17.105 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_nfs_max_gc": "300", 2023-12-06T18:27:17.105 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_nfs_namespace_expire_secs": "300", 2023-12-06T18:27:17.105 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_nfs_run_gc_threads": "false", 2023-12-06T18:27:17.105 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_nfs_run_lc_threads": "false", 2023-12-06T18:27:17.105 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_nfs_run_quota_threads": "false", 2023-12-06T18:27:17.105 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_nfs_run_sync_thread": "false", 2023-12-06T18:27:17.105 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_nfs_s3_fast_attrs": "false", 2023-12-06T18:27:17.105 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_nfs_write_completion_interval_s": "10", 2023-12-06T18:27:17.105 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_num_async_rados_threads": "32", 2023-12-06T18:27:17.105 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_num_control_oids": "8", 2023-12-06T18:27:17.105 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_numa_node": "-1", 2023-12-06T18:27:17.105 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_obj_stripe_size": "4194304", 2023-12-06T18:27:17.105 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_obj_tombstone_cache_size": "1000", 2023-12-06T18:27:17.105 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_objexp_chunk_size": "100", 2023-12-06T18:27:17.105 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_objexp_gc_interval": "600", 2023-12-06T18:27:17.106 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_objexp_hints_num_shards": "127", 2023-12-06T18:27:17.106 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_olh_pending_timeout_sec": "3600", 2023-12-06T18:27:17.106 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_op_thread_suicide_timeout": "0", 2023-12-06T18:27:17.106 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_op_thread_timeout": "600", 2023-12-06T18:27:17.106 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_op_tracing": "false", 2023-12-06T18:27:17.106 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_opa_token": "", 2023-12-06T18:27:17.106 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_opa_url": "", 2023-12-06T18:27:17.106 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_opa_verify_ssl": "true", 2023-12-06T18:27:17.106 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_ops_log_data_backlog": "5242880", 2023-12-06T18:27:17.106 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2023-12-06T18:27:17.106 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_ops_log_rados": "false", 2023-12-06T18:27:17.106 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_ops_log_socket_path": "", 2023-12-06T18:27:17.106 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_override_bucket_index_max_shards": "0", 2023-12-06T18:27:17.106 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2023-12-06T18:27:17.107 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2023-12-06T18:27:17.107 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_period_push_interval": "2.000000", 2023-12-06T18:27:17.107 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_period_push_interval_max": "30.000000", 2023-12-06T18:27:17.107 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_period_root_pool": ".rgw.root", 2023-12-06T18:27:17.107 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_policy_reject_invalid_principals": "true", 2023-12-06T18:27:17.107 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2023-12-06T18:27:17.109 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_posix_cache_lanes": "3", 2023-12-06T18:27:17.109 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_posix_cache_lmdb_count": "3", 2023-12-06T18:27:17.109 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_posix_cache_max_buckets": "100", 2023-12-06T18:27:17.109 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_posix_cache_partitions": "3", 2023-12-06T18:27:17.109 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2023-12-06T18:27:17.110 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_print_continue": "true", 2023-12-06T18:27:17.110 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_print_prohibited_content_length": "false", 2023-12-06T18:27:17.110 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_put_obj_max_window_size": "67108864", 2023-12-06T18:27:17.110 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_put_obj_min_window_size": "16777216", 2023-12-06T18:27:17.110 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2023-12-06T18:27:17.110 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_rados_pool_recovery_priority": "5", 2023-12-06T18:27:17.110 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_rados_tracing": "false", 2023-12-06T18:27:17.110 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_realm": "", 2023-12-06T18:27:17.110 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_realm_id": "", 2023-12-06T18:27:17.110 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_realm_root_pool": ".rgw.root", 2023-12-06T18:27:17.110 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_region": "", 2023-12-06T18:27:17.110 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_region_root_pool": ".rgw.root", 2023-12-06T18:27:17.110 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_relaxed_region_enforcement": "false", 2023-12-06T18:27:17.110 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_relaxed_s3_bucket_names": "false", 2023-12-06T18:27:17.110 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2023-12-06T18:27:17.111 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_request_uri": "", 2023-12-06T18:27:17.111 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_reshard_batch_size": "64", 2023-12-06T18:27:17.111 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_reshard_bucket_lock_duration": "360", 2023-12-06T18:27:17.111 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_reshard_max_aio": "128", 2023-12-06T18:27:17.111 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_reshard_num_logs": "16", 2023-12-06T18:27:17.111 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_reshard_thread_interval": "600", 2023-12-06T18:27:17.111 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_resolve_cname": "false", 2023-12-06T18:27:17.111 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_rest_getusage_op_compat": "false", 2023-12-06T18:27:17.111 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_run_sync_thread": "true", 2023-12-06T18:27:17.111 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_s3_auth_order": "sts, external, local", 2023-12-06T18:27:17.111 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_s3_auth_use_keystone": "false", 2023-12-06T18:27:17.111 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_s3_auth_use_ldap": "false", 2023-12-06T18:27:17.111 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_s3_auth_use_rados": "true", 2023-12-06T18:27:17.111 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_s3_auth_use_sts": "false", 2023-12-06T18:27:17.111 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_s3_client_max_sig_ver": "-1", 2023-12-06T18:27:17.112 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_s3_success_create_obj_status": "0", 2023-12-06T18:27:17.112 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_safe_max_objects_per_shard": "102400", 2023-12-06T18:27:17.112 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_scheduler_type": "throttler", 2023-12-06T18:27:17.112 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_script_uri": "", 2023-12-06T18:27:17.112 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_service_provider_name": "", 2023-12-06T18:27:17.112 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_shard_warning_threshold": "90.000000", 2023-12-06T18:27:17.112 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sts_client_id": "", 2023-12-06T18:27:17.112 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sts_client_secret": "", 2023-12-06T18:27:17.112 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sts_entry": "sts", 2023-12-06T18:27:17.112 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sts_key": "sts", 2023-12-06T18:27:17.112 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sts_max_session_duration": "43200", 2023-12-06T18:27:17.112 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sts_min_session_duration": "900", 2023-12-06T18:27:17.112 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sts_token_introspection_url": "", 2023-12-06T18:27:17.112 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_swift_account_in_url": "false", 2023-12-06T18:27:17.112 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_swift_auth_entry": "auth", 2023-12-06T18:27:17.113 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_swift_auth_url": "", 2023-12-06T18:27:17.113 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_swift_custom_header": "", 2023-12-06T18:27:17.113 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_swift_enforce_content_length": "false", 2023-12-06T18:27:17.113 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_swift_need_stats": "true", 2023-12-06T18:27:17.113 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_swift_tenant_name": "", 2023-12-06T18:27:17.113 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_swift_token_expiration": "86400", 2023-12-06T18:27:17.113 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_swift_url": "", 2023-12-06T18:27:17.113 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_swift_url_prefix": "swift", 2023-12-06T18:27:17.113 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_swift_versioning_enabled": "false", 2023-12-06T18:27:17.113 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2023-12-06T18:27:17.113 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2023-12-06T18:27:17.113 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sync_lease_period": "120", 2023-12-06T18:27:17.113 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2023-12-06T18:27:17.113 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sync_log_trim_interval": "1200", 2023-12-06T18:27:17.113 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sync_log_trim_max_buckets": "16", 2023-12-06T18:27:17.114 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2023-12-06T18:27:17.114 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2023-12-06T18:27:17.114 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sync_obj_etag_verify": "false", 2023-12-06T18:27:17.114 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sync_trace_history_size": "4096", 2023-12-06T18:27:17.114 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sync_trace_per_node_log_size": "32", 2023-12-06T18:27:17.114 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2023-12-06T18:27:17.114 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_thread_pool_size": "512", 2023-12-06T18:27:17.114 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_topic_persistency_max_retries": "0", 2023-12-06T18:27:17.114 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_topic_persistency_sleep_duration": "0", 2023-12-06T18:27:17.114 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_topic_persistency_time_to_live": "0", 2023-12-06T18:27:17.114 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_torrent_comment": "", 2023-12-06T18:27:17.114 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_torrent_createby": "", 2023-12-06T18:27:17.114 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_torrent_encoding": "", 2023-12-06T18:27:17.115 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_torrent_flag": "false", 2023-12-06T18:27:17.115 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_torrent_max_size": "5368709120", 2023-12-06T18:27:17.115 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_torrent_origin": "", 2023-12-06T18:27:17.115 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_torrent_sha_unit": "524288", 2023-12-06T18:27:17.115 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_torrent_tracker": "", 2023-12-06T18:27:17.115 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_trust_forwarded_https": "false", 2023-12-06T18:27:17.115 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_usage_log_flush_threshold": "1024", 2023-12-06T18:27:17.115 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_usage_log_tick_interval": "30", 2023-12-06T18:27:17.115 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_usage_max_shards": "32", 2023-12-06T18:27:17.115 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_usage_max_user_shards": "1", 2023-12-06T18:27:17.115 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_use_opa_authz": "false", 2023-12-06T18:27:17.115 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_user_counters_cache": "false", 2023-12-06T18:27:17.115 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_user_counters_cache_size": "10000", 2023-12-06T18:27:17.115 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_user_default_quota_max_objects": "-1", 2023-12-06T18:27:17.115 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_user_default_quota_max_size": "-1", 2023-12-06T18:27:17.116 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_user_max_buckets": "1000", 2023-12-06T18:27:17.116 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_user_policies_max_num": "100", 2023-12-06T18:27:17.116 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2023-12-06T18:27:17.116 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_user_quota_sync_idle_users": "false", 2023-12-06T18:27:17.116 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_user_quota_sync_interval": "86400", 2023-12-06T18:27:17.116 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_user_quota_sync_wait_time": "86400", 2023-12-06T18:27:17.116 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_user_unique_email": "true", 2023-12-06T18:27:17.116 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_verify_ssl": "true", 2023-12-06T18:27:17.116 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_website_routing_rules_max_num": "50", 2023-12-06T18:27:17.116 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_zone": "", 2023-12-06T18:27:17.116 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_zone_id": "", 2023-12-06T18:27:17.116 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_zone_root_pool": ".rgw.root", 2023-12-06T18:27:17.116 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_zonegroup": "", 2023-12-06T18:27:17.116 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_zonegroup_id": "", 2023-12-06T18:27:17.116 INFO:teuthology.orchestra.run.smithi061.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2023-12-06T18:27:17.117 INFO:teuthology.orchestra.run.smithi061.stdout: "rgwlc_auto_session_clear": "true", 2023-12-06T18:27:17.117 INFO:teuthology.orchestra.run.smithi061.stdout: "rgwlc_skip_bucket_step": "false", 2023-12-06T18:27:17.117 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_block_size": "4096", 2023-12-06T18:27:17.117 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_bloom_bits_per_key": "20", 2023-12-06T18:27:17.117 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2023-12-06T18:27:17.117 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2023-12-06T18:27:17.117 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_cache_row_ratio": "0.000000", 2023-12-06T18:27:17.117 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_cache_shard_bits": "4", 2023-12-06T18:27:17.117 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_cache_size": "536870912", 2023-12-06T18:27:17.117 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_cache_type": "binned_lru", 2023-12-06T18:27:17.117 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_cf_compact_on_deletion": "true", 2023-12-06T18:27:17.117 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2023-12-06T18:27:17.117 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2023-12-06T18:27:17.117 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_collect_compaction_stats": "false", 2023-12-06T18:27:17.118 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_collect_extended_stats": "false", 2023-12-06T18:27:17.118 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_collect_memory_stats": "false", 2023-12-06T18:27:17.118 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_delete_range_threshold": "1048576", 2023-12-06T18:27:17.118 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_index_type": "binary_search", 2023-12-06T18:27:17.118 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_log_to_ceph_log": "true", 2023-12-06T18:27:17.118 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_metadata_block_size": "4096", 2023-12-06T18:27:17.118 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_partition_filters": "false", 2023-12-06T18:27:17.118 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_perf": "false", 2023-12-06T18:27:17.118 INFO:teuthology.orchestra.run.smithi061.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2023-12-06T18:27:17.118 INFO:teuthology.orchestra.run.smithi061.stdout: "rotating_keys_bootstrap_timeout": "30", 2023-12-06T18:27:17.118 INFO:teuthology.orchestra.run.smithi061.stdout: "rotating_keys_renewal_timeout": "10", 2023-12-06T18:27:17.118 INFO:teuthology.orchestra.run.smithi061.stdout: "run_dir": "/var/run/ceph", 2023-12-06T18:27:17.118 INFO:teuthology.orchestra.run.smithi061.stdout: "seastore_block_create": "true", 2023-12-06T18:27:17.118 INFO:teuthology.orchestra.run.smithi061.stdout: "seastore_cache_lru_size": "67108864", 2023-12-06T18:27:17.118 INFO:teuthology.orchestra.run.smithi061.stdout: "seastore_cbjournal_size": "5368709120", 2023-12-06T18:27:17.119 INFO:teuthology.orchestra.run.smithi061.stdout: "seastore_data_delta_based_overwrite": "0", 2023-12-06T18:27:17.119 INFO:teuthology.orchestra.run.smithi061.stdout: "seastore_default_max_object_size": "16777216", 2023-12-06T18:27:17.119 INFO:teuthology.orchestra.run.smithi061.stdout: "seastore_default_object_metadata_reservation": "16777216", 2023-12-06T18:27:17.119 INFO:teuthology.orchestra.run.smithi061.stdout: "seastore_device_size": "53687091200", 2023-12-06T18:27:17.119 INFO:teuthology.orchestra.run.smithi061.stdout: "seastore_journal_batch_capacity": "16", 2023-12-06T18:27:17.119 INFO:teuthology.orchestra.run.smithi061.stdout: "seastore_journal_batch_flush_size": "16777216", 2023-12-06T18:27:17.119 INFO:teuthology.orchestra.run.smithi061.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2023-12-06T18:27:17.119 INFO:teuthology.orchestra.run.smithi061.stdout: "seastore_journal_iodepth_limit": "5", 2023-12-06T18:27:17.119 INFO:teuthology.orchestra.run.smithi061.stdout: "seastore_main_device_type": "SSD", 2023-12-06T18:27:17.119 INFO:teuthology.orchestra.run.smithi061.stdout: "seastore_max_concurrent_transactions": "8", 2023-12-06T18:27:17.119 INFO:teuthology.orchestra.run.smithi061.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2023-12-06T18:27:17.119 INFO:teuthology.orchestra.run.smithi061.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2023-12-06T18:27:17.119 INFO:teuthology.orchestra.run.smithi061.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2023-12-06T18:27:17.119 INFO:teuthology.orchestra.run.smithi061.stdout: "seastore_obj_data_write_amplification": "1.250000", 2023-12-06T18:27:17.119 INFO:teuthology.orchestra.run.smithi061.stdout: "seastore_segment_size": "67108864", 2023-12-06T18:27:17.120 INFO:teuthology.orchestra.run.smithi061.stdout: "set_keepcaps": "false", 2023-12-06T18:27:17.120 INFO:teuthology.orchestra.run.smithi061.stdout: "setgroup": "", 2023-12-06T18:27:17.120 INFO:teuthology.orchestra.run.smithi061.stdout: "setuser": "", 2023-12-06T18:27:17.120 INFO:teuthology.orchestra.run.smithi061.stdout: "setuser_match_path": "", 2023-12-06T18:27:17.120 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_misplaced_ratio": "0.050000", 2023-12-06T18:27:17.120 INFO:teuthology.orchestra.run.smithi061.stdout: "thp": "false", 2023-12-06T18:27:17.120 INFO:teuthology.orchestra.run.smithi061.stdout: "threadpool_default_timeout": "60", 2023-12-06T18:27:17.120 INFO:teuthology.orchestra.run.smithi061.stdout: "threadpool_empty_queue_max_wait": "2", 2023-12-06T18:27:17.120 INFO:teuthology.orchestra.run.smithi061.stdout: "throttler_perf_counter": "true" 2023-12-06T18:27:17.120 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:17.122 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2023-12-06T18:27:17.386 INFO:teuthology.orchestra.run.smithi061.stdout:mon.a: {} 2023-12-06T18:27:17.387 INFO:teuthology.orchestra.run.smithi061.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2023-12-06T18:27:17.481 INFO:teuthology.orchestra.run.smithi061.stdout:mon.b: {} 2023-12-06T18:27:17.482 INFO:teuthology.orchestra.run.smithi061.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2023-12-06T18:27:17.589 INFO:teuthology.orchestra.run.smithi061.stdout:mon.c: {} 2023-12-06T18:27:17.590 INFO:teuthology.orchestra.run.smithi061.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2023-12-06T18:27:17.601 DEBUG:teuthology.orchestra.run.smithi061:> type ceph-objectstore-tool 2023-12-06T18:27:17.615 INFO:teuthology.orchestra.run.smithi061.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2023-12-06T18:27:17.615 DEBUG:teuthology.orchestra.run.smithi152:> type ceph-objectstore-tool 2023-12-06T18:27:17.629 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2023-12-06T18:27:17.630 INFO:teuthology.run_tasks:Running task mon_recovery... 2023-12-06T18:27:17.640 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-06T18:27:17.640 INFO:tasks.thrashosds.thrasher:starting do_thrash 2023-12-06T18:27:17.640 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:27:17.640 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:27:17.640 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:27:17.641 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2023-12-06T18:27:17.641 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2023-12-06T18:27:17.641 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2023-12-06T18:27:17.641 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:27:17.642 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2023-12-06T18:27:17.642 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:27:17.642 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2023-12-06T18:27:17.642 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-12-06T18:27:17.662 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:17.656+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:17.752 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:17.744+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:17.840 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:27:17.840 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:17.850 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:17.845+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:17.852 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:27:17.950 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:17.945+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:18.030 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:27:18.030 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:27:18.046 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:18.043+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:18.078 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:18.079 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:18.088 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:27:18.180 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:18.174+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:18.200 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:27:18.200 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:27:18.249 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:18.245+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:18.292 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:18.292 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:18.302 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:27:18.382 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:27:18.382 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:27:18.388 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:18.379+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:18.450 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:18.447+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:18.532 INFO:tasks.ceph.mon.a.smithi061.stderr:2023-12-06T18:27:18.518+0000 ff5f640 -1 mon.a@0(leader).osd e24 definitely_dead 0 2023-12-06T18:27:18.552 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:18.548+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:18.590 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:27:18.590 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:27:18.591 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:18.591 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:18.600 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:27:18.660 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:27:18.661 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":24,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","created":"2023-12-06T18:26:07.122862+0000","modified":"2023-12-06T18:27:06.066062+0000","last_up_change":"2023-12-06T18:26:56.529403+0000","last_in_change":"2023-12-06T18:26:12.654320+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-06T18:26:52.648377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-12-06T18:27:01.712581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5e6e17b9-e72e-4e34-a531-0830d659bfe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6805","nonce":281956997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":281956997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":281956997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6807","nonce":281956997}]},"public_addr":"172.21.15.61:6805/281956997","cluster_addr":"172.21.15.61:6806/281956997","heartbeat_back_addr":"172.21.15.61:6808/281956997","heartbeat_front_addr":"172.21.15.61:6807/281956997","state":["exists","up"]},{"osd":1,"uuid":"b78c49b0-360d-4351-b44e-65ccaaeb768f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6809","nonce":3306625622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":3306625622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":3306625622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6811","nonce":3306625622}]},"public_addr":"172.21.15.61:6809/3306625622","cluster_addr":"172.21.15.61:6810/3306625622","heartbeat_back_addr":"172.21.15.61:6812/3306625622","heartbeat_front_addr":"172.21.15.61:6811/3306625622","state":["exists","up"]},{"osd":2,"uuid":"a2af3151-a4da-4c34-8fb6-78abf035785b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6801","nonce":3589270120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3589270120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3589270120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6803","nonce":3589270120}]},"public_addr":"172.21.15.61:6801/3589270120","cluster_addr":"172.21.15.61:6802/3589270120","heartbeat_back_addr":"172.21.15.61:6804/3589270120","heartbeat_front_addr":"172.21.15.61:6803/3589270120","state":["exists","up"]},{"osd":3,"uuid":"1c754ed6-c535-454f-b35b-00f62cd6ce5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":2849920842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":2849920842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":2849920842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6815","nonce":2849920842}]},"public_addr":"172.21.15.61:6813/2849920842","cluster_addr":"172.21.15.61:6814/2849920842","heartbeat_back_addr":"172.21.15.61:6816/2849920842","heartbeat_front_addr":"172.21.15.61:6815/2849920842","state":["exists","up"]},{"osd":4,"uuid":"766572af-b9ca-4f38-927d-36f26101ba51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2778770069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6801","nonce":2778770069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6803","nonce":2778770069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2778770069}]},"public_addr":"172.21.15.152:6800/2778770069","cluster_addr":"172.21.15.152:6801/2778770069","heartbeat_back_addr":"172.21.15.152:6803/2778770069","heartbeat_front_addr":"172.21.15.152:6802/2778770069","state":["exists","up"]},{"osd":5,"uuid":"7a566b60-1543-41d7-894c-e2fbb63e48dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6809","nonce":263671335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":263671335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":263671335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":263671335}]},"public_addr":"172.21.15.152:6809/263671335","cluster_addr":"172.21.15.152:6810/263671335","heartbeat_back_addr":"172.21.15.152:6814/263671335","heartbeat_front_addr":"172.21.15.152:6812/263671335","state":["exists","up"]},{"osd":6,"uuid":"28cc2358-4ac4-40b2-91a4-20edfe625a73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":4245639079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":4245639079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":4245639079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6813","nonce":4245639079}]},"public_addr":"172.21.15.152:6808/4245639079","cluster_addr":"172.21.15.152:6811/4245639079","heartbeat_back_addr":"172.21.15.152:6815/4245639079","heartbeat_front_addr":"172.21.15.152:6813/4245639079","state":["exists","up"]},{"osd":7,"uuid":"8f79279b-d2d0-438e-b3c2-88df1241ff30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":3940055127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6805","nonce":3940055127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6807","nonce":3940055127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":3940055127}]},"public_addr":"172.21.15.152:6804/3940055127","cluster_addr":"172.21.15.152:6805/3940055127","heartbeat_back_addr":"172.21.15.152:6807/3940055127","heartbeat_front_addr":"172.21.15.152:6806/3940055127","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:52.171904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:52.428840+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:51.681954+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:53.687417+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:50.158682+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:50.443807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:49.750276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:49.770987+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-06T18:27:18.661 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:18.657+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:18.675 INFO:tasks.mon_recovery.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-06T18:26:52.648377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-12-06T18:27:01.712581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-06T18:27:18.675 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-06T18:27:18.718 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:27:18.718 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:27:18.758 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:18.753+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:18.796 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:18.796 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:18.806 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:27:18.867 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:27:18.867 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:27:18.868 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:18.860+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:18.955 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:18.951+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:18.986 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:18.986 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:18.995 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:27:19.004 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:27:19.004 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:27:19.056 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:19.052+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:19.141 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:27:19.141 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:27:19.154 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:19.155 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:19.162 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:19.156+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:19.163 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:27:19.258 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:19.254+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:19.359 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:19.355+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:19.363 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:19.364 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:19.373 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:27:19.385 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 1 2023-12-06T18:27:19.399 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-12-06T18:27:19.460 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:19.456+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:19.526 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:19.526 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:19.536 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:27:19.575 INFO:teuthology.orchestra.run.smithi061.stderr:marked out osd.5. 2023-12-06T18:27:19.576 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:19.573+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:19.619 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is set 2023-12-06T18:27:19.661 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:19.657+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:19.722 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:19.722 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:19.734 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:27:19.838 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:19.835+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:19.876 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:19.876 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:19.886 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:27:20.003 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:19.999+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:20.007 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:20.003+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:20.064 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:20.061+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:20.128 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:20.128 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:20.139 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:27:20.164 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:20.161+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:20.262 INFO:teuthology.orchestra.run.smithi061.stdout:pg_num: 8 2023-12-06T18:27:20.265 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:20.261+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:20.275 INFO:tasks.mon_recovery:mon ids = ['a', 'c', 'b'] 2023-12-06T18:27:20.276 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2023-12-06T18:27:20.276 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:27:20.276 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:20.276 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:20.286 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:27:20.367 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:20.363+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:20.416 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:20.416 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:20.426 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:27:20.467 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:20.464+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:20.568 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:20.565+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:20.596 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:20.597 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:20.605 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:27:20.669 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:20.666+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:20.771 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:20.767+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:20.786 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:20.786 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:20.795 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:27:20.880 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:20.876+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:20.972 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:20.969+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:20.977 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:20.977 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:20.986 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:27:21.071 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":74,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:27:21.073 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:21.070+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:21.090 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2023-12-06T18:27:21.090 INFO:tasks.mon_recovery:verifying all monitors are in the quorum 2023-12-06T18:27:21.090 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2023-12-06T18:27:21.156 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:27:21.177 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:21.172+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:21.208 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:21.208 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:21.217 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:27:21.301 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:21.298+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:21.366 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:21.366 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "a", 2023-12-06T18:27:21.366 INFO:teuthology.orchestra.run.smithi061.stdout: "rank": 0, 2023-12-06T18:27:21.366 INFO:teuthology.orchestra.run.smithi061.stdout: "state": "leader", 2023-12-06T18:27:21.366 INFO:teuthology.orchestra.run.smithi061.stdout: "election_epoch": 10, 2023-12-06T18:27:21.366 INFO:teuthology.orchestra.run.smithi061.stdout: "quorum": [ 2023-12-06T18:27:21.366 INFO:teuthology.orchestra.run.smithi061.stdout: 0, 2023-12-06T18:27:21.366 INFO:teuthology.orchestra.run.smithi061.stdout: 1, 2023-12-06T18:27:21.366 INFO:teuthology.orchestra.run.smithi061.stdout: 2 2023-12-06T18:27:21.366 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:27:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: "quorum_age": 74, 2023-12-06T18:27:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: "features": { 2023-12-06T18:27:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: "required_con": "2449958758054445060", 2023-12-06T18:27:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: "required_mon": [ 2023-12-06T18:27:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: "kraken", 2023-12-06T18:27:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: "luminous", 2023-12-06T18:27:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: "mimic", 2023-12-06T18:27:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap-prune", 2023-12-06T18:27:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: "nautilus", 2023-12-06T18:27:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: "octopus", 2023-12-06T18:27:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: "pacific", 2023-12-06T18:27:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: "elector-pinging", 2023-12-06T18:27:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: "quincy", 2023-12-06T18:27:21.367 INFO:teuthology.orchestra.run.smithi061.stdout: "reef" 2023-12-06T18:27:21.368 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:27:21.368 INFO:teuthology.orchestra.run.smithi061.stdout: "quorum_con": "4540138322906710015", 2023-12-06T18:27:21.368 INFO:teuthology.orchestra.run.smithi061.stdout: "quorum_mon": [ 2023-12-06T18:27:21.368 INFO:teuthology.orchestra.run.smithi061.stdout: "kraken", 2023-12-06T18:27:21.368 INFO:teuthology.orchestra.run.smithi061.stdout: "luminous", 2023-12-06T18:27:21.368 INFO:teuthology.orchestra.run.smithi061.stdout: "mimic", 2023-12-06T18:27:21.368 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap-prune", 2023-12-06T18:27:21.368 INFO:teuthology.orchestra.run.smithi061.stdout: "nautilus", 2023-12-06T18:27:21.368 INFO:teuthology.orchestra.run.smithi061.stdout: "octopus", 2023-12-06T18:27:21.368 INFO:teuthology.orchestra.run.smithi061.stdout: "pacific", 2023-12-06T18:27:21.368 INFO:teuthology.orchestra.run.smithi061.stdout: "elector-pinging", 2023-12-06T18:27:21.368 INFO:teuthology.orchestra.run.smithi061.stdout: "quincy", 2023-12-06T18:27:21.368 INFO:teuthology.orchestra.run.smithi061.stdout: "reef" 2023-12-06T18:27:21.368 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:21.368 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.369 INFO:teuthology.orchestra.run.smithi061.stdout: "outside_quorum": [], 2023-12-06T18:27:21.369 INFO:teuthology.orchestra.run.smithi061.stdout: "extra_probe_peers": [], 2023-12-06T18:27:21.369 INFO:teuthology.orchestra.run.smithi061.stdout: "sync_provider": [], 2023-12-06T18:27:21.369 INFO:teuthology.orchestra.run.smithi061.stdout: "monmap": { 2023-12-06T18:27:21.369 INFO:teuthology.orchestra.run.smithi061.stdout: "epoch": 1, 2023-12-06T18:27:21.369 INFO:teuthology.orchestra.run.smithi061.stdout: "fsid": "497e3283-cc16-40ff-a72c-a7487474fb4b", 2023-12-06T18:27:21.369 INFO:teuthology.orchestra.run.smithi061.stdout: "modified": "2023-12-06T18:25:15.366722Z", 2023-12-06T18:27:21.369 INFO:teuthology.orchestra.run.smithi061.stdout: "created": "2023-12-06T18:25:15.366722Z", 2023-12-06T18:27:21.369 INFO:teuthology.orchestra.run.smithi061.stdout: "min_mon_release": 18, 2023-12-06T18:27:21.369 INFO:teuthology.orchestra.run.smithi061.stdout: "min_mon_release_name": "reef", 2023-12-06T18:27:21.369 INFO:teuthology.orchestra.run.smithi061.stdout: "election_strategy": 3, 2023-12-06T18:27:21.369 INFO:teuthology.orchestra.run.smithi061.stdout: "disallowed_leaders: ": "", 2023-12-06T18:27:21.369 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode": false, 2023-12-06T18:27:21.369 INFO:teuthology.orchestra.run.smithi061.stdout: "tiebreaker_mon": "", 2023-12-06T18:27:21.369 INFO:teuthology.orchestra.run.smithi061.stdout: "removed_ranks: ": "", 2023-12-06T18:27:21.370 INFO:teuthology.orchestra.run.smithi061.stdout: "features": { 2023-12-06T18:27:21.370 INFO:teuthology.orchestra.run.smithi061.stdout: "persistent": [ 2023-12-06T18:27:21.370 INFO:teuthology.orchestra.run.smithi061.stdout: "kraken", 2023-12-06T18:27:21.370 INFO:teuthology.orchestra.run.smithi061.stdout: "luminous", 2023-12-06T18:27:21.370 INFO:teuthology.orchestra.run.smithi061.stdout: "mimic", 2023-12-06T18:27:21.370 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap-prune", 2023-12-06T18:27:21.370 INFO:teuthology.orchestra.run.smithi061.stdout: "nautilus", 2023-12-06T18:27:21.370 INFO:teuthology.orchestra.run.smithi061.stdout: "octopus", 2023-12-06T18:27:21.370 INFO:teuthology.orchestra.run.smithi061.stdout: "pacific", 2023-12-06T18:27:21.370 INFO:teuthology.orchestra.run.smithi061.stdout: "elector-pinging", 2023-12-06T18:27:21.370 INFO:teuthology.orchestra.run.smithi061.stdout: "quincy", 2023-12-06T18:27:21.370 INFO:teuthology.orchestra.run.smithi061.stdout: "reef" 2023-12-06T18:27:21.370 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:27:21.370 INFO:teuthology.orchestra.run.smithi061.stdout: "optional": [] 2023-12-06T18:27:21.371 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.371 INFO:teuthology.orchestra.run.smithi061.stdout: "mons": [ 2023-12-06T18:27:21.371 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.371 INFO:teuthology.orchestra.run.smithi061.stdout: "rank": 0, 2023-12-06T18:27:21.371 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "a", 2023-12-06T18:27:21.371 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:27:21.371 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:27:21.371 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.371 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:27:21.371 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:3300", 2023-12-06T18:27:21.371 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 0 2023-12-06T18:27:21.371 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.371 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.371 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v1", 2023-12-06T18:27:21.372 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6789", 2023-12-06T18:27:21.372 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 0 2023-12-06T18:27:21.372 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.372 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:21.372 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.372 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6789/0", 2023-12-06T18:27:21.372 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6789/0", 2023-12-06T18:27:21.372 INFO:teuthology.orchestra.run.smithi061.stdout: "priority": 0, 2023-12-06T18:27:21.372 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 0, 2023-12-06T18:27:21.372 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_location": "{}" 2023-12-06T18:27:21.372 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.372 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.372 INFO:teuthology.orchestra.run.smithi061.stdout: "rank": 1, 2023-12-06T18:27:21.372 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "b", 2023-12-06T18:27:21.372 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:27:21.372 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:27:21.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.373 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:27:21.373 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:3300", 2023-12-06T18:27:21.373 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 0 2023-12-06T18:27:21.373 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.373 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v1", 2023-12-06T18:27:21.373 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6789", 2023-12-06T18:27:21.373 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 0 2023-12-06T18:27:21.373 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.373 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:21.373 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.373 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6789/0", 2023-12-06T18:27:21.373 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6789/0", 2023-12-06T18:27:21.374 INFO:teuthology.orchestra.run.smithi061.stdout: "priority": 0, 2023-12-06T18:27:21.374 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 0, 2023-12-06T18:27:21.374 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_location": "{}" 2023-12-06T18:27:21.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.374 INFO:teuthology.orchestra.run.smithi061.stdout: "rank": 2, 2023-12-06T18:27:21.374 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "c", 2023-12-06T18:27:21.374 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:27:21.374 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:27:21.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.374 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:27:21.374 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:3301", 2023-12-06T18:27:21.374 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 0 2023-12-06T18:27:21.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v1", 2023-12-06T18:27:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6790", 2023-12-06T18:27:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 0 2023-12-06T18:27:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6790/0", 2023-12-06T18:27:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6790/0", 2023-12-06T18:27:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "priority": 0, 2023-12-06T18:27:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 0, 2023-12-06T18:27:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_location": "{}" 2023-12-06T18:27:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:21.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.376 INFO:teuthology.orchestra.run.smithi061.stdout: "feature_map": { 2023-12-06T18:27:21.376 INFO:teuthology.orchestra.run.smithi061.stdout: "mon": [ 2023-12-06T18:27:21.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.376 INFO:teuthology.orchestra.run.smithi061.stdout: "features": "0x3f01cfbffffdffff", 2023-12-06T18:27:21.376 INFO:teuthology.orchestra.run.smithi061.stdout: "release": "luminous", 2023-12-06T18:27:21.376 INFO:teuthology.orchestra.run.smithi061.stdout: "num": 1 2023-12-06T18:27:21.376 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.376 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:27:21.376 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": [ 2023-12-06T18:27:21.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.376 INFO:teuthology.orchestra.run.smithi061.stdout: "features": "0x3f01cfbffffdffff", 2023-12-06T18:27:21.376 INFO:teuthology.orchestra.run.smithi061.stdout: "release": "luminous", 2023-12-06T18:27:21.376 INFO:teuthology.orchestra.run.smithi061.stdout: "num": 1 2023-12-06T18:27:21.376 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.376 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:27:21.377 INFO:teuthology.orchestra.run.smithi061.stdout: "client": [ 2023-12-06T18:27:21.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.377 INFO:teuthology.orchestra.run.smithi061.stdout: "features": "0x3f01cfbffffdffff", 2023-12-06T18:27:21.377 INFO:teuthology.orchestra.run.smithi061.stdout: "release": "luminous", 2023-12-06T18:27:21.377 INFO:teuthology.orchestra.run.smithi061.stdout: "num": 3 2023-12-06T18:27:21.377 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.377 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:27:21.377 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr": [ 2023-12-06T18:27:21.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.377 INFO:teuthology.orchestra.run.smithi061.stdout: "features": "0x3f01cfbffffdffff", 2023-12-06T18:27:21.377 INFO:teuthology.orchestra.run.smithi061.stdout: "release": "luminous", 2023-12-06T18:27:21.377 INFO:teuthology.orchestra.run.smithi061.stdout: "num": 1 2023-12-06T18:27:21.377 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.377 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:21.377 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.378 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode": false 2023-12-06T18:27:21.378 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:21.379 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2023-12-06T18:27:21.380 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:21.375+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:21.419 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:21.420 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:21.429 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:27:21.441 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:27:21.442 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:27:21.512 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:21.509+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:21.559 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:21.559 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "c", 2023-12-06T18:27:21.559 INFO:teuthology.orchestra.run.smithi061.stdout: "rank": 2, 2023-12-06T18:27:21.559 INFO:teuthology.orchestra.run.smithi061.stdout: "state": "peon", 2023-12-06T18:27:21.559 INFO:teuthology.orchestra.run.smithi061.stdout: "election_epoch": 10, 2023-12-06T18:27:21.559 INFO:teuthology.orchestra.run.smithi061.stdout: "quorum": [ 2023-12-06T18:27:21.559 INFO:teuthology.orchestra.run.smithi061.stdout: 0, 2023-12-06T18:27:21.559 INFO:teuthology.orchestra.run.smithi061.stdout: 1, 2023-12-06T18:27:21.559 INFO:teuthology.orchestra.run.smithi061.stdout: 2 2023-12-06T18:27:21.559 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:27:21.560 INFO:teuthology.orchestra.run.smithi061.stdout: "quorum_age": 74, 2023-12-06T18:27:21.560 INFO:teuthology.orchestra.run.smithi061.stdout: "features": { 2023-12-06T18:27:21.560 INFO:teuthology.orchestra.run.smithi061.stdout: "required_con": "2449958758054445060", 2023-12-06T18:27:21.560 INFO:teuthology.orchestra.run.smithi061.stdout: "required_mon": [ 2023-12-06T18:27:21.560 INFO:teuthology.orchestra.run.smithi061.stdout: "kraken", 2023-12-06T18:27:21.560 INFO:teuthology.orchestra.run.smithi061.stdout: "luminous", 2023-12-06T18:27:21.560 INFO:teuthology.orchestra.run.smithi061.stdout: "mimic", 2023-12-06T18:27:21.560 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap-prune", 2023-12-06T18:27:21.560 INFO:teuthology.orchestra.run.smithi061.stdout: "nautilus", 2023-12-06T18:27:21.560 INFO:teuthology.orchestra.run.smithi061.stdout: "octopus", 2023-12-06T18:27:21.560 INFO:teuthology.orchestra.run.smithi061.stdout: "pacific", 2023-12-06T18:27:21.560 INFO:teuthology.orchestra.run.smithi061.stdout: "elector-pinging", 2023-12-06T18:27:21.560 INFO:teuthology.orchestra.run.smithi061.stdout: "quincy", 2023-12-06T18:27:21.560 INFO:teuthology.orchestra.run.smithi061.stdout: "reef" 2023-12-06T18:27:21.560 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:27:21.561 INFO:teuthology.orchestra.run.smithi061.stdout: "quorum_con": "4540138322906710015", 2023-12-06T18:27:21.561 INFO:teuthology.orchestra.run.smithi061.stdout: "quorum_mon": [ 2023-12-06T18:27:21.561 INFO:teuthology.orchestra.run.smithi061.stdout: "kraken", 2023-12-06T18:27:21.561 INFO:teuthology.orchestra.run.smithi061.stdout: "luminous", 2023-12-06T18:27:21.561 INFO:teuthology.orchestra.run.smithi061.stdout: "mimic", 2023-12-06T18:27:21.561 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap-prune", 2023-12-06T18:27:21.561 INFO:teuthology.orchestra.run.smithi061.stdout: "nautilus", 2023-12-06T18:27:21.561 INFO:teuthology.orchestra.run.smithi061.stdout: "octopus", 2023-12-06T18:27:21.561 INFO:teuthology.orchestra.run.smithi061.stdout: "pacific", 2023-12-06T18:27:21.561 INFO:teuthology.orchestra.run.smithi061.stdout: "elector-pinging", 2023-12-06T18:27:21.561 INFO:teuthology.orchestra.run.smithi061.stdout: "quincy", 2023-12-06T18:27:21.562 INFO:teuthology.orchestra.run.smithi061.stdout: "reef" 2023-12-06T18:27:21.562 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:21.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.562 INFO:teuthology.orchestra.run.smithi061.stdout: "outside_quorum": [], 2023-12-06T18:27:21.562 INFO:teuthology.orchestra.run.smithi061.stdout: "extra_probe_peers": [], 2023-12-06T18:27:21.562 INFO:teuthology.orchestra.run.smithi061.stdout: "sync_provider": [], 2023-12-06T18:27:21.562 INFO:teuthology.orchestra.run.smithi061.stdout: "monmap": { 2023-12-06T18:27:21.563 INFO:teuthology.orchestra.run.smithi061.stdout: "epoch": 1, 2023-12-06T18:27:21.563 INFO:teuthology.orchestra.run.smithi061.stdout: "fsid": "497e3283-cc16-40ff-a72c-a7487474fb4b", 2023-12-06T18:27:21.563 INFO:teuthology.orchestra.run.smithi061.stdout: "modified": "2023-12-06T18:25:15.366722Z", 2023-12-06T18:27:21.563 INFO:teuthology.orchestra.run.smithi061.stdout: "created": "2023-12-06T18:25:15.366722Z", 2023-12-06T18:27:21.563 INFO:teuthology.orchestra.run.smithi061.stdout: "min_mon_release": 18, 2023-12-06T18:27:21.563 INFO:teuthology.orchestra.run.smithi061.stdout: "min_mon_release_name": "reef", 2023-12-06T18:27:21.563 INFO:teuthology.orchestra.run.smithi061.stdout: "election_strategy": 3, 2023-12-06T18:27:21.563 INFO:teuthology.orchestra.run.smithi061.stdout: "disallowed_leaders: ": "", 2023-12-06T18:27:21.564 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode": false, 2023-12-06T18:27:21.564 INFO:teuthology.orchestra.run.smithi061.stdout: "tiebreaker_mon": "", 2023-12-06T18:27:21.564 INFO:teuthology.orchestra.run.smithi061.stdout: "removed_ranks: ": "", 2023-12-06T18:27:21.564 INFO:teuthology.orchestra.run.smithi061.stdout: "features": { 2023-12-06T18:27:21.564 INFO:teuthology.orchestra.run.smithi061.stdout: "persistent": [ 2023-12-06T18:27:21.564 INFO:teuthology.orchestra.run.smithi061.stdout: "kraken", 2023-12-06T18:27:21.565 INFO:teuthology.orchestra.run.smithi061.stdout: "luminous", 2023-12-06T18:27:21.565 INFO:teuthology.orchestra.run.smithi061.stdout: "mimic", 2023-12-06T18:27:21.565 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap-prune", 2023-12-06T18:27:21.565 INFO:teuthology.orchestra.run.smithi061.stdout: "nautilus", 2023-12-06T18:27:21.565 INFO:teuthology.orchestra.run.smithi061.stdout: "octopus", 2023-12-06T18:27:21.565 INFO:teuthology.orchestra.run.smithi061.stdout: "pacific", 2023-12-06T18:27:21.566 INFO:teuthology.orchestra.run.smithi061.stdout: "elector-pinging", 2023-12-06T18:27:21.566 INFO:teuthology.orchestra.run.smithi061.stdout: "quincy", 2023-12-06T18:27:21.566 INFO:teuthology.orchestra.run.smithi061.stdout: "reef" 2023-12-06T18:27:21.566 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:27:21.566 INFO:teuthology.orchestra.run.smithi061.stdout: "optional": [] 2023-12-06T18:27:21.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.567 INFO:teuthology.orchestra.run.smithi061.stdout: "mons": [ 2023-12-06T18:27:21.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.567 INFO:teuthology.orchestra.run.smithi061.stdout: "rank": 0, 2023-12-06T18:27:21.567 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "a", 2023-12-06T18:27:21.567 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:27:21.567 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:27:21.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.567 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:27:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:3300", 2023-12-06T18:27:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 0 2023-12-06T18:27:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v1", 2023-12-06T18:27:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6789", 2023-12-06T18:27:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 0 2023-12-06T18:27:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6789/0", 2023-12-06T18:27:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6789/0", 2023-12-06T18:27:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "priority": 0, 2023-12-06T18:27:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 0, 2023-12-06T18:27:21.568 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_location": "{}" 2023-12-06T18:27:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "rank": 1, 2023-12-06T18:27:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "b", 2023-12-06T18:27:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:27:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:27:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:27:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:3300", 2023-12-06T18:27:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 0 2023-12-06T18:27:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v1", 2023-12-06T18:27:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6789", 2023-12-06T18:27:21.569 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 0 2023-12-06T18:27:21.570 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.570 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:21.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.570 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6789/0", 2023-12-06T18:27:21.570 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6789/0", 2023-12-06T18:27:21.570 INFO:teuthology.orchestra.run.smithi061.stdout: "priority": 0, 2023-12-06T18:27:21.570 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 0, 2023-12-06T18:27:21.570 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_location": "{}" 2023-12-06T18:27:21.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.570 INFO:teuthology.orchestra.run.smithi061.stdout: "rank": 2, 2023-12-06T18:27:21.570 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "c", 2023-12-06T18:27:21.570 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:27:21.570 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:27:21.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.571 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:27:21.571 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:3301", 2023-12-06T18:27:21.571 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 0 2023-12-06T18:27:21.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.571 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v1", 2023-12-06T18:27:21.571 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6790", 2023-12-06T18:27:21.571 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 0 2023-12-06T18:27:21.571 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.571 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:21.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.571 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6790/0", 2023-12-06T18:27:21.571 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6790/0", 2023-12-06T18:27:21.572 INFO:teuthology.orchestra.run.smithi061.stdout: "priority": 0, 2023-12-06T18:27:21.572 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 0, 2023-12-06T18:27:21.572 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_location": "{}" 2023-12-06T18:27:21.572 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.572 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:21.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.572 INFO:teuthology.orchestra.run.smithi061.stdout: "feature_map": { 2023-12-06T18:27:21.572 INFO:teuthology.orchestra.run.smithi061.stdout: "mon": [ 2023-12-06T18:27:21.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.572 INFO:teuthology.orchestra.run.smithi061.stdout: "features": "0x3f01cfbffffdffff", 2023-12-06T18:27:21.572 INFO:teuthology.orchestra.run.smithi061.stdout: "release": "luminous", 2023-12-06T18:27:21.572 INFO:teuthology.orchestra.run.smithi061.stdout: "num": 1 2023-12-06T18:27:21.572 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.572 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:27:21.572 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": [ 2023-12-06T18:27:21.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.573 INFO:teuthology.orchestra.run.smithi061.stdout: "features": "0x3f01cfbffffdffff", 2023-12-06T18:27:21.573 INFO:teuthology.orchestra.run.smithi061.stdout: "release": "luminous", 2023-12-06T18:27:21.573 INFO:teuthology.orchestra.run.smithi061.stdout: "num": 3 2023-12-06T18:27:21.573 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.573 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:27:21.573 INFO:teuthology.orchestra.run.smithi061.stdout: "client": [ 2023-12-06T18:27:21.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.573 INFO:teuthology.orchestra.run.smithi061.stdout: "features": "0x3f01cfbffffdffff", 2023-12-06T18:27:21.573 INFO:teuthology.orchestra.run.smithi061.stdout: "release": "luminous", 2023-12-06T18:27:21.573 INFO:teuthology.orchestra.run.smithi061.stdout: "num": 3 2023-12-06T18:27:21.574 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.574 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:27:21.574 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr": [ 2023-12-06T18:27:21.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.574 INFO:teuthology.orchestra.run.smithi061.stdout: "features": "0x3f01cfbffffdffff", 2023-12-06T18:27:21.574 INFO:teuthology.orchestra.run.smithi061.stdout: "release": "luminous", 2023-12-06T18:27:21.574 INFO:teuthology.orchestra.run.smithi061.stdout: "num": 1 2023-12-06T18:27:21.574 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.574 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:21.574 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.574 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode": false 2023-12-06T18:27:21.574 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:21.575 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2023-12-06T18:27:21.580 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:21.576+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:21.605 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:21.605 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:21.613 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:27:21.628 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:27:21.628 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:27:21.637 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-12-06T18:27:21.699 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:21.695+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:21.782 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:21.778+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:21.789 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:21.789 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:21.798 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:27:21.814 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:27:21.814 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:27:21.840 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "b", 2023-12-06T18:27:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: "rank": 1, 2023-12-06T18:27:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: "state": "peon", 2023-12-06T18:27:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: "election_epoch": 10, 2023-12-06T18:27:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: "quorum": [ 2023-12-06T18:27:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: 0, 2023-12-06T18:27:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: 1, 2023-12-06T18:27:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: 2 2023-12-06T18:27:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:27:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "quorum_age": 74, 2023-12-06T18:27:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "features": { 2023-12-06T18:27:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "required_con": "2449958758054445060", 2023-12-06T18:27:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "required_mon": [ 2023-12-06T18:27:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "kraken", 2023-12-06T18:27:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "luminous", 2023-12-06T18:27:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "mimic", 2023-12-06T18:27:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap-prune", 2023-12-06T18:27:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "nautilus", 2023-12-06T18:27:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "octopus", 2023-12-06T18:27:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "pacific", 2023-12-06T18:27:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "elector-pinging", 2023-12-06T18:27:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "quincy", 2023-12-06T18:27:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "reef" 2023-12-06T18:27:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:27:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "quorum_con": "4540138322906710015", 2023-12-06T18:27:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "quorum_mon": [ 2023-12-06T18:27:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "kraken", 2023-12-06T18:27:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "luminous", 2023-12-06T18:27:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "mimic", 2023-12-06T18:27:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap-prune", 2023-12-06T18:27:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "nautilus", 2023-12-06T18:27:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "octopus", 2023-12-06T18:27:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "pacific", 2023-12-06T18:27:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "elector-pinging", 2023-12-06T18:27:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "quincy", 2023-12-06T18:27:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "reef" 2023-12-06T18:27:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "outside_quorum": [], 2023-12-06T18:27:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "extra_probe_peers": [], 2023-12-06T18:27:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "sync_provider": [], 2023-12-06T18:27:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "monmap": { 2023-12-06T18:27:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "epoch": 1, 2023-12-06T18:27:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "fsid": "497e3283-cc16-40ff-a72c-a7487474fb4b", 2023-12-06T18:27:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "modified": "2023-12-06T18:25:15.366722Z", 2023-12-06T18:27:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "created": "2023-12-06T18:25:15.366722Z", 2023-12-06T18:27:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "min_mon_release": 18, 2023-12-06T18:27:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "min_mon_release_name": "reef", 2023-12-06T18:27:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "election_strategy": 3, 2023-12-06T18:27:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "disallowed_leaders: ": "", 2023-12-06T18:27:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode": false, 2023-12-06T18:27:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "tiebreaker_mon": "", 2023-12-06T18:27:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "removed_ranks: ": "", 2023-12-06T18:27:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "features": { 2023-12-06T18:27:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "persistent": [ 2023-12-06T18:27:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "kraken", 2023-12-06T18:27:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "luminous", 2023-12-06T18:27:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "mimic", 2023-12-06T18:27:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "osdmap-prune", 2023-12-06T18:27:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "nautilus", 2023-12-06T18:27:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "octopus", 2023-12-06T18:27:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "pacific", 2023-12-06T18:27:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "elector-pinging", 2023-12-06T18:27:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "quincy", 2023-12-06T18:27:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "reef" 2023-12-06T18:27:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:27:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "optional": [] 2023-12-06T18:27:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "mons": [ 2023-12-06T18:27:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "rank": 0, 2023-12-06T18:27:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "a", 2023-12-06T18:27:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:27:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:27:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:27:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:3300", 2023-12-06T18:27:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 0 2023-12-06T18:27:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v1", 2023-12-06T18:27:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6789", 2023-12-06T18:27:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 0 2023-12-06T18:27:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6789/0", 2023-12-06T18:27:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6789/0", 2023-12-06T18:27:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: "priority": 0, 2023-12-06T18:27:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 0, 2023-12-06T18:27:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_location": "{}" 2023-12-06T18:27:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "rank": 1, 2023-12-06T18:27:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "b", 2023-12-06T18:27:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:27:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:27:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:27:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:3300", 2023-12-06T18:27:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 0 2023-12-06T18:27:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v1", 2023-12-06T18:27:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6789", 2023-12-06T18:27:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 0 2023-12-06T18:27:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6789/0", 2023-12-06T18:27:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6789/0", 2023-12-06T18:27:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: "priority": 0, 2023-12-06T18:27:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 0, 2023-12-06T18:27:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_location": "{}" 2023-12-06T18:27:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: "rank": 2, 2023-12-06T18:27:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: "name": "c", 2023-12-06T18:27:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:27:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:27:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:27:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:3301", 2023-12-06T18:27:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 0 2023-12-06T18:27:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v1", 2023-12-06T18:27:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6790", 2023-12-06T18:27:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 0 2023-12-06T18:27:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6790/0", 2023-12-06T18:27:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6790/0", 2023-12-06T18:27:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: "priority": 0, 2023-12-06T18:27:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 0, 2023-12-06T18:27:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_location": "{}" 2023-12-06T18:27:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: "feature_map": { 2023-12-06T18:27:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: "mon": [ 2023-12-06T18:27:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: "features": "0x3f01cfbffffdffff", 2023-12-06T18:27:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: "release": "luminous", 2023-12-06T18:27:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: "num": 1 2023-12-06T18:27:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:27:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": [ 2023-12-06T18:27:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: "features": "0x3f01cfbffffdffff", 2023-12-06T18:27:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: "release": "luminous", 2023-12-06T18:27:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: "num": 4 2023-12-06T18:27:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:27:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: "client": [ 2023-12-06T18:27:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: "features": "0x3f01cfbffffdffff", 2023-12-06T18:27:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: "release": "luminous", 2023-12-06T18:27:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: "num": 4 2023-12-06T18:27:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:21.852 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode": false 2023-12-06T18:27:21.852 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:21.853 INFO:tasks.mon_recovery:restarting each monitor in turn 2023-12-06T18:27:21.854 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-12-06T18:27:21.854 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:27:21.874 INFO:tasks.ceph.mon.a.smithi061.stderr:2023-12-06T18:27:21.857+0000 9f54640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 86916) UID: 0 2023-12-06T18:27:21.874 INFO:tasks.ceph.mon.a.smithi061.stderr:2023-12-06T18:27:21.859+0000 9f54640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-12-06T18:27:21.933 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:27:21.933 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:27:21.935 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:21.931+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:21.984 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:21.981+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:21.990 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:21.991 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:22.001 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:27:22.043 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:27:22.043 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:27:22.086 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:22.082+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:22.138 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:27:22.138 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:27:22.176 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:22.176 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:22.185 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:27:22.186 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:22.182+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:22.231 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:27:22.231 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:27:22.287 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:22.284+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:22.323 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:27:22.323 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:27:22.367 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:22.367 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:27:22.367 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:27:22.367 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:27:22.367 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:22.376 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:27:22.425 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:22.422+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:22.522 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:22.519+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:22.536 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:22.536 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:22.536 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:27:22.536 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:22.545 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:27:22.585 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:33.924 86952== 2023-12-06T18:27:22.585 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:33.924 86952== Exit program on first error (--exit-on-first-error=yes) 2023-12-06T18:27:22.590 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:22.586+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:22.597 INFO:tasks.ceph.mon.a.smithi061.stderr:daemon-helper: command failed with exit status 42 2023-12-06T18:27:22.691 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:22.687+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:22.724 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:22.725 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:22.725 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:27:22.725 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:27:22.725 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:22.734 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:27:22.792 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:22.788+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:22.871 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:22.872 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:27:22.872 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:27:22.872 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:27:22.872 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:22.881 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:27:22.892 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:22.889+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:22.994 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:22.991+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:23.020 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:23.021 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:23.021 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:27:23.021 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:23.030 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:27:23.115 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:23.112+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:23.196 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:23.193+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:23.210 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:23.210 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:23.210 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:27:23.211 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:27:23.211 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:23.221 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:27:23.306 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:23.303+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:23.398 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:23.394+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:23.471 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:23.472 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:27:23.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:27:23.472 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:27:23.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.472 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:27:23.472 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:27:23.472 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 28.046469222999999, 2023-12-06T18:27:23.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:27:23.472 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.472 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.472 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.472 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.472 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.472 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:27:23.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.473 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:27:23.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:27:23.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:27:23.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:27:23.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:27:23.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:27:23.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:27:23.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:27:23.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:27:23.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:27:23.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:27:23.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:27:23.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:27:23.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:27:23.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:27:23.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:27:23.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:27:23.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:27:23.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:27:23.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:27:23.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:27:23.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:27:23.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:27:23.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:27:23.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:27:23.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:27:23.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:27:23.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:27:23.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:27:23.478 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.478 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.478 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.478 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:23.478 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:27:23.478 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 25.937895984000001, 2023-12-06T18:27:23.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:27:23.478 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.478 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.478 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.478 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.478 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.479 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:27:23.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.479 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:27:23.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:27:23.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:27:23.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:27:23.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:27:23.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:27:23.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:27:23.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:27:23.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:27:23.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:27:23.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:27:23.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:27:23.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:27:23.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:27:23.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:27:23.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:27:23.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:27:23.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:27:23.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:27:23.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:27:23.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:27:23.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:27:23.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:27:23.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:27:23.483 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.483 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.483 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.483 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:23.483 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:27:23.483 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 25.937150752000001, 2023-12-06T18:27:23.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:27:23.483 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.483 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.484 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.484 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.484 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.484 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:27:23.484 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.484 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.484 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.484 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.484 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:27:23.484 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.484 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.484 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.484 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.484 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:27:23.484 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.485 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.485 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:27:23.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:27:23.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.485 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.485 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:27:23.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:27:23.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.485 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.485 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:27:23.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:27:23.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:27:23.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:27:23.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:27:23.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:27:23.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:27:23.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:27:23.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:27:23.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:27:23.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:27:23.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:27:23.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:27:23.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:27:23.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:27:23.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:27:23.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:27:23.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:27:23.488 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.488 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.488 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.489 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:23.489 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:27:23.489 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 25.936466997, 2023-12-06T18:27:23.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:27:23.489 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.489 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.489 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.489 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.489 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.489 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:27:23.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.489 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:27:23.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.490 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.490 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:27:23.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.490 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.490 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:27:23.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:27:23.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.490 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.490 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:27:23.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:27:23.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:27:23.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:27:23.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:27:23.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:27:23.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:27:23.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:27:23.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:27:23.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:27:23.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:27:23.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:27:23.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:27:23.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:27:23.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:27:23.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:27:23.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:27:23.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:27:23.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:27:23.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:27:23.493 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.493 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.494 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.494 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:23.494 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:27:23.494 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 25.930893741999999, 2023-12-06T18:27:23.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:27:23.494 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.494 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.494 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.494 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.494 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.494 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:27:23.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.495 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:27:23.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:27:23.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:27:23.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:27:23.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.496 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:27:23.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:27:23.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.496 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:27:23.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:27:23.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.496 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:27:23.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:27:23.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:27:23.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:27:23.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:27:23.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:27:23.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:27:23.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:27:23.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:27:23.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:27:23.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:27:23.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:27:23.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:27:23.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:27:23.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:27:23.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:27:23.499 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.499 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.499 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.499 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:23.499 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:27:23.499 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 25.930220102, 2023-12-06T18:27:23.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:27:23.499 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.500 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.500 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.500 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.500 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.500 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:27:23.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.500 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:27:23.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:27:23.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.501 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.501 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:27:23.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:27:23.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.501 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.501 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:27:23.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:27:23.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.501 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.501 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:27:23.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:27:23.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:27:23.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:27:23.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:27:23.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:27:23.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:27:23.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:27:23.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:27:23.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:27:23.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:27:23.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:27:23.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:27:23.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:27:23.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:27:23.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:27:23.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:27:23.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:27:23.504 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.504 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.504 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.504 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:23.505 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:27:23.505 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 25.929557902999999, 2023-12-06T18:27:23.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:27:23.505 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.505 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.505 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.505 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.505 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.505 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:27:23.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.505 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:27:23.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:27:23.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:27:23.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:27:23.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:27:23.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:27:23.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:27:23.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:27:23.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:27:23.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:27:23.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:27:23.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:27:23.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:27:23.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:27:23.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:27:23.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:27:23.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:27:23.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:27:23.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:27:23.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:27:23.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:27:23.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:27:23.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:27:23.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:27:23.509 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.509 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.510 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.510 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:23.510 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:27:23.510 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 25.856848467999999, 2023-12-06T18:27:23.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:27:23.510 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.510 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.510 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.510 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.510 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.510 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:27:23.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.510 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:27:23.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:27:23.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:27:23.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:27:23.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:27:23.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:27:23.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:27:23.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:27:23.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:27:23.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:27:23.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:27:23.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:27:23.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:27:23.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:27:23.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:27:23.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:27:23.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:27:23.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:27:23.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:27:23.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:27:23.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:27:23.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:27:23.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:27:23.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:27:23.515 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.515 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.515 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.515 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:23.515 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:27:23.515 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 25.856166759000001, 2023-12-06T18:27:23.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:27:23.515 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.515 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.515 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.515 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.516 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.516 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:27:23.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.516 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:27:23.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:27:23.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:27:23.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:27:23.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:27:23.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:27:23.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:27:23.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:27:23.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:27:23.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:27:23.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:27:23.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:27:23.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:27:23.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:27:23.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:27:23.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:27:23.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:27:23.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:27:23.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:27:23.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:27:23.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:27:23.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:27:23.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:27:23.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:27:23.520 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.520 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.520 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.520 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:27:23.520 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:27:23.520 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 24.777362670999999, 2023-12-06T18:27:23.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:27:23.521 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.521 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.521 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.521 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.521 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.521 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:27:23.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.521 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:27:23.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:27:23.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.522 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.522 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:27:23.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:27:23.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.522 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.522 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:27:23.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:27:23.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.522 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.522 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:27:23.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:27:23.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:27:23.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:27:23.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:27:23.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:27:23.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:27:23.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:27:23.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:27:23.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:27:23.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:27:23.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:27:23.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:27:23.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:27:23.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:27:23.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:27:23.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:27:23.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:27:23.525 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.525 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.525 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.525 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:27:23.525 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:27:23.526 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 24.775946096999999, 2023-12-06T18:27:23.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:27:23.526 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.526 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.526 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.526 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.526 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.526 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:27:23.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.526 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:27:23.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:27:23.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:27:23.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:27:23.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:27:23.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:27:23.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:27:23.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:27:23.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:27:23.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:27:23.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:27:23.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:27:23.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:27:23.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:27:23.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:27:23.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:27:23.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:27:23.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:27:23.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:27:23.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:27:23.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:27:23.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:27:23.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:27:23.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:27:23.530 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.530 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.530 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.531 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:27:23.531 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:27:23.531 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 24.774860971999999, 2023-12-06T18:27:23.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:27:23.531 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.531 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.531 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.531 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.531 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.531 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:27:23.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.531 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:27:23.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:27:23.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:27:23.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:27:23.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:27:23.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:27:23.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:27:23.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:27:23.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:27:23.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:27:23.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:27:23.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:27:23.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:27:23.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:27:23.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:27:23.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:27:23.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:27:23.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:27:23.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:27:23.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:27:23.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:27:23.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:27:23.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:27:23.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:27:23.536 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.536 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.536 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.536 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:27:23.536 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:27:23.536 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 24.739711870000001, 2023-12-06T18:27:23.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:27:23.536 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.536 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.536 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.536 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.536 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.536 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:27:23.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.537 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:27:23.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:27:23.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:27:23.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:27:23.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.538 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:27:23.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:27:23.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.538 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:27:23.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:27:23.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.538 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:27:23.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:27:23.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:27:23.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:27:23.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:27:23.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:27:23.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:27:23.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:27:23.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:27:23.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:27:23.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:27:23.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:27:23.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:27:23.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:27:23.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:27:23.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:27:23.541 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.541 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.541 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.541 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:23.541 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:27:23.541 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 18.189981345, 2023-12-06T18:27:23.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:27:23.541 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.541 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.542 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.542 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.542 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.542 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:27:23.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.542 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:27:23.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:27:23.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:27:23.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:27:23.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:27:23.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:27:23.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:27:23.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:27:23.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:27:23.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:27:23.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:27:23.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:27:23.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:27:23.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:27:23.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:27:23.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:27:23.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:27:23.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:27:23.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:27:23.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:27:23.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:27:23.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:27:23.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:27:23.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:27:23.546 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.546 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.546 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.546 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:23.546 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:27:23.546 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 18.189175736999999, 2023-12-06T18:27:23.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:27:23.547 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.547 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.547 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.547 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.547 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.547 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:27:23.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.547 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:27:23.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:27:23.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:27:23.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:27:23.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:27:23.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:27:23.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:27:23.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:27:23.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:27:23.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:27:23.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:27:23.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:27:23.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:27:23.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:27:23.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:27:23.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:27:23.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:27:23.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:27:23.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:27:23.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:27:23.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:27:23.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:27:23.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:27:23.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:27:23.551 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.551 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.552 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.552 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:23.552 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:27:23.552 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 18.188484928000001, 2023-12-06T18:27:23.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:27:23.552 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.552 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.552 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.552 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.552 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.552 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:27:23.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.552 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:27:23.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:27:23.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:27:23.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:27:23.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:27:23.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:27:23.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:27:23.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:27:23.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:27:23.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:27:23.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:27:23.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:27:23.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:27:23.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:27:23.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:27:23.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:27:23.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.556 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:27:23.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:27:23.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.556 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.556 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:27:23.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:27:23.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.556 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.556 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:27:23.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:27:23.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.556 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:27:23.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:27:23.557 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.557 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.557 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.557 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:23.557 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:27:23.557 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 18.184187420000001, 2023-12-06T18:27:23.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:27:23.557 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.557 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.557 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.557 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.557 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.558 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:27:23.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.558 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:27:23.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:27:23.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:27:23.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:27:23.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:27:23.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:27:23.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:27:23.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:27:23.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:27:23.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:27:23.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:27:23.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:27:23.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:27:23.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:27:23.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:27:23.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:27:23.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:27:23.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:27:23.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:27:23.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:27:23.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:27:23.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:27:23.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:27:23.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:27:23.562 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.562 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.562 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.563 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:23.563 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:27:23.563 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 18.183519745000002, 2023-12-06T18:27:23.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:27:23.563 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.563 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.563 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.563 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.563 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.563 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:27:23.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.563 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:27:23.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:27:23.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:27:23.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:27:23.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:27:23.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:27:23.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:27:23.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:27:23.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:27:23.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:27:23.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:27:23.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:27:23.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:27:23.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:27:23.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:27:23.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:27:23.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.567 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:27:23.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:27:23.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.567 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.567 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:27:23.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:27:23.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.567 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.567 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:27:23.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:27:23.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.567 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.568 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:27:23.568 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:27:23.568 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.568 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.568 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.568 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:23.568 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:27:23.568 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 18.182864317, 2023-12-06T18:27:23.568 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:27:23.568 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.568 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.568 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.568 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.568 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.569 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:27:23.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.569 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:27:23.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:27:23.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:27:23.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:27:23.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.570 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.570 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:27:23.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:27:23.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.570 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.570 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:27:23.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:27:23.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.570 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.570 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:27:23.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:27:23.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:27:23.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:27:23.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:27:23.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:27:23.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:27:23.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:27:23.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.572 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:27:23.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:27:23.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.572 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:27:23.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:27:23.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.572 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:27:23.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:27:23.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.573 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.573 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:27:23.573 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:27:23.573 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.573 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.573 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.573 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:23.573 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:27:23.573 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 18.142209770000001, 2023-12-06T18:27:23.573 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:27:23.573 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:23.573 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:23.574 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:23.574 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:23.574 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:23.574 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:27:23.574 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.574 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:23.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:23.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:27:23.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.574 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:23.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:27:23.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:23.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.575 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:23.575 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:27:23.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:27:23.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.575 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:23.575 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:27:23.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:27:23.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.575 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:23.575 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:27:23.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:27:23.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.576 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.576 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:23.576 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:27:23.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:27:23.576 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.576 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.576 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:23.576 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:27:23.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:27:23.576 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.576 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.576 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:23.576 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:27:23.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:27:23.576 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.576 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.577 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:23.577 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:27:23.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:27:23.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.577 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:23.577 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:27:23.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:27:23.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.577 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:23.577 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:27:23.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:27:23.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.577 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:23.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:27:23.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:27:23.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:23.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:23.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:23.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:27:23.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:27:23.578 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.578 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.578 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.578 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:23.578 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:23.578 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:23.580 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:27:23.580 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:23.576+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:23.681 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:23.677+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:23.763 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:23.763 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:23.764 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:27:23.764 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:23.773 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:27:23.817 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:23.814+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:23.883 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:23.879+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:23.911 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:23.911 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:23.911 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:27:23.911 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:27:23.911 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:23.921 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:27:24.006 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:24.003+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:24.084 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:24.082+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:24.112 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:24.112 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:27:24.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:27:24.113 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:27:24.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:24.113 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:27:24.113 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:27:24.113 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 2.286491029, 2023-12-06T18:27:24.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:27:24.113 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:24.113 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:27:24.113 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:24.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:24.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:24.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:27:24.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:24.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:24.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:24.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:24.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:27:24.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:24.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:24.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:24.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:24.114 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:27:24.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:27:24.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:24.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:24.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:24.114 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:27:24.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:27:24.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:24.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:24.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:24.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:27:24.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:27:24.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:24.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:24.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:24.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:27:24.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:27:24.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:24.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:24.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:24.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:27:24.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:27:24.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:24.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:24.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:24.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:27:24.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:27:24.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:24.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:24.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:24.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:27:24.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:27:24.116 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:24.116 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:24.116 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:24.116 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:24.116 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:24.116 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:24.122 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:27:24.184 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-12-06T18:27:24.186 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:24.182+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:24.287 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:24.284+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:24.305 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:24.305 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:24.305 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:27:24.305 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:24.314 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:27:24.340 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:27:24.388 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:24.385+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:24.489 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:24.487+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:24.508 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:24.508 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:24.508 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:27:24.508 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:27:24.509 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:24.519 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:27:24.590 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:27:24.591 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:27:24.591 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:24.587+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:24.616 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:27:24.617 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:27:24.691 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:24.688+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:24.733 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:24.733 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:27:24.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:27:24.733 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:27:24.733 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:24.745 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:27:24.746 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:27:24.746 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:27:24.839 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:27:24.839 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:27:24.839 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:24.837+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:24.891 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:24.891 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:27:24.891 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:27:24.891 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:24.893 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:24.891+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:24.900 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:27:24.931 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:27:24.931 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:27:24.995 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:24.991+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:25.048 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:27:25.049 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:27:25.091 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:25.091 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:25.097 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:25.094+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:25.100 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:27:25.142 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:27:25.142 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:27:25.197 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:25.194+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:25.232 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:27:25.232 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:27:25.272 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:25.272 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:25.281 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:27:25.298 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:25.295+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:25.323 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:27:25.323 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:27:25.399 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:25.397+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:25.453 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:25.453 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:25.462 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:27:25.500 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:25.497+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:25.591 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:25.591 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:25.600 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:27:25.602 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:25.599+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:25.703 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:25.699+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:25.730 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:25.730 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:25.741 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:27:25.805 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:25.802+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:25.906 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:25.904+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:25.913 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:25.913 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:25.922 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:27:26.007 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:26.004+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:26.049 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:26.049 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:26.058 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:27:26.108 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:26.105+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:26.187 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:26.187 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:26.197 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:27:26.239 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:26.235+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:26.310 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:26.307+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:26.367 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:26.367 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:26.376 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:27:26.412 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:26.409+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:26.512 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:26.508+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:26.543 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:26.543 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:26.552 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:27:26.614 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:26.611+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:26.717 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:26.715+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:26.724 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:26.724 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:26.733 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:27:26.816 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:26.813+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:26.880 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:26.880 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:26.890 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:27:26.917 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:26.913+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:27.018 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:27.014+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:27.067 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:27.068 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:27.077 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:27:27.122 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:27.119+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:27.215 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:27.215 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:27.219 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:27.216+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:27.224 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:27:27.320 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:27.317+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:27.335 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:27:27.404 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:27.404 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:27.413 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:27:27.458 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:27.455+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:27.522 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:27.519+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:27.561 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:27.561 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:27.570 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:27:27.622 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:27.619+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:27.724 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:27.720+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:27.751 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:27.751 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:27.760 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:27:27.824 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:27.821+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:27.897 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:27.897 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:27.906 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:27:27.925 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:27.922+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:27.955 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-12-06T18:27:27.955 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi061 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a" 2023-12-06T18:27:27.956 INFO:tasks.ceph.mon.a:Stopped 2023-12-06T18:27:27.956 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2023-12-06T18:27:27.956 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:27:28.026 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:28.023+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:28.084 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:28.084 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:28.093 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:27:28.139 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:28.136+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:28.228 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:28.226+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:28.241 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:28.241 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:28.251 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:27:28.335 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:28.333+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:28.430 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:28.427+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:28.456 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:28.456 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:28.466 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:27:28.551 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:28.548+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:28.633 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:28.629+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:28.645 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:28.646 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:28.656 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:27:28.701 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":81,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:27:28.756 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:28.753+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:28.833 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:28.834 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:28.834 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:28.831+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:28.842 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:27:28.935 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:28.931+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:28.984 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:28.984 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:28.993 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:27:29.036 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:29.033+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:29.137 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:29.133+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:29.161 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:29.161 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:29.170 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:27:29.255 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:29.251+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:29.338 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:29.339 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:29.339 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:29.336+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:29.348 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:27:29.439 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:29.436+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:29.517 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:29.517 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:29.526 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:27:29.540 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:29.538+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:29.591 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:27:29.591 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:27:29.641 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:29.638+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:29.696 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:29.696 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:29.705 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:27:29.748 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:29.743+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:29.843 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:29.839+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:29.874 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:29.874 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:29.884 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:27:29.969 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:29.965+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:30.045 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:30.041+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:30.054 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:30.054 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:30.063 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:27:30.147 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:30.144+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:30.235 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:30.235 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:30.244 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:27:30.286 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:30.282+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:30.347 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:30.344+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:30.391 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-12-06T18:27:30.414 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:30.414 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:30.424 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:27:30.468 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:30.464+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:30.548 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:30.545+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:30.552 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:30.552 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:30.561 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:27:30.649 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:30.645+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:30.729 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:30.729 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:30.738 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:27:30.780 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:30.776+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:30.850 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:30.847+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:30.916 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:30.916 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:30.924 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:27:30.965 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:30.962+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:31.051 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:31.048+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:31.081 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:31.081 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:31.090 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:27:31.153 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:31.149+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:31.228 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:31.228 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:31.238 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:27:31.284 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:31.281+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:31.355 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:31.352+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:31.376 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:31.377 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:31.386 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:27:31.455 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:31.452+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:31.556 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:31.553+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:31.564 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:31.564 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:31.574 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:27:31.658 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:31.655+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:31.710 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:31.711 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:31.715 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:27:31.720 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:27:31.761 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:31.758+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:31.859 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:31.857+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:31.900 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:31.900 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:31.909 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:27:31.960 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:31.956+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:32.061 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:32.058+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:32.094 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:32.094 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:32.103 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:27:32.162 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:32.158+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:32.263 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:32.259+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:32.280 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:32.281 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:32.290 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:27:32.374 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:32.370+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:32.464 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:32.461+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:32.466 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:32.467 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:32.475 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:27:32.566 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:32.562+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:32.654 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:32.654 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:32.664 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:27:32.667 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:32.663+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:32.767 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:32.764+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:32.841 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:32.842 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:32.851 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:27:32.869 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:32.865+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:32.971 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:32.967+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:33.029 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:33.029 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:33.038 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:27:33.070 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:33.067+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:33.172 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:33.168+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:33.178 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:33.178 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:33.188 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:27:33.273 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:33.269+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:33.317 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:33.317 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:33.326 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:27:33.374 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:33.370+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:33.454 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:33.454 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:33.464 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:27:33.509 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:33.505+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:33.575 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:33.572+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:33.593 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:33.593 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:33.603 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:27:33.688 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:33.684+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:33.772 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:33.772 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:33.778 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:33.774+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:33.782 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:27:33.879 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:33.875+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:33.949 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:33.949 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:33.957 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:27:33.979 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:33.976+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:34.081 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:34.077+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:34.125 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:34.126 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:34.135 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:27:34.181 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:34.178+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:34.262 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:34.262 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:34.271 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:27:34.283 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:34.279+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:34.384 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:34.380+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:34.442 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:34.443 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:34.452 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:27:34.497 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:34.493+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:34.586 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:34.582+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:34.588 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:34.588 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:34.592 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:27:34.592 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-12-06T18:27:34.597 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:27:34.687 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:34.683+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:34.767 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:34.767 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:34.777 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:27:34.821 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:34.817+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:34.889 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:34.885+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:34.911 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:34.911 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:34.920 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:27:34.989 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:34.986+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:35.060 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:35.060 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:35.069 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:27:35.090 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:35.087+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:35.191 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:35.188+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:35.205 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:35.205 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:35.215 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:27:35.293 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:35.289+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:35.352 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:35.353 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:35.362 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:27:35.394 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:35.390+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:35.494 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:35.491+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:35.543 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:35.544 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:35.553 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:27:35.595 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:35.592+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:35.695 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:35.692+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:35.731 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:35.731 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:35.740 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:27:35.796 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:35.793+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:35.876 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:35.877 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:35.885 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:27:35.898 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:35.894+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:35.998 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:35.995+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:36.063 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:36.063 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:36.072 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:27:36.116 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:36.112+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:36.200 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:36.196+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:36.209 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:36.210 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:36.219 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:27:36.300 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:36.297+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:36.397 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:36.397 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:36.402 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:36.398+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:36.406 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:27:36.502 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:36.499+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:36.548 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:36.549 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:36.558 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:27:36.604 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:36.600+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:36.701 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:36.701 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:36.704 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:36.700+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:36.710 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:27:36.805 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:36.801+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:36.893 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:36.893 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:36.903 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:27:36.905 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:36.901+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:37.006 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:37.003+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:37.039 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:37.039 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:37.048 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:27:37.133 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:37.129+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:37.209 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:37.205+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:37.221 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:37.221 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:37.230 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:27:37.315 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:37.311+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:37.398 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:37.399 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:37.408 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:27:37.410 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:37.406+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:37.511 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:37.507+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:37.576 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:37.577 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:37.585 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:27:37.611 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:37.608+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:37.712 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:37.709+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:37.753 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:37.753 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:37.762 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:27:37.813 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:37.810+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:37.891 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:37.891 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:37.900 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:27:37.944 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:37.940+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:38.015 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:38.012+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:38.030 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:38.030 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:38.038 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:27:38.116 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:38.112+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:38.207 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:38.207 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:38.216 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:27:38.381 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:38.213+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:38.383 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:38.380+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:38.485 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:38.481+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:38.510 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:38.510 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:38.519 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:27:38.586 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:38.582+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:38.648 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:38.648 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:38.658 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:27:38.687 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:38.683+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:38.787 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:38.787 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:38.788 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:38.784+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:38.796 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:27:38.889 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:38.886+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:38.926 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:38.926 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:38.936 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:27:38.990 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:38.986+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:39.085 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:39.085 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:39.090 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:39.087+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:39.094 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:27:39.191 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:39.188+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:39.273 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:39.273 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:39.282 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:27:39.292 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:39.289+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:39.393 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:39.390+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:39.418 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:39.418 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:39.427 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:27:39.494 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:39.491+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:39.568 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:39.568 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:39.578 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:27:39.595 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:39.591+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:39.696 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:39.693+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:39.715 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:39.715 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:39.724 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:27:39.798 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:39.794+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:39.861 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:39.861 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:39.870 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:27:39.900 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:39.896+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:40.001 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:39.997+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:40.049 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:40.050 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:40.058 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:27:40.101 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:40.097+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:40.202 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:40.199+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:40.239 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:40.239 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:40.249 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:27:40.303 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:40.299+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:40.389 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:40.389 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:40.399 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:27:40.444 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:40.440+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:40.504 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:40.500+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:40.538 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:40.538 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:40.547 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:27:40.605 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:40.602+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:40.706 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:40.702+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:40.727 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:40.727 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:40.736 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:27:40.809 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:40.804+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:40.874 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:40.874 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:40.883 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:27:40.908 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:40.904+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:41.009 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:41.006+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:41.025 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:41.025 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:41.034 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:27:41.111 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:41.107+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:41.207 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:41.208 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:41.211 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:41.208+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:41.217 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:27:41.312 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:41.309+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:41.385 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:41.385 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:41.395 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:27:41.413 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:41.410+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:41.515 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:41.511+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:41.526 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:41.526 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:41.535 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:27:41.615 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:41.612+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:41.663 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:41.663 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:41.672 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:27:41.717 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:41.714+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:41.801 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:41.801 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:41.812 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:27:41.817 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:41.814+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:41.918 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:41.914+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:41.983 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:41.983 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:41.992 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:27:42.033 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:42.030+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:42.120 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:42.117+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:42.163 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:42.163 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:42.172 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:27:42.221 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:42.218+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:42.259 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:27:42.259 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:27:42.322 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:42.318+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:42.354 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:42.354 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:42.354 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:27:42.355 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:27:42.365 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:27:42.423 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:42.420+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:42.451 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:27:42.451 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:27:42.524 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:42.521+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:42.543 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:27:42.544 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:27:42.566 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:42.566 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:42.575 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:27:42.625 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:42.622+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:42.662 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:27:42.663 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:27:42.726 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:42.722+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:42.760 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:42.760 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:42.770 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:27:42.779 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:27:42.779 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:27:42.826 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:42.823+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:42.868 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":14,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:27:42.883 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2023-12-06T18:27:42.883 INFO:tasks.ceph.mon.a:Restarting daemon 2023-12-06T18:27:42.883 DEBUG:teuthology.orchestra.run.smithi061:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2023-12-06T18:27:42.887 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:27:42.887 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:27:42.889 INFO:tasks.ceph.mon.a:Started 2023-12-06T18:27:42.889 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2023-12-06T18:27:42.889 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:27:42.933 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:42.929+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:42.937 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:42.937 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:42.948 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:27:42.999 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:27:42.999 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:27:43.028 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:43.024+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:43.110 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:43.110 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:43.110 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:27:43.110 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:43.119 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:27:43.129 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:43.125+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:43.230 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:43.226+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:43.278 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:43.279 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:43.279 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:27:43.279 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:27:43.279 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:43.288 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:27:43.342 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:43.327+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:43.432 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:43.428+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:43.433 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:27:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:27:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:27:43.434 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:43.442 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:27:43.484 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is set 2023-12-06T18:27:43.533 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:43.530+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:43.606 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:43.606 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:43.606 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:27:43.606 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:43.615 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:27:43.619 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":14,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:27:43.634 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:43.631+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:43.735 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:43.731+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:43.760 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:27:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:27:43.761 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:43.771 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:27:43.836 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:43.832+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:43.937 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:43.933+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:44.009 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:44.009 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:27:44.009 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:27:44.009 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:27:44.009 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.009 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:27:44.010 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:27:44.010 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 48.597971944000001, 2023-12-06T18:27:44.010 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:27:44.010 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.010 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.010 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.010 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.010 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.010 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:27:44.010 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.010 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.010 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.010 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.011 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:27:44.011 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.011 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.011 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.011 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.011 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:27:44.011 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.011 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.011 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.011 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.011 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:27:44.011 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:27:44.011 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.011 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.012 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.012 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:27:44.012 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:27:44.012 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.012 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.012 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.012 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:27:44.012 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:27:44.012 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.012 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.012 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.012 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:27:44.012 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:27:44.012 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.012 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.013 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.013 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:27:44.013 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:27:44.013 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.013 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.013 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:27:44.013 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:27:44.013 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:27:44.013 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.013 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.013 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.013 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:27:44.013 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:27:44.013 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.013 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.014 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.014 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:27:44.014 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:27:44.014 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.014 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.014 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.014 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:27:44.014 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:27:44.014 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.014 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.014 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.014 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:27:44.014 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:27:44.014 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.014 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.014 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:27:44.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:27:44.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:27:44.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:27:44.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:27:44.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:27:44.015 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.015 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.015 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.016 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:44.016 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:27:44.016 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 46.489398704999999, 2023-12-06T18:27:44.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:27:44.016 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.016 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.016 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.016 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.016 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.016 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:27:44.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.016 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:27:44.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:27:44.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:27:44.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:27:44.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:27:44.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:27:44.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:27:44.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:27:44.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:27:44.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:27:44.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:27:44.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:27:44.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:27:44.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:27:44.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:27:44.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:27:44.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:27:44.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:27:44.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:27:44.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:27:44.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:27:44.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:27:44.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:27:44.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:27:44.020 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.020 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.020 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.020 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:44.021 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:27:44.021 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 46.488653472999999, 2023-12-06T18:27:44.021 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:27:44.021 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.021 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.021 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.021 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.021 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.021 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:27:44.021 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.021 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.021 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.021 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.021 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:27:44.021 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.021 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:27:44.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:27:44.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:27:44.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:27:44.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:27:44.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:27:44.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:27:44.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:27:44.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:27:44.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:27:44.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:27:44.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:27:44.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:27:44.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:27:44.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:27:44.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:27:44.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:27:44.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:27:44.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:27:44.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:27:44.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:27:44.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:27:44.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:27:44.025 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.025 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.025 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.026 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:44.026 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:27:44.026 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 46.487969718000002, 2023-12-06T18:27:44.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:27:44.026 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.026 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.026 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.026 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.026 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.026 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:27:44.026 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.026 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.026 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.026 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.026 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:27:44.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.027 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:27:44.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.027 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:27:44.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:27:44.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.027 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:27:44.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:27:44.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:27:44.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:27:44.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:27:44.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:27:44.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:27:44.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:27:44.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:27:44.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:27:44.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:27:44.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:27:44.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:27:44.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:27:44.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:27:44.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:27:44.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:27:44.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:27:44.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:27:44.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:27:44.030 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.030 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.031 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.031 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:44.031 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:27:44.031 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 46.482396463000001, 2023-12-06T18:27:44.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:27:44.031 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.031 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.031 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.031 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.031 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.031 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:27:44.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.031 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:27:44.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:27:44.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:27:44.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:27:44.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:27:44.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:27:44.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:27:44.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:27:44.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:27:44.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:27:44.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.034 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.034 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:27:44.034 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:27:44.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.034 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.034 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:27:44.034 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:27:44.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.034 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.034 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:27:44.034 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:27:44.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:27:44.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:27:44.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:27:44.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:27:44.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:27:44.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:27:44.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:27:44.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:27:44.036 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.036 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.036 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.036 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:44.036 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:27:44.036 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 46.481722822999998, 2023-12-06T18:27:44.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:27:44.036 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.036 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.036 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.037 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.037 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.037 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:27:44.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.037 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.037 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.037 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:27:44.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.037 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.037 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:27:44.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:27:44.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:27:44.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:27:44.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:27:44.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:27:44.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:27:44.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:27:44.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:27:44.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:27:44.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:27:44.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:27:44.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:27:44.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:27:44.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:27:44.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:27:44.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:27:44.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:27:44.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:27:44.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:27:44.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:27:44.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.041 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.041 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:27:44.041 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:27:44.041 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.041 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.041 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.041 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:44.041 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:27:44.041 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 46.481060624000001, 2023-12-06T18:27:44.041 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:27:44.041 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.041 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.041 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.041 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.041 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.042 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:27:44.042 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.042 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.042 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.042 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.042 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:27:44.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.042 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.042 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.042 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.042 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:27:44.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.042 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.042 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.042 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:27:44.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:27:44.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:27:44.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:27:44.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:27:44.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:27:44.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:27:44.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:27:44.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:27:44.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:27:44.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:27:44.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:27:44.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:27:44.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:27:44.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:27:44.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:27:44.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:27:44.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:27:44.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:27:44.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:27:44.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.046 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.046 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:27:44.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:27:44.046 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.046 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.046 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.046 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:44.046 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:27:44.046 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 46.408351189000001, 2023-12-06T18:27:44.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:27:44.046 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.046 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.046 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.047 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.047 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.047 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:27:44.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.047 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:27:44.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:27:44.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:27:44.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:27:44.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:27:44.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:27:44.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:27:44.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:27:44.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:27:44.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:27:44.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:27:44.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:27:44.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:27:44.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:27:44.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:27:44.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:27:44.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:27:44.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:27:44.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:27:44.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:27:44.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:27:44.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:27:44.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:27:44.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:27:44.050 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.051 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.051 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.051 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:44.051 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:27:44.051 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 46.407669480000003, 2023-12-06T18:27:44.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:27:44.051 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.051 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.051 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.051 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.051 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:27:44.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.051 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:27:44.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:27:44.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:27:44.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:27:44.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:27:44.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:27:44.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:27:44.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:27:44.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:27:44.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:27:44.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:27:44.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:27:44.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.054 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:27:44.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:27:44.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.054 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:27:44.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:27:44.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.054 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:27:44.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:27:44.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.055 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.055 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:27:44.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:27:44.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.055 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.055 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:27:44.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:27:44.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.055 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.055 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:27:44.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:27:44.056 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.056 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.056 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.056 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:27:44.056 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:27:44.056 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 45.328865391999997, 2023-12-06T18:27:44.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:27:44.056 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.056 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.056 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.056 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.056 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.056 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:27:44.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.057 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:27:44.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:27:44.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:27:44.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:27:44.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:27:44.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:27:44.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:27:44.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:27:44.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:27:44.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:27:44.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:27:44.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:27:44.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:27:44.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:27:44.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:27:44.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:27:44.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.060 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:27:44.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:27:44.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.060 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:27:44.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:27:44.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.060 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:27:44.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:27:44.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.061 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.061 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:27:44.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:27:44.061 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.061 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.061 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.061 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:27:44.061 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:27:44.061 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 45.327448818000001, 2023-12-06T18:27:44.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:27:44.061 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.062 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.062 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.062 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.062 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.062 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:27:44.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.062 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:27:44.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:27:44.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:27:44.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:27:44.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:27:44.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:27:44.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:27:44.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:27:44.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:27:44.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:27:44.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:27:44.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:27:44.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:27:44.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:27:44.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:27:44.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:27:44.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:27:44.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:27:44.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:27:44.066 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:27:44.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.066 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.066 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.066 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:27:44.066 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:27:44.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.066 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.066 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.066 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:27:44.066 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:27:44.066 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.066 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.066 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.067 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.067 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.067 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:27:44.067 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:27:44.067 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 45.326363692999998, 2023-12-06T18:27:44.067 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:27:44.067 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.067 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.067 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.067 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.067 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.067 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:27:44.067 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.067 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.067 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:27:44.068 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:27:44.068 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:27:44.068 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:27:44.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:27:44.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:27:44.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:27:44.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:27:44.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:27:44.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:27:44.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:27:44.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:27:44.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.070 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.070 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:27:44.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:27:44.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.070 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.070 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:27:44.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:27:44.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.070 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.070 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:27:44.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:27:44.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.071 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.071 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:27:44.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:27:44.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.071 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.071 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:27:44.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:27:44.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.071 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.071 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:27:44.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:27:44.072 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.072 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.072 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.072 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.072 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:27:44.072 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:27:44.072 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 45.291214590999999, 2023-12-06T18:27:44.072 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:27:44.072 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.072 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.072 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.072 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.072 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.072 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:27:44.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.073 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:27:44.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:27:44.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:27:44.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:27:44.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.074 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:27:44.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:27:44.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.074 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:27:44.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:27:44.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.074 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:27:44.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:27:44.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.075 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:27:44.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:27:44.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.075 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:27:44.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:27:44.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.075 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:27:44.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:27:44.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.076 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:27:44.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:27:44.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.076 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:27:44.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:27:44.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.076 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:27:44.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:27:44.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.077 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.077 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:27:44.077 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:27:44.077 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.077 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.077 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.077 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.077 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.077 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:44.077 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:27:44.077 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 38.741484065999998, 2023-12-06T18:27:44.077 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:27:44.077 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.077 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.077 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.077 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.078 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:27:44.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.078 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.078 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.078 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:27:44.078 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.078 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.078 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:27:44.078 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:27:44.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:27:44.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:27:44.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:27:44.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:27:44.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:27:44.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:27:44.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:27:44.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:27:44.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:27:44.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:27:44.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:27:44.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:27:44.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:27:44.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:27:44.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:27:44.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.081 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:27:44.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:27:44.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.081 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:27:44.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:27:44.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.082 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.082 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:27:44.082 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:27:44.082 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.082 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.082 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.082 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.082 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.082 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:44.082 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:27:44.082 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 38.740678457999998, 2023-12-06T18:27:44.082 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:27:44.082 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.082 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.082 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.083 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.083 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.083 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:27:44.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.083 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.083 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.083 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.083 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:27:44.083 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.083 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.083 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.083 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:27:44.083 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:27:44.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:27:44.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:27:44.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:27:44.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:27:44.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:27:44.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:27:44.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:27:44.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:27:44.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:27:44.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:27:44.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:27:44.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:27:44.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:27:44.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:27:44.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:27:44.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:27:44.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:27:44.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:27:44.087 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:27:44.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.087 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:27:44.087 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:27:44.087 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.089 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.089 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.089 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:44.089 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:27:44.089 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 38.739987649, 2023-12-06T18:27:44.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:27:44.089 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.089 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.090 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.090 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.090 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.090 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:27:44.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.090 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:27:44.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:27:44.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:27:44.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:27:44.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:27:44.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:27:44.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:27:44.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:27:44.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:27:44.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:27:44.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:27:44.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:27:44.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:27:44.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:27:44.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.093 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.093 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:27:44.093 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:27:44.093 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.093 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.093 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.093 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:27:44.093 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:27:44.093 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.093 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.093 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.093 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:27:44.093 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:27:44.093 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.093 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.093 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.094 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:27:44.094 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:27:44.094 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.094 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.094 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.094 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:27:44.094 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:27:44.094 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.094 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.094 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.094 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.094 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.094 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:44.094 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:27:44.095 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 38.735690140999999, 2023-12-06T18:27:44.095 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:27:44.095 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.095 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.095 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.095 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.095 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.095 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:27:44.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.095 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.095 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.095 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.095 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:27:44.095 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.095 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.095 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.096 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:27:44.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.096 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.096 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:27:44.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:27:44.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.096 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.096 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:27:44.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:27:44.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.096 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.096 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:27:44.097 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:27:44.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.097 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.097 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:27:44.097 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:27:44.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.097 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.097 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:27:44.097 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:27:44.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.097 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.097 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:27:44.097 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:27:44.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.098 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.098 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:27:44.098 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:27:44.098 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.098 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.098 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.098 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:27:44.098 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:27:44.098 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.098 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.098 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.098 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:27:44.098 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:27:44.098 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.098 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.099 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.099 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:27:44.099 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:27:44.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.099 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.099 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.099 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:27:44.099 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:27:44.099 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.099 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.099 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.099 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.099 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:44.099 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:27:44.099 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 38.735022465999997, 2023-12-06T18:27:44.099 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:27:44.100 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.100 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.100 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.100 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.100 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.100 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:27:44.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.100 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.100 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:27:44.100 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.100 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:27:44.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:27:44.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:27:44.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:27:44.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:27:44.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:27:44.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:27:44.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:27:44.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:27:44.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:27:44.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:27:44.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:27:44.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:27:44.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.103 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.103 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.103 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:27:44.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:27:44.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.103 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.103 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.103 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:27:44.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:27:44.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.103 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.103 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.103 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:27:44.104 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:27:44.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.104 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.104 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:27:44.104 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:27:44.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.104 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.104 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:27:44.104 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:27:44.104 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.104 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.104 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.105 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:44.105 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:27:44.105 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 38.734367038000002, 2023-12-06T18:27:44.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:27:44.105 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.105 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.105 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.105 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.105 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.105 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:27:44.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.105 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.105 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.105 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:27:44.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.106 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.106 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:27:44.106 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.106 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.106 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:27:44.106 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:27:44.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.106 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.106 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:27:44.106 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:27:44.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.107 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:27:44.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:27:44.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.107 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:27:44.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:27:44.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.108 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.108 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:27:44.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:27:44.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.108 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.108 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:27:44.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:27:44.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.108 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.108 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:27:44.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:27:44.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.109 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.109 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:27:44.109 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:27:44.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.109 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.109 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.109 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:27:44.109 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:27:44.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.109 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.109 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.109 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:27:44.109 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:27:44.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.109 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.110 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.110 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:27:44.110 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:27:44.110 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.110 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.110 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.110 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.110 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.110 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:44.110 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:27:44.110 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 38.693712490999999, 2023-12-06T18:27:44.110 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:27:44.110 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.110 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:44.110 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:44.111 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:44.111 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:44.111 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:27:44.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.111 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:27:44.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:27:44.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:27:44.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:27:44.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:27:44.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:27:44.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:27:44.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:27:44.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:27:44.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:27:44.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:27:44.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:27:44.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:27:44.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:27:44.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:44.114 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:27:44.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:27:44.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:44.114 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:27:44.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:27:44.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:44.114 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:27:44.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:27:44.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:44.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:27:44.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:27:44.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:27:44.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:27:44.115 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.115 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.115 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.115 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.115 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.115 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:44.117 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:27:44.118 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:44.114+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:44.219 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:44.216+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:44.257 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:44.257 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:44.257 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:27:44.257 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:44.267 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:27:44.320 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:44.316+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:44.405 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:44.405 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:44.405 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:27:44.405 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:27:44.405 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:44.415 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:27:44.420 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:44.417+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:44.522 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:44.518+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:44.564 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.0 to 1 (10000) 2023-12-06T18:27:44.583 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-12-06T18:27:44.615 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:44.615 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:27:44.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:27:44.616 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:27:44.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.616 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:27:44.616 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:27:44.616 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 22.798997310000001, 2023-12-06T18:27:44.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:27:44.616 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:44.616 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:27:44.616 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:44.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:44.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:27:44.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:44.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:27:44.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:44.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:44.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:27:44.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:27:44.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:44.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:27:44.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:27:44.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:44.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:27:44.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:27:44.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:44.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:27:44.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:27:44.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:44.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:27:44.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:27:44.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.619 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.619 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:44.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:27:44.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:27:44.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:44.619 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:44.619 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:44.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:27:44.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:27:44.619 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.619 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.619 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.620 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:44.620 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:44.620 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:44.622 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:44.619+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:44.630 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:27:44.723 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:44.720+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:44.824 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:44.824 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:44.824 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:27:44.824 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:44.824 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:44.821+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:44.833 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:27:44.925 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:44.922+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:45.012 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:45.012 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:45.012 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:27:45.012 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:27:45.012 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:45.016 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:27:45.022 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:27:45.026 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:45.023+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:45.127 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:45.124+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:45.217 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:45.217 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:27:45.217 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:27:45.217 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:27:45.217 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:45.226 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:27:45.228 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:45.224+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:45.276 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:27:45.277 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:27:45.329 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:45.325+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:45.355 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:45.355 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:27:45.355 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:27:45.355 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:45.364 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:27:45.372 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:27:45.372 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:27:45.430 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:45.426+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:45.470 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.1 to 1 (10000) 2023-12-06T18:27:45.472 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:27:45.472 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:27:45.486 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-12-06T18:27:45.505 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-12-06T18:27:45.522 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:45.522 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:27:45.522 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:27:45.522 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:27:45.522 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:45.530 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:45.526+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:45.532 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:27:45.592 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:27:45.593 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:27:45.637 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:45.633+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:45.693 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:27:45.693 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:27:45.733 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:45.729+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:45.736 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:45.736 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:45.744 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:27:45.790 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:27:45.790 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:27:45.833 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:45.829+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:45.877 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:45.877 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:45.882 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:27:45.883 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:27:45.887 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:27:45.972 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:45.968+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:45.979 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:27:45.980 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:27:46.039 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:46.035+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:46.064 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:46.065 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:46.073 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:27:46.135 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:46.132+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:46.236 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:46.233+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:46.253 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:46.253 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:46.263 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:27:46.347 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:46.344+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:46.446 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:46.441+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:46.451 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:46.451 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:46.460 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:27:46.540 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:46.537+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:46.633 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:46.633 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:46.634 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:27:46.643 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:27:46.678 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:46.674+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:46.688 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.2 to 1 (10000) 2023-12-06T18:27:46.703 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-12-06T18:27:46.746 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:46.741+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:46.765 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is unset 2023-12-06T18:27:46.779 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:46.779 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:46.789 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:27:46.844 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:46.840+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:46.919 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:46.919 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:46.929 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:27:46.971 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:46.967+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:47.046 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:47.042+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:47.102 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:47.102 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:47.112 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:27:47.153 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:47.150+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:47.248 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:47.245+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:47.282 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:47.282 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:47.291 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:27:47.349 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:47.346+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:47.381 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":14,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":5,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:27:47.448 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:47.448 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:47.450 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:47.446+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:47.458 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:27:47.550 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:47.547+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:47.595 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:47.595 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:47.604 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:27:47.651 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:47.648+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:47.744 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:47.745 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:47.752 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:47.748+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:47.754 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:27:47.768 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.3 to 1 (10000) 2023-12-06T18:27:47.783 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2023-12-06T18:27:47.854 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:47.850+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:47.908 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:47.908 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:47.918 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:27:47.954 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:47.951+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:47.997 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:27:48.056 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:48.052+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:48.059 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:48.059 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:48.068 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:27:48.156 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:48.153+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:48.252 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:48.253 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:48.257 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:48.254+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:48.262 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:27:48.272 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:27:48.272 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:27:48.358 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:48.354+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:48.396 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:27:48.397 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:27:48.443 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:48.443 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:48.456 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:27:48.459 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:48.456+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:48.504 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:27:48.504 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:27:48.560 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:48.556+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:48.618 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:27:48.619 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:27:48.636 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:48.637 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:48.637 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:27:48.637 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:27:48.637 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:48.646 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:27:48.660 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:48.657+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:48.705 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:27:48.706 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:27:48.762 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:48.758+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:48.779 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-12-06T18:27:48.795 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:27:48.795 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:27:48.830 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:27:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:27:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:27:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:27:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:27:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 27.013877907000001, 2023-12-06T18:27:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:27:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:27:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:27:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:27:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:27:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:27:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:27:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:27:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:27:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:27:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:27:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:27:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:27:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:27:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:27:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:27:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:27:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:27:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:48.834 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:48.839 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:27:48.863 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:48.860+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:48.898 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:27:48.898 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:27:48.964 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:48.961+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:49.012 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:27:49.012 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:27:49.023 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:49.023 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:49.023 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:27:49.023 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:49.033 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:27:49.076 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:49.072+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:49.166 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:49.162+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:49.227 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:49.227 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:49.227 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:27:49.227 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:27:49.227 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:49.239 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:27:49.266 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:49.263+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:49.372 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:49.368+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:49.372 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.4 to 1 (10000) 2023-12-06T18:27:49.392 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2023-12-06T18:27:49.434 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:49.435 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:27:49.435 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:27:49.435 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:27:49.435 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:49.445 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:27:49.490 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:49.486+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:49.573 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:49.569+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:49.590 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:49.590 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:27:49.590 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:27:49.590 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:49.601 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:27:49.685 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:49.681+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:49.769 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:49.769 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:27:49.769 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:27:49.769 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:27:49.769 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:49.770 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:49.767+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:49.778 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:27:49.871 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:49.867+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:49.954 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:49.954 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:27:49.954 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:27:49.954 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:27:49.954 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:49.963 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:27:49.971 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:49.968+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:50.072 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:50.069+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:50.097 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:50.098 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:27:50.098 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:27:50.098 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:50.107 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:27:50.173 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:50.170+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:50.243 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:50.243 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:27:50.243 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:27:50.243 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:27:50.243 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:50.254 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:27:50.274 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:50.270+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:50.375 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:50.371+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:50.396 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:27:50.459 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is unset 2023-12-06T18:27:50.463 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.6 to 1 (10000) 2023-12-06T18:27:50.475 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:50.471+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:50.478 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2023-12-06T18:27:50.480 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:50.481 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:27:50.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:27:50.481 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:27:50.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.481 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:27:50.481 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:27:50.481 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 53.849259547999999, 2023-12-06T18:27:50.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:27:50.481 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.481 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.481 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:27:50.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:27:50.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:27:50.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:27:50.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:27:50.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:27:50.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:27:50.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:27:50.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:27:50.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:27:50.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:27:50.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:27:50.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:27:50.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:27:50.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:27:50.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:27:50.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:27:50.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:27:50.485 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.486 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.486 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.486 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:27:50.486 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:27:50.486 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 52.826432539000002, 2023-12-06T18:27:50.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:27:50.486 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.486 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.486 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.487 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:27:50.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.487 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.487 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:27:50.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.487 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.487 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:27:50.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:27:50.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:27:50.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:27:50.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:27:50.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:27:50.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:27:50.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:27:50.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:27:50.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:27:50.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:27:50.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:27:50.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:27:50.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:27:50.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:27:50.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:27:50.490 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.490 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.490 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.490 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:27:50.490 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:27:50.491 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 52.814385901000001, 2023-12-06T18:27:50.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:27:50.491 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.491 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.491 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:27:50.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:27:50.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:27:50.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:27:50.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:27:50.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:27:50.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.495 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:27:50.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:27:50.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.495 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:27:50.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:27:50.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:27:50.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:27:50.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:27:50.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:27:50.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:27:50.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:27:50.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:27:50.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:27:50.498 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.498 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.499 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.499 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:27:50.499 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:27:50.499 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 52.792985293999998, 2023-12-06T18:27:50.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:27:50.500 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.500 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.500 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:27:50.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:27:50.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:27:50.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:27:50.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:27:50.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:27:50.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:27:50.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:27:50.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:27:50.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:27:50.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:27:50.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:27:50.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:27:50.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:27:50.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:27:50.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:27:50.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:27:50.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:27:50.503 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.503 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.503 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.504 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:27:50.504 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:27:50.504 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 52.791805971000002, 2023-12-06T18:27:50.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:27:50.504 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.504 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.504 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:27:50.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:27:50.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:27:50.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:27:50.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:27:50.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:27:50.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:27:50.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:27:50.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:27:50.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:27:50.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:27:50.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:27:50.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:27:50.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:27:50.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:27:50.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:27:50.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:27:50.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:27:50.508 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.508 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.508 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.508 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:27:50.508 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:27:50.508 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 52.791233990000002, 2023-12-06T18:27:50.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:27:50.509 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.509 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.509 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:27:50.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:27:50.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:27:50.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:27:50.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:27:50.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:27:50.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:27:50.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:27:50.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:27:50.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:27:50.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:27:50.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:27:50.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:27:50.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:27:50.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:27:50.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:27:50.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:27:50.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:27:50.512 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.512 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.512 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.512 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:27:50.512 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:27:50.512 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 52.345398520000003, 2023-12-06T18:27:50.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:27:50.513 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.513 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.513 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:27:50.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:27:50.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:27:50.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:27:50.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:27:50.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:27:50.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:27:50.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:27:50.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:27:50.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:27:50.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:27:50.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:27:50.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:27:50.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:27:50.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:27:50.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:27:50.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:27:50.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:27:50.516 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.516 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.517 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.517 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:27:50.517 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:27:50.517 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 52.343798663999998, 2023-12-06T18:27:50.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:27:50.517 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.517 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.517 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:27:50.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:27:50.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:27:50.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:27:50.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:27:50.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:27:50.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:27:50.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:27:50.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:27:50.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:27:50.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:27:50.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:27:50.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:27:50.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:27:50.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:27:50.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:27:50.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:27:50.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:27:50.521 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.521 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.521 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.522 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:27:50.522 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:27:50.522 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 51.964593047999998, 2023-12-06T18:27:50.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:27:50.522 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.522 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.522 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.522 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.522 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:27:50.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:27:50.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:27:50.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:27:50.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:27:50.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:27:50.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:27:50.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:27:50.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:27:50.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:27:50.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:27:50.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:27:50.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:27:50.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:27:50.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:27:50.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:27:50.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.526 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:27:50.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:27:50.526 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.526 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.526 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.526 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:27:50.527 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:27:50.527 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 51.727090865999998, 2023-12-06T18:27:50.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:27:50.527 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.527 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.527 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:27:50.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:27:50.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:27:50.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:27:50.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:27:50.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:27:50.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:27:50.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:27:50.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.530 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:27:50.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:27:50.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.530 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:27:50.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:27:50.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:27:50.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:27:50.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:27:50.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:27:50.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:27:50.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:27:50.532 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.532 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.532 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.532 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:27:50.532 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:27:50.532 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 51.726362866999999, 2023-12-06T18:27:50.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:27:50.532 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.532 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.532 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.533 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.533 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:27:50.533 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.533 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.533 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:27:50.533 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.533 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.533 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:27:50.533 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:27:50.534 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.534 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.534 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.534 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:27:50.534 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:27:50.534 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.534 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.534 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.534 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:27:50.535 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:27:50.535 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.535 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.535 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.535 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:27:50.535 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:27:50.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:27:50.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:27:50.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:27:50.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:27:50.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.537 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:27:50.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:27:50.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.538 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.538 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.538 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:27:50.538 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:27:50.538 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.538 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.538 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.538 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.539 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:27:50.539 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:27:50.539 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 51.721350457, 2023-12-06T18:27:50.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:27:50.539 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.539 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.540 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:27:50.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:27:50.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:27:50.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:27:50.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:27:50.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:27:50.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:27:50.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:27:50.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:27:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:27:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:27:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:27:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:27:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:27:50.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.543 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.543 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:27:50.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:27:50.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.543 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.543 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:27:50.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:27:50.543 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.543 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.544 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.544 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.544 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:27:50.544 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:27:50.544 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 51.715823378000003, 2023-12-06T18:27:50.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:27:50.545 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.545 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.545 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.545 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:27:50.546 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.546 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.546 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.546 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.546 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:27:50.546 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:27:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:27:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:27:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:27:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:27:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:27:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:27:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:27:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:27:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:27:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:27:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:27:50.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:27:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:27:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:27:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:27:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.551 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:27:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:27:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 51.715312541000003, 2023-12-06T18:27:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:27:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:27:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.552 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:27:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:27:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:27:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:27:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:27:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:27:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:27:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:27:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:27:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:27:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:27:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.554 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:27:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:27:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:27:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:27:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:27:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:27:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.555 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:27:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:27:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 51.71485328, 2023-12-06T18:27:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:27:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:27:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:27:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:27:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:27:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:27:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:27:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:27:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:27:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.558 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:27:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:27:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:27:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:27:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:27:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:27:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.559 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:27:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:27:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:27:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:27:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:27:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:27:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 45.134937473000001, 2023-12-06T18:27:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:27:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:27:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:27:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:27:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:27:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:27:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:27:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.562 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:27:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:27:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:27:50.563 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:27:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:27:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:27:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.564 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:27:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:27:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:27:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:27:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:27:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:27:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.566 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.567 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:27:50.567 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:27:50.567 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 45.128699668000003, 2023-12-06T18:27:50.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:27:50.567 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.568 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.568 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.568 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.568 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:27:50.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:27:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:27:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:27:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:27:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:27:50.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:27:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:27:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:27:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:27:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:27:50.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:27:50.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.573 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.573 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.573 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:27:50.573 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:27:50.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:27:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:27:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:27:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:27:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.574 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:27:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:27:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 45.122636589000003, 2023-12-06T18:27:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:27:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:27:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:27:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:27:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:27:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:27:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:27:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:27:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:27:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:27:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:27:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.577 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:27:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:27:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:27:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:27:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:27:50.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:27:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:27:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:27:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:27:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:27:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 44.985249504000002, 2023-12-06T18:27:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:27:50.579 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:27:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:27:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:27:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:27:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:27:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:27:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:27:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:27:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:27:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:27:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:27:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:27:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:27:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:27:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:27:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:27:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:27:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:27:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:27:50.583 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:27:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 44.978578777000003, 2023-12-06T18:27:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:27:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:27:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:27:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:27:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:27:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.584 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:27:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:27:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:27:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:27:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:27:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:27:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:27:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:27:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:27:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:27:50.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:27:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:27:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:27:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:27:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:27:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:27:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:27:50.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:27:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:27:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:27:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:27:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:27:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:27:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:27:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:27:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:27:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:27:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:27:50.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:27:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:27:50.588 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:27:50.588 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:50.590 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:27:50.591 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:50.586+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:50.690 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:50.687+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:50.764 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:50.764 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:27:50.764 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:27:50.764 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:50.774 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:27:50.818 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:50.814+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:50.893 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:50.889+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:50.920 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:50.920 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:27:50.920 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:27:50.921 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:27:50.921 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:50.930 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:27:50.993 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:50.990+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:51.026 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:27:51.094 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:51.091+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:51.102 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:51.102 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:27:51.102 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:27:51.102 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:27:51.103 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:51.103 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":14,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":9,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:27:51.112 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:27:51.198 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:51.194+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:51.287 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:51.288 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:27:51.288 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:27:51.288 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:51.297 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:27:51.299 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:51.295+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:51.306 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:27:51.306 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:27:51.397 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:51.393+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:51.409 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:27:51.409 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:27:51.433 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:51.433 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:27:51.433 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:27:51.433 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:27:51.434 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:51.443 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:27:51.498 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:51.494+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:51.502 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:27:51.502 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:27:51.586 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:51.586 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:27:51.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:27:51.586 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:27:51.586 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:51.598 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:27:51.598 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:51.594+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:51.638 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.7 to 1 (10000) 2023-12-06T18:27:51.652 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-12-06T18:27:51.652 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:27:51.653 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:27:51.653 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:27:51.699 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:51.695+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:51.743 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:27:51.743 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:27:51.747 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:51.747 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:51.757 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:27:51.801 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:51.797+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:51.833 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:08.877 95127== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:51.895 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:51.895 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:51.901 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:51.897+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:51.905 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:27:52.002 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:51.998+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:52.035 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:27:52.035 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:27:52.076 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:09.120 95127== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:52.087 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:52.087 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:52.097 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:27:52.102 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:52.099+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:52.126 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:27:52.127 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:27:52.203 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:52.200+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:52.215 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:27:52.216 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:27:52.235 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:27:52.235 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:27:52.251 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2023-12-06T18:27:52.224809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":191736,"kb_used_data":2576,"kb_used_omap":10,"kb_used_meta":189045,"kb_avail":660411144,"statfs":{"total":676457349120,"available":676261011456,"internally_reserved":0,"allocated":2637824,"data_stored":1511710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11157,"internal_metadata":193582187},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":49,"apply_latency_ms":49,"commit_latency_ns":49000000,"apply_latency_ns":49000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.448878"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-06T18:27:51.309590+0000","last_change":"2023-12-06T18:27:04.849231+0000","last_active":"2023-12-06T18:27:51.309590+0000","last_peered":"2023-12-06T18:27:51.309590+0000","last_clean":"2023-12-06T18:27:51.309590+0000","last_became_active":"2023-12-06T18:27:04.837862+0000","last_became_peered":"2023-12-06T18:27:04.837862+0000","last_unstale":"2023-12-06T18:27:51.309590+0000","last_undegraded":"2023-12-06T18:27:51.309590+0000","last_fullsized":"2023-12-06T18:27:51.309590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:03:55.366586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-12-06T18:27:46.448276+0000","last_change":"2023-12-06T18:27:04.883840+0000","last_active":"2023-12-06T18:27:46.448276+0000","last_peered":"2023-12-06T18:27:46.448276+0000","last_clean":"2023-12-06T18:27:46.448276+0000","last_became_active":"2023-12-06T18:27:04.858001+0000","last_became_peered":"2023-12-06T18:27:04.858001+0000","last_unstale":"2023-12-06T18:27:46.448276+0000","last_undegraded":"2023-12-06T18:27:46.448276+0000","last_fullsized":"2023-12-06T18:27:46.448276+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:44:02.916244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-06T18:27:51.312346+0000","last_change":"2023-12-06T18:27:04.336778+0000","last_active":"2023-12-06T18:27:51.312346+0000","last_peered":"2023-12-06T18:27:51.312346+0000","last_clean":"2023-12-06T18:27:51.312346+0000","last_became_active":"2023-12-06T18:27:04.321269+0000","last_became_peered":"2023-12-06T18:27:04.321269+0000","last_unstale":"2023-12-06T18:27:51.312346+0000","last_undegraded":"2023-12-06T18:27:51.312346+0000","last_fullsized":"2023-12-06T18:27:51.312346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T22:25:15.703579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-12-06T18:27:45.495843+0000","last_change":"2023-12-06T18:27:21.292214+0000","last_active":"2023-12-06T18:27:45.495843+0000","last_peered":"2023-12-06T18:27:45.495843+0000","last_clean":"2023-12-06T18:27:45.495843+0000","last_became_active":"2023-12-06T18:27:21.258857+0000","last_became_peered":"2023-12-06T18:27:21.258857+0000","last_unstale":"2023-12-06T18:27:45.495843+0000","last_undegraded":"2023-12-06T18:27:45.495843+0000","last_fullsized":"2023-12-06T18:27:45.495843+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:06:23.175627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-06T18:27:51.300529+0000","last_change":"2023-12-06T18:27:04.408081+0000","last_active":"2023-12-06T18:27:51.300529+0000","last_peered":"2023-12-06T18:27:51.300529+0000","last_clean":"2023-12-06T18:27:51.300529+0000","last_became_active":"2023-12-06T18:27:04.399765+0000","last_became_peered":"2023-12-06T18:27:04.399765+0000","last_unstale":"2023-12-06T18:27:51.300529+0000","last_undegraded":"2023-12-06T18:27:51.300529+0000","last_fullsized":"2023-12-06T18:27:51.300529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:40:33.570114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-12-06T18:27:46.442061+0000","last_change":"2023-12-06T18:27:21.214629+0000","last_active":"2023-12-06T18:27:46.442061+0000","last_peered":"2023-12-06T18:27:46.442061+0000","last_clean":"2023-12-06T18:27:46.442061+0000","last_became_active":"2023-12-06T18:27:21.200403+0000","last_became_peered":"2023-12-06T18:27:21.200403+0000","last_unstale":"2023-12-06T18:27:46.442061+0000","last_undegraded":"2023-12-06T18:27:46.442061+0000","last_fullsized":"2023-12-06T18:27:46.442061+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:55:38.273587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":28,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-06T18:27:51.298596+0000","last_change":"2023-12-06T18:27:21.099425+0000","last_active":"2023-12-06T18:27:51.298596+0000","last_peered":"2023-12-06T18:27:51.298596+0000","last_clean":"2023-12-06T18:27:51.298596+0000","last_became_active":"2023-12-06T18:27:21.091574+0000","last_became_peered":"2023-12-06T18:27:21.091574+0000","last_unstale":"2023-12-06T18:27:51.298596+0000","last_undegraded":"2023-12-06T18:27:51.298596+0000","last_fullsized":"2023-12-06T18:27:51.298596+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T22:44:22.719325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-06T18:27:51.306046+0000","last_change":"2023-12-06T18:27:04.609381+0000","last_active":"2023-12-06T18:27:51.306046+0000","last_peered":"2023-12-06T18:27:51.306046+0000","last_clean":"2023-12-06T18:27:51.306046+0000","last_became_active":"2023-12-06T18:27:04.598701+0000","last_became_peered":"2023-12-06T18:27:04.598701+0000","last_unstale":"2023-12-06T18:27:51.306046+0000","last_undegraded":"2023-12-06T18:27:51.306046+0000","last_fullsized":"2023-12-06T18:27:51.306046+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:29:58.962692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":140,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-12-06T18:27:48.337851+0000","last_change":"2023-12-06T18:27:21.895470+0000","last_active":"2023-12-06T18:27:48.337851+0000","last_peered":"2023-12-06T18:27:48.337851+0000","last_clean":"2023-12-06T18:27:48.337851+0000","last_became_active":"2023-12-06T18:27:21.241270+0000","last_became_peered":"2023-12-06T18:27:21.241270+0000","last_unstale":"2023-12-06T18:27:48.337851+0000","last_undegraded":"2023-12-06T18:27:48.337851+0000","last_fullsized":"2023-12-06T18:27:48.337851+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T19:49:17.808005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1204224,"data_stored":1193520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411342,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":110886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.1319999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.9960000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":20.544}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.6680000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.888000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.968999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.393999999999998}]}]},{"osd":2,"up_from":16,"seq":68719476749,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":500130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.431}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.1720000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.504}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.5099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.603999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.236000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.912000000000001}]}]},{"osd":1,"up_from":16,"seq":68719476750,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":504428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.9269999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.914999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":34.057000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.894}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.882}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.375999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.789999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476751,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":102290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.036000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.026}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.257}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.6449999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.381}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.405000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":20.084}]}]},{"osd":5,"up_from":0,"seq":64424509455,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":102290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.864000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.179}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.845000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.715}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.4239999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.143999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.989999999999998}]}]},{"osd":7,"up_from":14,"seq":60129542157,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":97992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.473000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":41.316000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.266}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":27.245999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":27.927}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":28.890999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":33.777999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542157,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":93694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2759999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.905000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.938000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.336}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.106000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":25.588000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.106}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:27:52.253 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:27:52.281 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:52.281 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:52.291 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:27:52.333 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:52.328+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:52.406 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:52.402+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:52.476 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:52.476 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:52.482 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-12-06T18:27:52.485 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:27:52.527 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:52.523+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:52.608 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:52.604+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:52.677 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:52.678 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:52.687 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:27:52.709 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:52.706+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:52.810 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:52.806+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:52.861 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:27:52.861 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:27:52.869 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:52.869 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:52.876 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2023-12-06T18:27:52.224809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":191736,"kb_used_data":2576,"kb_used_omap":10,"kb_used_meta":189045,"kb_avail":660411144,"statfs":{"total":676457349120,"available":676261011456,"internally_reserved":0,"allocated":2637824,"data_stored":1511710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":11157,"internal_metadata":193582187},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":49,"apply_latency_ms":49,"commit_latency_ns":49000000,"apply_latency_ns":49000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.448878"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-06T18:27:51.309590+0000","last_change":"2023-12-06T18:27:04.849231+0000","last_active":"2023-12-06T18:27:51.309590+0000","last_peered":"2023-12-06T18:27:51.309590+0000","last_clean":"2023-12-06T18:27:51.309590+0000","last_became_active":"2023-12-06T18:27:04.837862+0000","last_became_peered":"2023-12-06T18:27:04.837862+0000","last_unstale":"2023-12-06T18:27:51.309590+0000","last_undegraded":"2023-12-06T18:27:51.309590+0000","last_fullsized":"2023-12-06T18:27:51.309590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:03:55.366586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-12-06T18:27:46.448276+0000","last_change":"2023-12-06T18:27:04.883840+0000","last_active":"2023-12-06T18:27:46.448276+0000","last_peered":"2023-12-06T18:27:46.448276+0000","last_clean":"2023-12-06T18:27:46.448276+0000","last_became_active":"2023-12-06T18:27:04.858001+0000","last_became_peered":"2023-12-06T18:27:04.858001+0000","last_unstale":"2023-12-06T18:27:46.448276+0000","last_undegraded":"2023-12-06T18:27:46.448276+0000","last_fullsized":"2023-12-06T18:27:46.448276+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:44:02.916244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-06T18:27:51.312346+0000","last_change":"2023-12-06T18:27:04.336778+0000","last_active":"2023-12-06T18:27:51.312346+0000","last_peered":"2023-12-06T18:27:51.312346+0000","last_clean":"2023-12-06T18:27:51.312346+0000","last_became_active":"2023-12-06T18:27:04.321269+0000","last_became_peered":"2023-12-06T18:27:04.321269+0000","last_unstale":"2023-12-06T18:27:51.312346+0000","last_undegraded":"2023-12-06T18:27:51.312346+0000","last_fullsized":"2023-12-06T18:27:51.312346+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T22:25:15.703579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-12-06T18:27:45.495843+0000","last_change":"2023-12-06T18:27:21.292214+0000","last_active":"2023-12-06T18:27:45.495843+0000","last_peered":"2023-12-06T18:27:45.495843+0000","last_clean":"2023-12-06T18:27:45.495843+0000","last_became_active":"2023-12-06T18:27:21.258857+0000","last_became_peered":"2023-12-06T18:27:21.258857+0000","last_unstale":"2023-12-06T18:27:45.495843+0000","last_undegraded":"2023-12-06T18:27:45.495843+0000","last_fullsized":"2023-12-06T18:27:45.495843+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:06:23.175627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-06T18:27:51.300529+0000","last_change":"2023-12-06T18:27:04.408081+0000","last_active":"2023-12-06T18:27:51.300529+0000","last_peered":"2023-12-06T18:27:51.300529+0000","last_clean":"2023-12-06T18:27:51.300529+0000","last_became_active":"2023-12-06T18:27:04.399765+0000","last_became_peered":"2023-12-06T18:27:04.399765+0000","last_unstale":"2023-12-06T18:27:51.300529+0000","last_undegraded":"2023-12-06T18:27:51.300529+0000","last_fullsized":"2023-12-06T18:27:51.300529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:40:33.570114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-12-06T18:27:46.442061+0000","last_change":"2023-12-06T18:27:21.214629+0000","last_active":"2023-12-06T18:27:46.442061+0000","last_peered":"2023-12-06T18:27:46.442061+0000","last_clean":"2023-12-06T18:27:46.442061+0000","last_became_active":"2023-12-06T18:27:21.200403+0000","last_became_peered":"2023-12-06T18:27:21.200403+0000","last_unstale":"2023-12-06T18:27:46.442061+0000","last_undegraded":"2023-12-06T18:27:46.442061+0000","last_fullsized":"2023-12-06T18:27:46.442061+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:55:38.273587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":28,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-06T18:27:51.298596+0000","last_change":"2023-12-06T18:27:21.099425+0000","last_active":"2023-12-06T18:27:51.298596+0000","last_peered":"2023-12-06T18:27:51.298596+0000","last_clean":"2023-12-06T18:27:51.298596+0000","last_became_active":"2023-12-06T18:27:21.091574+0000","last_became_peered":"2023-12-06T18:27:21.091574+0000","last_unstale":"2023-12-06T18:27:51.298596+0000","last_undegraded":"2023-12-06T18:27:51.298596+0000","last_fullsized":"2023-12-06T18:27:51.298596+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T22:44:22.719325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-06T18:27:51.306046+0000","last_change":"2023-12-06T18:27:04.609381+0000","last_active":"2023-12-06T18:27:51.306046+0000","last_peered":"2023-12-06T18:27:51.306046+0000","last_clean":"2023-12-06T18:27:51.306046+0000","last_became_active":"2023-12-06T18:27:04.598701+0000","last_became_peered":"2023-12-06T18:27:04.598701+0000","last_unstale":"2023-12-06T18:27:51.306046+0000","last_undegraded":"2023-12-06T18:27:51.306046+0000","last_fullsized":"2023-12-06T18:27:51.306046+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:29:58.962692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":140,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-12-06T18:27:48.337851+0000","last_change":"2023-12-06T18:27:21.895470+0000","last_active":"2023-12-06T18:27:48.337851+0000","last_peered":"2023-12-06T18:27:48.337851+0000","last_clean":"2023-12-06T18:27:48.337851+0000","last_became_active":"2023-12-06T18:27:21.241270+0000","last_became_peered":"2023-12-06T18:27:21.241270+0000","last_unstale":"2023-12-06T18:27:48.337851+0000","last_undegraded":"2023-12-06T18:27:48.337851+0000","last_fullsized":"2023-12-06T18:27:48.337851+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T19:49:17.808005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":397840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1204224,"data_stored":1193520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411342,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":110886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.1319999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.9960000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":20.544}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.6680000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.888000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.968999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.393999999999998}]}]},{"osd":2,"up_from":16,"seq":68719476749,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":663552,"data_stored":500130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.431}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.1720000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.504}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.5099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.603999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":18.236000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.912000000000001}]}]},{"osd":1,"up_from":16,"seq":68719476750,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":504428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.9269999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.914999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":34.057000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.894}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.882}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.375999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.789999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476751,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":102290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":23.036000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.026}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.257}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.6449999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.381}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":17.405000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":20.084}]}]},{"osd":5,"up_from":0,"seq":64424509455,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":102290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.864000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.179}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.845000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.715}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.4239999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.143999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.989999999999998}]}]},{"osd":7,"up_from":14,"seq":60129542157,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":97992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.473000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":41.316000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.266}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":27.245999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":27.927}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":28.890999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":33.777999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542157,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":93694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2759999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.905000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.938000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.336}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":21.106000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":25.588000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.106}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:27:52.877 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-12-06T18:27:52.877 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:27:52.877 INFO:tasks.thrashosds.thrasher:primary_affinity 2023-12-06T18:27:52.877 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.000000 2023-12-06T18:27:52.878 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 0 2023-12-06T18:27:52.881 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:27:52.924 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:52.920+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:53.012 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:53.008+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:53.088 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:53.088 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:53.099 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:27:53.112 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:53.109+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:53.214 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:53.210+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:53.306 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:53.307 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:53.314 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:53.311+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:53.318 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:27:53.415 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:53.411+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:53.506 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:02:04.827 86580== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:53.510 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:02:04.831 86580== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:53.516 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:53.512+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:53.522 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:53.522 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:53.531 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:27:53.556 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:02:04.876 86580== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:53.560 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:02:04.899 86953== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:53.573 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:02:04.912 86953== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:53.617 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:53.614+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:53.643 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:02:04.982 86953== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:53.718 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:53.714+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:53.725 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:53.725 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:53.734 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:27:53.819 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:53.816+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:53.913 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:53.913 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:53.920 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:53.916+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:53.923 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:27:54.021 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:54.018+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:54.056 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:54.056 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:54.065 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:27:54.122 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:27:54.150 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:54.146+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:54.223 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:54.220+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:54.232 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:27:54.236 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:54.236 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:54.245 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:27:54.324 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:54.321+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:54.419 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:54.419 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:54.426 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:54.422+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:54.428 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:27:54.526 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:54.522+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:54.603 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:54.603 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:54.612 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:27:54.626 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:54.623+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:54.716 INFO:teuthology.orchestra.run.smithi061.stderr:set osd.6 primary-affinity to 0 (802) 2023-12-06T18:27:54.718 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is set 2023-12-06T18:27:54.727 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:54.723+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:54.743 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:54.744 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:54.754 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:27:54.819 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:27:54.819 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:27:54.838 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:54.835+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:54.849 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":14,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:27:54.913 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:27:54.914 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:27:54.922 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:54.922 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:54.931 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:27:54.954 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:54.950+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:55.018 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:27:55.019 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:27:55.029 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:55.026+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:55.101 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:55.102 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:55.111 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:27:55.124 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:27:55.124 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:27:55.153 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:55.149+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:55.216 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:27:55.216 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:27:55.230 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:55.227+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:55.284 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:55.284 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:55.293 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:27:55.304 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:27:55.304 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:27:55.331 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:55.327+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:55.396 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:27:55.396 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:27:55.432 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:55.433 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:27:55.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:27:55.433 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:27:55.433 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:55.433 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:55.429+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:55.442 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:27:55.486 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:27:55.486 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:27:55.533 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:55.529+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:55.534 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:12.578 95127== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:55.592 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:12.636 95127== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:55.612 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:55.612 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:27:55.612 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:27:55.612 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:55.622 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:27:55.662 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:55.659+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:55.734 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:55.731+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:55.793 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:55.793 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:27:55.793 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:27:55.793 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:27:55.793 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:55.801 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:27:55.845 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:55.841+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:55.849 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:12.893 95127== Warning: unimplemented fcntl command: 1036 2023-12-06T18:27:55.932 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:27:55.932 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:27:55.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:27:55.932 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:27:55.932 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:27:55.936 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:55.932+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:55.941 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:27:56.038 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:56.035+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:56.087 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:56.087 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:56.087 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:27:56.088 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:56.098 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:27:56.139 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:56.136+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:56.240 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:56.236+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:56.240 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:56.240 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:56.240 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:27:56.240 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:27:56.240 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:56.251 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:27:56.341 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:56.337+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:56.399 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:56.400 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:27:56.400 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:27:56.400 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:27:56.400 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:56.409 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:27:56.451 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:56.447+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:56.542 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:56.538+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:56.595 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:56.595 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:56.596 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:27:56.596 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:56.604 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:27:56.643 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:56.640+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:56.740 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-12-06T18:27:56.743 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:56.743 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:56.743 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:27:56.743 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:27:56.743 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:56.753 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:27:56.754 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:56.750+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:56.845 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:56.842+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:56.946 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:56.944+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:57.014 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:57.014 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:27:57.014 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:27:57.014 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:27:57.014 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.014 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:27:57.014 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:27:57.015 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 61.600173447000003, 2023-12-06T18:27:57.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:27:57.015 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.015 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.015 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.015 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.015 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.015 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:27:57.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.015 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:27:57.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:27:57.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:27:57.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:27:57.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:27:57.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:27:57.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:27:57.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:27:57.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:27:57.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:27:57.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:27:57.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:27:57.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:27:57.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:27:57.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:27:57.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:27:57.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:27:57.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:27:57.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:27:57.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:27:57.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:27:57.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:27:57.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:27:57.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:27:57.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:27:57.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:27:57.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:27:57.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:27:57.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:27:57.020 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.020 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.020 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.021 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.021 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:57.021 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:27:57.021 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 59.491600208000001, 2023-12-06T18:27:57.021 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:27:57.021 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.021 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.021 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.021 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.021 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.021 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:27:57.021 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.021 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.021 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.021 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:27:57.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:27:57.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:27:57.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:27:57.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:27:57.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:27:57.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:27:57.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:27:57.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:27:57.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:27:57.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:27:57.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:27:57.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:27:57.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:27:57.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:27:57.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:27:57.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:27:57.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:27:57.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:27:57.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:27:57.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:27:57.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:27:57.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:27:57.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:27:57.026 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.026 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.026 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.026 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.026 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.026 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:57.026 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:27:57.026 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 59.490854976000001, 2023-12-06T18:27:57.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:27:57.026 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.026 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.026 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.026 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.027 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.027 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:27:57.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.027 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.027 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:27:57.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.027 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:27:57.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:27:57.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:27:57.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:27:57.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:27:57.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:27:57.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:27:57.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:27:57.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:27:57.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:27:57.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:27:57.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:27:57.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:27:57.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:27:57.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:27:57.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:27:57.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:27:57.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:27:57.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:27:57.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:27:57.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:27:57.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:27:57.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:27:57.031 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.031 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.031 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.031 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:57.031 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:27:57.032 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 59.490171220999997, 2023-12-06T18:27:57.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:27:57.032 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.032 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.032 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.032 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.032 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.032 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:27:57.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.032 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:27:57.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:27:57.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:27:57.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:27:57.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:27:57.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:27:57.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.034 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.034 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:27:57.034 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:27:57.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.034 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.034 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:27:57.034 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:27:57.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.034 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.034 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:27:57.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:27:57.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:27:57.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:27:57.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:27:57.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:27:57.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:27:57.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:27:57.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:27:57.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:27:57.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:27:57.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:27:57.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:27:57.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:27:57.037 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.037 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.037 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.037 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:57.037 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:27:57.037 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 59.484597966000003, 2023-12-06T18:27:57.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:27:57.037 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.037 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.037 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.037 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.038 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.038 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:27:57.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.038 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:27:57.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:27:57.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:27:57.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:27:57.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:27:57.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:27:57.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:27:57.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:27:57.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:27:57.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:27:57.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:27:57.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:27:57.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:27:57.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:27:57.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.041 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.041 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:27:57.041 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:27:57.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.041 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.041 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:27:57.041 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:27:57.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.041 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.041 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:27:57.041 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:27:57.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.042 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.042 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:27:57.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:27:57.042 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.042 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.042 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.042 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:27:57.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:27:57.042 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.042 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.042 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.042 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.042 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.042 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:57.042 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:27:57.043 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 59.483924326, 2023-12-06T18:27:57.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:27:57.043 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.043 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.043 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.043 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.043 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.043 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:27:57.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.043 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:27:57.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:27:57.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:27:57.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:27:57.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:27:57.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:27:57.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:27:57.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:27:57.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:27:57.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:27:57.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:27:57.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:27:57.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.046 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:27:57.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:27:57.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.046 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.046 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:27:57.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:27:57.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.046 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.046 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:27:57.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:27:57.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:27:57.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:27:57.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:27:57.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:27:57.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:27:57.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:27:57.047 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.047 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.048 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.048 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:57.048 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:27:57.048 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 59.483262127000003, 2023-12-06T18:27:57.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:27:57.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.048 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.048 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.048 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.048 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.048 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:27:57.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.048 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:27:57.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:27:57.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:27:57.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:27:57.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:27:57.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:27:57.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:27:57.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:27:57.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:27:57.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:27:57.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:27:57.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:27:57.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:27:57.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:27:57.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:27:57.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:27:57.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:27:57.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:27:57.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:27:57.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:27:57.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:27:57.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:27:57.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:27:57.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:27:57.053 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.053 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.053 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.053 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:57.053 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:27:57.053 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 59.410552692000003, 2023-12-06T18:27:57.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:27:57.053 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.053 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.054 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.054 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.054 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.054 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:27:57.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.054 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.054 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:27:57.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.054 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:27:57.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.055 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.055 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:27:57.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:27:57.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.055 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.055 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:27:57.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:27:57.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.055 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.055 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:27:57.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:27:57.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:27:57.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:27:57.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:27:57.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:27:57.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:27:57.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:27:57.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:27:57.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:27:57.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:27:57.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:27:57.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:27:57.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:27:57.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:27:57.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:27:57.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:27:57.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:27:57.058 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.058 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.058 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.059 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:27:57.059 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:27:57.059 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 59.409870982999998, 2023-12-06T18:27:57.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:27:57.059 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.059 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.059 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.059 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.059 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.059 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:27:57.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.059 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.060 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:27:57.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.060 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:27:57.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.060 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:27:57.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:27:57.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.061 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.061 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:27:57.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:27:57.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.061 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.061 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:27:57.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:27:57.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.061 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.061 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:27:57.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:27:57.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:27:57.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:27:57.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:27:57.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:27:57.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:27:57.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:27:57.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:27:57.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:27:57.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:27:57.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:27:57.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:27:57.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:27:57.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:27:57.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:27:57.064 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.064 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.064 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.064 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:27:57.064 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:27:57.064 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 58.331066894999999, 2023-12-06T18:27:57.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:27:57.064 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.064 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.065 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.065 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.065 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.065 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:27:57.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.065 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:27:57.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:27:57.066 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.066 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.066 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.066 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:27:57.066 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:27:57.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.066 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.066 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.066 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:27:57.066 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:27:57.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.066 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.066 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.067 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:27:57.067 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:27:57.067 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.067 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.067 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.067 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:27:57.067 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:27:57.067 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.067 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.067 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.067 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:27:57.067 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:27:57.067 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.067 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:27:57.068 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:27:57.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:27:57.068 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:27:57.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:27:57.068 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:27:57.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:27:57.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:27:57.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:27:57.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:27:57.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:27:57.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:27:57.069 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.069 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.069 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.070 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:27:57.070 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:27:57.070 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 58.329650321000003, 2023-12-06T18:27:57.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:27:57.070 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.070 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.070 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.070 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.070 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.070 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:27:57.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.070 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.071 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.071 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:27:57.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.071 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.071 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:27:57.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.071 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.071 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:27:57.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:27:57.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.071 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.072 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:27:57.072 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:27:57.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.072 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.072 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.072 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:27:57.072 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:27:57.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.072 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.072 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.072 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:27:57.072 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:27:57.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.072 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.072 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.072 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:27:57.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:27:57.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:27:57.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:27:57.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:27:57.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:27:57.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:27:57.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:27:57.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.074 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:27:57.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:27:57.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.074 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:27:57.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:27:57.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.074 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:27:57.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:27:57.075 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.075 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.075 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.075 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:27:57.075 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:27:57.075 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 58.328565196, 2023-12-06T18:27:57.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:27:57.075 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.075 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.075 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.075 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.075 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.075 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:27:57.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.076 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.076 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:27:57.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.076 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:27:57.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.076 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:27:57.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:27:57.077 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.077 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.077 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.077 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:27:57.077 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:27:57.077 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.077 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.077 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.077 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:27:57.077 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:27:57.077 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.077 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.077 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.077 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:27:57.077 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:27:57.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.078 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.078 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:27:57.078 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:27:57.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.078 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.078 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:27:57.078 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:27:57.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.078 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.078 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:27:57.078 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:27:57.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:27:57.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:27:57.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:27:57.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:27:57.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:27:57.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:27:57.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:27:57.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:27:57.080 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.080 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.080 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.080 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:27:57.080 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:27:57.080 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 58.293416094000001, 2023-12-06T18:27:57.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:27:57.080 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.081 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.081 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.081 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.081 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.081 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:27:57.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.081 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.081 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:27:57.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.082 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.082 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:27:57.082 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.082 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.082 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.082 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.082 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:27:57.082 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:27:57.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.083 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.083 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.083 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:27:57.083 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:27:57.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.083 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.083 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.083 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:27:57.083 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:27:57.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.083 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.083 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.083 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:27:57.083 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:27:57.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:27:57.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:27:57.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:27:57.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:27:57.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:27:57.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:27:57.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:27:57.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:27:57.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:27:57.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:27:57.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:27:57.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:27:57.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:27:57.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:27:57.086 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.086 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.086 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.086 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:57.086 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:27:57.086 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 51.743685569, 2023-12-06T18:27:57.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:27:57.086 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.087 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.087 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.087 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.087 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.087 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:27:57.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.087 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.087 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:27:57.087 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.087 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:27:57.088 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.088 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.088 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.088 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.088 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:27:57.088 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:27:57.088 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.088 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.088 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.088 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:27:57.088 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:27:57.088 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.088 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.088 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.088 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:27:57.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:27:57.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:27:57.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:27:57.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:27:57.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:27:57.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:27:57.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:27:57.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:27:57.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:27:57.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:27:57.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:27:57.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:27:57.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:27:57.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:27:57.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:27:57.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:27:57.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:27:57.091 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.091 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.091 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.092 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:57.092 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:27:57.092 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 51.742879961, 2023-12-06T18:27:57.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:27:57.092 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.092 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.092 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.092 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.092 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.092 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:27:57.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.092 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.093 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.093 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:27:57.093 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.093 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.093 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.093 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.093 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:27:57.093 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.093 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.093 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.093 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.093 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:27:57.093 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:27:57.093 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.093 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.094 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.094 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:27:57.094 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:27:57.094 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.094 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.094 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.094 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:27:57.094 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:27:57.094 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.094 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.094 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.094 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:27:57.094 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:27:57.094 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.094 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.095 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.095 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:27:57.095 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:27:57.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.095 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.095 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.095 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:27:57.095 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:27:57.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.095 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.095 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.095 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:27:57.095 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:27:57.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.095 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.096 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.096 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:27:57.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:27:57.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.096 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.096 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:27:57.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:27:57.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.096 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.096 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:27:57.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:27:57.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.097 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.097 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:27:57.097 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:27:57.097 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.097 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.097 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.097 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:57.097 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:27:57.097 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 51.742189152000002, 2023-12-06T18:27:57.098 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:27:57.098 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.098 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.098 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.098 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.098 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.098 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:27:57.098 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.098 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.098 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.098 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.098 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:27:57.098 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.098 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.099 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.099 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.099 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:27:57.099 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.099 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.099 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.099 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:27:57.099 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:27:57.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.099 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.099 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.099 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:27:57.099 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:27:57.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.100 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:27:57.100 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:27:57.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.100 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:27:57.100 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:27:57.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.100 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:27:57.100 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:27:57.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:27:57.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:27:57.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:27:57.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:27:57.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:27:57.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:27:57.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:27:57.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:27:57.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:27:57.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:27:57.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:27:57.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:27:57.102 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.103 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.103 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.103 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.103 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:57.103 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:27:57.103 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 51.737891644000001, 2023-12-06T18:27:57.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:27:57.103 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.103 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.103 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.103 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.103 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.103 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:27:57.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.104 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.104 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.104 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:27:57.104 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.104 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.104 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:27:57.104 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.104 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.104 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:27:57.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:27:57.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.105 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.105 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:27:57.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:27:57.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.105 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.105 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:27:57.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:27:57.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.105 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.105 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:27:57.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:27:57.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.106 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.106 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:27:57.106 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:27:57.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.106 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.106 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:27:57.106 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:27:57.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.106 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:27:57.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:27:57.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.107 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:27:57.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:27:57.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.107 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:27:57.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:27:57.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.107 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:27:57.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:27:57.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.108 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.108 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:27:57.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:27:57.108 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.108 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.108 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.109 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:57.109 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:27:57.109 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 51.737223968999999, 2023-12-06T18:27:57.109 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:27:57.109 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.109 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.109 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.109 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.109 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.109 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:27:57.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.109 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.109 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.109 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.109 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:27:57.110 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.110 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.110 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.110 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.110 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:27:57.110 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.110 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.110 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.110 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:27:57.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:27:57.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:27:57.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:27:57.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:27:57.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:27:57.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:27:57.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:27:57.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:27:57.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:27:57.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:27:57.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:27:57.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:27:57.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:27:57.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:27:57.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:27:57.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:27:57.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:27:57.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.114 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:27:57.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:27:57.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.114 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:27:57.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:27:57.114 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.114 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.114 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.114 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:57.114 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:27:57.115 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 51.736568540999997, 2023-12-06T18:27:57.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:27:57.115 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.115 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.115 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.115 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.115 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.115 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:27:57.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.115 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:27:57.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:27:57.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:27:57.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:27:57.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:27:57.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:27:57.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:27:57.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:27:57.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:27:57.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:27:57.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:27:57.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:27:57.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:27:57.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:27:57.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:27:57.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:27:57.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:27:57.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:27:57.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:27:57.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:27:57.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:27:57.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:27:57.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:27:57.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:27:57.119 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.120 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.120 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.120 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:27:57.120 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:27:57.120 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 51.695913994000001, 2023-12-06T18:27:57.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:27:57.120 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.120 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:27:57.120 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:27:57.120 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:27:57.120 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:27:57.120 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:27:57.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.120 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:27:57.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:27:57.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:27:57.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:27:57.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:27:57.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:27:57.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:27:57.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:27:57.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:27:57.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:27:57.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:27:57.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:27:57.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:27:57.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:27:57.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:27:57.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:27:57.124 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:27:57.124 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.124 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.124 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:27:57.124 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:27:57.124 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:27:57.124 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.124 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.124 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:27:57.124 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:27:57.124 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:27:57.124 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.124 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.124 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:27:57.124 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:27:57.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:27:57.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.125 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.125 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:27:57.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:27:57.125 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.125 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.125 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.125 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.125 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.125 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:57.127 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:27:57.129 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:57.124+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:57.228 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:57.225+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:57.314 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:57.314 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:57.314 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:27:57.314 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:57.327 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:27:57.329 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:57.326+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:57.430 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:57.426+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:57.465 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:57.466 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:57.466 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:27:57.466 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:27:57.466 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:57.475 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:27:57.499 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:27:57.530 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:57.527+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:57.614 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:57.615 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:27:57.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:27:57.615 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:27:57.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.615 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:27:57.615 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:27:57.615 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 35.798612872, 2023-12-06T18:27:57.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:27:57.615 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:27:57.615 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:27:57.615 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:27:57.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:27:57.615 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:27:57.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:27:57.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:27:57.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:27:57.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:27:57.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:27:57.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:27:57.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:27:57.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:27:57.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:27:57.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:27:57.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:27:57.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:27:57.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:27:57.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:27:57.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:27:57.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:27:57.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:27:57.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:27:57.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:27:57.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:27:57.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:27:57.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:27:57.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:27:57.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:27:57.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:27:57.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:27:57.618 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.618 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.618 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.618 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:27:57.619 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:27:57.619 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:57.624 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:27:57.668 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:57.664+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:57.766 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:57.766 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:57.766 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:27:57.766 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:57.770 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:27:57.770 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:27:57.771 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:57.767+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:57.776 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:27:57.833 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:57.830+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:57.863 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:27:57.863 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:27:57.863 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:27:57.934 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:57.930+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:57.983 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:27:57.984 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:27:58.019 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:27:58.020 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:27:58.020 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:27:58.020 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:27:58.020 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:27:58.029 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:27:58.074 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:58.070+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:58.092 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:27:58.092 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:27:58.135 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:58.131+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:58.169 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:58.169 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:58.179 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:27:58.226 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:27:58.226 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:27:58.235 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:58.232+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:58.326 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:58.327 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:58.333 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:27:58.333 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:27:58.336 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:58.332+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:58.337 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:27:58.426 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:27:58.427 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:27:58.436 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:58.433+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:58.508 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:58.508 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:58.517 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:27:58.518 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:27:58.518 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:27:58.560 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:58.557+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:58.639 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:58.635+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:58.646 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:58.646 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:58.656 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:27:58.683 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:27:58.696 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2023-12-06T18:27:58.696 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-12-06T18:27:58.696 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:27:58.739 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:58.735+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:58.771 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is set 2023-12-06T18:27:58.786 INFO:tasks.ceph.mon.c.smithi061.stderr:2023-12-06T18:27:58.771+0000 9f54640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 86918) UID: 0 2023-12-06T18:27:58.786 INFO:tasks.ceph.mon.c.smithi061.stderr:2023-12-06T18:27:58.773+0000 9f54640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-12-06T18:27:58.833 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:58.833 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:58.843 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:27:58.843 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:58.839+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:58.944 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:58.940+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:27:58.993 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:58.993 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:59.002 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:27:59.045 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:27:59.041+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:27:59.169 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:59.165+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:59.184 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:59.184 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:59.194 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:27:59.250 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:27:59.244+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:27:59.319 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:02:10.658 86953== 2023-12-06T18:27:59.319 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:02:10.658 86953== Exit program on first error (--exit-on-first-error=yes) 2023-12-06T18:27:59.328 INFO:tasks.ceph.mon.c.smithi061.stderr:daemon-helper: command failed with exit status 42 2023-12-06T18:27:59.348 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:27:59.345+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:27:59.366 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:59.366 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:59.376 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:27:59.450 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:59.446+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:59.548 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:59.548 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:59.550 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:27:59.547+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:27:59.556 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:27:59.651 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:27:59.648+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:27:59.690 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:59.690 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:59.699 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:27:59.739 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:27:59.739 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:27:59.784 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:27:59.780+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:27:59.853 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:27:59.849+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:27:59.867 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:27:59.867 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:27:59.876 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:27:59.961 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:27:59.957+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:00.044 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:00.044 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:00.053 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:28:00.054 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:00.051+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:00.156 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:00.152+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:00.221 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:00.221 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:00.230 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:28:00.275 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:00.271+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:00.357 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:00.353+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:00.369 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:00.369 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:00.379 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:28:00.458 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:00.455+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:00.531 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:28:00.560 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:00.561 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:00.562 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:28:00.558+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:28:00.571 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:28:00.660 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:00.656+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:00.760 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:00.757+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:00.785 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-12-06T18:28:00.820 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:28:00.820 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:28:00.821 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:00.821 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:00.821 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:00.821 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:00.821 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:00.835 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:28:00.875 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:00.872+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:00.916 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:28:00.916 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:28:00.962 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:00.958+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:01.018 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:28:01.018 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:28:01.020 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:01.020 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:01.021 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:01.021 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:01.032 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:28:01.063 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:01.059+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:01.116 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:28:01.117 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:28:01.164 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:01.160+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:01.206 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:28:01.207 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:28:01.213 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:01.213 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:01.213 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:01.213 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:01.213 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:01.222 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:28:01.264 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:01.260+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:01.291 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:28:01.291 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:28:01.365 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:28:01.362+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:28:01.392 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:28:01.392 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:28:01.460 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:01.460 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:01.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:01.460 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:01.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.461 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:28:01.461 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:01.461 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 66.049592059999995, 2023-12-06T18:28:01.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:28:01.461 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.461 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.461 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.461 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.461 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.461 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:28:01.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.461 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:01.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:01.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:28:01.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:28:01.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:28:01.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:28:01.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:28:01.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:28:01.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:28:01.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:28:01.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.464 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.464 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.464 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:28:01.464 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:28:01.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.464 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.464 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:28:01.464 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:28:01.464 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:28:01.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.464 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.464 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.464 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:28:01.464 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:28:01.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:28:01.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:28:01.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:28:01.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:28:01.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:28:01.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:28:01.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:28:01.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:28:01.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:28:01.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:28:01.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:28:01.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:28:01.467 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.467 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.467 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.467 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:01.467 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:01.467 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 63.941018821, 2023-12-06T18:28:01.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:28:01.467 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.467 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.467 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.467 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.467 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.468 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:28:01.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.468 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:01.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:01.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:28:01.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:28:01.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:28:01.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:28:01.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:28:01.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:28:01.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:28:01.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:28:01.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:28:01.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:28:01.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:28:01.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:28:01.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:28:01.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:28:01.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:28:01.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:28:01.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:28:01.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:28:01.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:28:01.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:28:01.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:28:01.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:28:01.472 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.472 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.472 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.472 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:01.472 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:01.473 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 63.940273589, 2023-12-06T18:28:01.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:28:01.473 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.473 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.473 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.473 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.473 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.473 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:28:01.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.473 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:01.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:01.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:28:01.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:28:01.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:28:01.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:28:01.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:28:01.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:28:01.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:28:01.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:28:01.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:28:01.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:28:01.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:28:01.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:28:01.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:28:01.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:28:01.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:28:01.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:28:01.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:28:01.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:28:01.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:28:01.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:28:01.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.478 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:28:01.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:28:01.478 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.478 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.478 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.478 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:01.478 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:01.478 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 63.939589834000003, 2023-12-06T18:28:01.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:28:01.478 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.479 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.479 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.479 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.479 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.479 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:28:01.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.479 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:01.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:01.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:28:01.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:28:01.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:28:01.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:28:01.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:28:01.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:28:01.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:28:01.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:28:01.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:28:01.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:28:01.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:28:01.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:28:01.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:28:01.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:28:01.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:28:01.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:28:01.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:28:01.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:28:01.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:28:01.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:28:01.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:28:01.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:28:01.484 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.484 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.484 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.484 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.484 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.484 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:01.484 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:01.484 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 63.934016579000001, 2023-12-06T18:28:01.484 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:28:01.484 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.484 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.484 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.484 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.484 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.484 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:28:01.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.485 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.485 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.485 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:01.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.485 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.485 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:01.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.485 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.485 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:28:01.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:28:01.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:28:01.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:28:01.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:28:01.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:28:01.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:28:01.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:28:01.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:28:01.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:28:01.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:28:01.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:28:01.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:28:01.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:28:01.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:28:01.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:28:01.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:28:01.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:28:01.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:28:01.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:28:01.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:28:01.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:28:01.489 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.489 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.489 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.489 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:01.489 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:01.489 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 63.933342938999999, 2023-12-06T18:28:01.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:28:01.490 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.490 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.490 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.490 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.490 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.490 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:28:01.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.490 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.490 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.490 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:01.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:01.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:28:01.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:28:01.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:28:01.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:28:01.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:28:01.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:28:01.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:28:01.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:28:01.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:28:01.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:28:01.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:28:01.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:28:01.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:28:01.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:28:01.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:28:01.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:28:01.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:28:01.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:28:01.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:28:01.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:28:01.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:28:01.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:28:01.495 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.495 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.495 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.495 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:01.495 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:01.495 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 63.932680740000002, 2023-12-06T18:28:01.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:28:01.495 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.495 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.495 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.495 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.495 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.495 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:28:01.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.496 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.496 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:01.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.496 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:01.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.496 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:28:01.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:28:01.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:28:01.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:28:01.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:28:01.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:28:01.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:28:01.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:28:01.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:28:01.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:28:01.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:28:01.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:28:01.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:28:01.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:28:01.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:28:01.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:28:01.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:28:01.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:28:01.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:28:01.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:28:01.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:28:01.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:28:01.500 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.500 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.500 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.501 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:01.501 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:01.501 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 63.859971305000002, 2023-12-06T18:28:01.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:28:01.501 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.501 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.501 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.501 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.501 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.501 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:28:01.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.501 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.501 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:01.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:01.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:28:01.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:28:01.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:28:01.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:28:01.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:28:01.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:28:01.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:28:01.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:28:01.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:28:01.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:28:01.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:28:01.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:28:01.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:28:01.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:28:01.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:28:01.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:28:01.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:28:01.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:28:01.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:28:01.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:28:01.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:28:01.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:28:01.506 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.506 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.506 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.506 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:01.506 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:01.507 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 63.859289595999996, 2023-12-06T18:28:01.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:28:01.507 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.507 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.507 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.507 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.507 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.507 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:28:01.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.507 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:01.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:01.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:28:01.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:28:01.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:28:01.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:28:01.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:28:01.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:28:01.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:28:01.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:28:01.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:28:01.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:28:01.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.510 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.510 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:28:01.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:28:01.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.510 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.510 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:28:01.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:28:01.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.510 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.510 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:28:01.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:28:01.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:28:01.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:28:01.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:28:01.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:28:01.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:28:01.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:28:01.511 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.512 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.512 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.512 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:01.512 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:01.512 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 62.780485507999998, 2023-12-06T18:28:01.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:28:01.512 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.512 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.512 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.512 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.512 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.512 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:28:01.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.513 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:01.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:01.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:28:01.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:28:01.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:28:01.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:28:01.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:28:01.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:28:01.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:28:01.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:28:01.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:28:01.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:28:01.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:28:01.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:28:01.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:28:01.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:28:01.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:28:01.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:28:01.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:28:01.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:28:01.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:28:01.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:28:01.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:28:01.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:28:01.517 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.517 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.517 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.517 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:01.517 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:01.517 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 62.779068934000001, 2023-12-06T18:28:01.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:28:01.518 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.518 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.518 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.518 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.518 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.518 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:28:01.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.518 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:01.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:01.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:28:01.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:28:01.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:28:01.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:28:01.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:28:01.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:28:01.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:28:01.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:28:01.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:28:01.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:28:01.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:28:01.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:28:01.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:28:01.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:28:01.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:28:01.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:28:01.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.522 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.522 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:28:01.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:28:01.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.522 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.522 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:28:01.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:28:01.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.522 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:28:01.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:28:01.523 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.523 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.523 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.523 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:01.523 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:01.523 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 62.777983808999998, 2023-12-06T18:28:01.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:28:01.523 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.523 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.523 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.523 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.524 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.524 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:28:01.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.524 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:01.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:01.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:28:01.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:28:01.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:28:01.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:28:01.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:28:01.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:28:01.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:28:01.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:28:01.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:28:01.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:28:01.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:28:01.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:28:01.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:28:01.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:28:01.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:28:01.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:28:01.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:28:01.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:28:01.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:28:01.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:28:01.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:28:01.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:28:01.528 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.528 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.528 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.528 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:01.529 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:01.529 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 62.742834707, 2023-12-06T18:28:01.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:28:01.529 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.529 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.529 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.529 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.529 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.529 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:28:01.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.529 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:01.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:01.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:28:01.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:28:01.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:28:01.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:28:01.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:28:01.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:28:01.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:28:01.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:28:01.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:28:01.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:28:01.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:28:01.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:28:01.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:28:01.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:28:01.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:28:01.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:28:01.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:28:01.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:28:01.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:28:01.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:28:01.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:28:01.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:28:01.534 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.534 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.534 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.534 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:01.534 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:01.534 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 56.193104181999999, 2023-12-06T18:28:01.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:28:01.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.534 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.534 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.535 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.535 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.535 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:28:01.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.535 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:01.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:01.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:28:01.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:28:01.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:28:01.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:28:01.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:28:01.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:28:01.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:28:01.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:28:01.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:28:01.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:28:01.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.538 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:28:01.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:28:01.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.538 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:28:01.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:28:01.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.538 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:28:01.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:28:01.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:28:01.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:28:01.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:28:01.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:28:01.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:28:01.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:28:01.539 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.539 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.540 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.540 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:01.540 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:01.540 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 56.192298573999999, 2023-12-06T18:28:01.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:28:01.540 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.540 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.540 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.540 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.540 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.540 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:28:01.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.541 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:01.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:01.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:28:01.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:28:01.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:28:01.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:28:01.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:28:01.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:28:01.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:28:01.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:28:01.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:28:01.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:28:01.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:28:01.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:28:01.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:28:01.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:28:01.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:28:01.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:28:01.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:28:01.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:28:01.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:28:01.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:28:01.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:28:01.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:28:01.545 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.545 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.545 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.545 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:01.545 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:01.545 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 56.191607765000001, 2023-12-06T18:28:01.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:28:01.546 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.546 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.546 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.546 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.546 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.546 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:28:01.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.546 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:01.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:01.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:28:01.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:28:01.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:28:01.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:28:01.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:28:01.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:28:01.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:28:01.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:28:01.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:28:01.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:28:01.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:28:01.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:28:01.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:28:01.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:28:01.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:28:01.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:28:01.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:28:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:28:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:28:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:28:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:28:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:28:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 56.187310257, 2023-12-06T18:28:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:28:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:28:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:28:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:28:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:28:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:28:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:28:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:28:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:28:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:28:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:28:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:28:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:28:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:28:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:28:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:28:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:28:01.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:28:01.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:28:01.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:28:01.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:28:01.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:28:01.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.556 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.556 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:28:01.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:28:01.556 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.556 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.556 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.556 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:01.556 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:01.556 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 56.186642581999998, 2023-12-06T18:28:01.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:28:01.556 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.556 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.557 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.557 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.557 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.557 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:28:01.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.557 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:01.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:01.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:28:01.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:28:01.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:28:01.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:28:01.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:28:01.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:28:01.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:28:01.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:28:01.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:28:01.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:28:01.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:28:01.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:28:01.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:28:01.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:28:01.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:28:01.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:28:01.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:28:01.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:28:01.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:28:01.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:28:01.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:28:01.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:28:01.561 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.561 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.561 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.561 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:01.562 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:01.562 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 56.185987154000003, 2023-12-06T18:28:01.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:28:01.562 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.562 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.562 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.562 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.562 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.562 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:28:01.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.562 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:01.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:01.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:28:01.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:28:01.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:28:01.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:28:01.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:28:01.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:28:01.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:28:01.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:28:01.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:28:01.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:28:01.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:28:01.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:28:01.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:28:01.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:28:01.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:28:01.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:28:01.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:28:01.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:28:01.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:28:01.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:28:01.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:28:01.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:28:01.567 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.567 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.567 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.567 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:01.567 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:01.567 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 56.145332607, 2023-12-06T18:28:01.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:28:01.567 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:01.567 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:01.567 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:01.567 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:01.568 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:01.568 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:28:01.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.568 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:01.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.568 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:01.568 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:01.568 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.568 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:01.568 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:01.568 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:01.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:01.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:28:01.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:28:01.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:01.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:28:01.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:28:01.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:01.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:28:01.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:28:01.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.570 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:01.570 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:28:01.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:28:01.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.570 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:01.570 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:28:01.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:28:01.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.570 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:01.570 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:28:01.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:28:01.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:01.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:28:01.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:28:01.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:01.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:28:01.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:28:01.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:01.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:28:01.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:28:01.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:01.572 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:28:01.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:28:01.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:01.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:01.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:01.572 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:28:01.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:28:01.572 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.572 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.572 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.572 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:01.572 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:01.572 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:01.574 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:28:01.574 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:28:01.575 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:28:01.575 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:01.571+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:01.674 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-12-06T18:28:01.675 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:01.671+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:01.755 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:01.756 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:01.756 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:01.756 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:01.765 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:28:01.807 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:01.803+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:01.876 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:01.873+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:01.942 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:01.942 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:01.943 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:01.943 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:01.943 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:01.952 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:28:01.978 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:01.974+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:02.079 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:02.076+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:02.130 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:02.131 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:02.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:02.131 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:02.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:02.131 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:28:02.131 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:02.131 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 40.314734166000001, 2023-12-06T18:28:02.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:28:02.131 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:02.131 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:28:02.131 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:02.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:02.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:02.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:02.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:02.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:02.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:02.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:02.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:02.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:02.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:02.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:02.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:02.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:28:02.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:28:02.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:02.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:02.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:02.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:28:02.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:28:02.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:02.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:02.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:02.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:28:02.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:28:02.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:02.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:02.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:02.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:28:02.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:28:02.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:02.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:02.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:02.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:28:02.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:28:02.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:02.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:02.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:02.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:28:02.134 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:28:02.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:02.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:02.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:02.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:28:02.134 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:28:02.134 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:02.134 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:02.134 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:02.134 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:02.134 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:02.134 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:02.140 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:28:02.186 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:02.182+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:02.278 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:02.278 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:02.278 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:02.278 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:02.281 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:02.277+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:02.287 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:28:02.382 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:02.378+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:02.466 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:02.466 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:02.466 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:02.466 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:02.466 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:02.475 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:28:02.516 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:02.512+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:02.584 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:02.581+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:02.652 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:02.652 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:02.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:02.652 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:02.652 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:02.661 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:28:02.706 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:02.702+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:02.785 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:02.781+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:02.794 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:02.795 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:02.795 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:02.795 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:02.804 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:28:02.888 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:02.885+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:02.971 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:02.972 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:02.972 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:02.972 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:02.972 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:02.983 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:28:03.025 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:03.020+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:03.088 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:28:03.085+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:28:03.151 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:03.151 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:03.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:03.151 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:28:03.151 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:03.160 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:28:03.189 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:03.186+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:03.290 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:03.286+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:03.328 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:03.328 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:03.328 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:03.329 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:03.338 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:28:03.391 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:03.388+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:03.492 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:03.489+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:03.509 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:03.509 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:03.509 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:03.509 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:03.509 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:03.518 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:28:03.576 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:28:03.621 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:03.617+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:03.694 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:03.690+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:03.730 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:03.730 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:03.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:03.731 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:28:03.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.731 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:28:03.731 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:03.731 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 67.111692454999996, 2023-12-06T18:28:03.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:28:03.731 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.731 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.731 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:03.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:03.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:28:03.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:28:03.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:28:03.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:28:03.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:28:03.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:28:03.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:28:03.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:28:03.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:28:03.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:28:03.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:28:03.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:28:03.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:28:03.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:28:03.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:28:03.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:28:03.735 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.735 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.735 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.735 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:28:03.735 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:03.735 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 66.088865446, 2023-12-06T18:28:03.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:28:03.735 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.735 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.735 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:03.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:03.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:28:03.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:28:03.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:28:03.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:28:03.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:28:03.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:28:03.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:28:03.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:28:03.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:28:03.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:28:03.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:28:03.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:28:03.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:28:03.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:28:03.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:28:03.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:28:03.739 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.739 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.739 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.739 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:28:03.739 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:03.739 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 66.076818807999999, 2023-12-06T18:28:03.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:28:03.740 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.740 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.740 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:03.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:03.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:28:03.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:28:03.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:28:03.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:28:03.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:28:03.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:28:03.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:28:03.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:28:03.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:28:03.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:28:03.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:28:03.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:28:03.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:28:03.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:28:03.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:28:03.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:28:03.743 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.743 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.743 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.743 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:28:03.743 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:03.743 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 66.055418200999995, 2023-12-06T18:28:03.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:28:03.744 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.744 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.744 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:03.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:03.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:28:03.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:28:03.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:28:03.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:28:03.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:28:03.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:28:03.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:28:03.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:28:03.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:28:03.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:28:03.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:28:03.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:28:03.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:28:03.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:28:03.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:28:03.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:28:03.747 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.747 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.747 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.747 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:28:03.747 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:03.747 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 66.054238878000007, 2023-12-06T18:28:03.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:28:03.747 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.747 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.748 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:03.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:03.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:28:03.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:28:03.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:28:03.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:28:03.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:28:03.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:28:03.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:28:03.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:28:03.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.750 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:28:03.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:28:03.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.750 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:28:03.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:28:03.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.750 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:28:03.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:28:03.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:28:03.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:28:03.751 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.751 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.751 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.751 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:28:03.751 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:03.751 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 66.053666896999999, 2023-12-06T18:28:03.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:28:03.752 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.752 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.752 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:03.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:03.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:28:03.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:28:03.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:28:03.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:28:03.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:28:03.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:28:03.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:28:03.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:28:03.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:28:03.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:28:03.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:28:03.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:28:03.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:28:03.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:28:03.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:28:03.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:28:03.755 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.755 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.755 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.755 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:28:03.755 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:03.755 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 65.607831426999994, 2023-12-06T18:28:03.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:28:03.756 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.756 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.756 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:03.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:03.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:28:03.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:28:03.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:28:03.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:28:03.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:28:03.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:28:03.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:28:03.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:28:03.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:28:03.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:28:03.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:28:03.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:28:03.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:28:03.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:28:03.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:28:03.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:28:03.759 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.759 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.759 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.760 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:28:03.760 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:03.760 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 65.606231570999995, 2023-12-06T18:28:03.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:28:03.760 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.760 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.760 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.760 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:03.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:03.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:28:03.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:28:03.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:28:03.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:28:03.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:28:03.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:28:03.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:28:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:28:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:28:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:28:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:28:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:28:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:28:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:28:03.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:28:03.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:28:03.764 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.764 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.764 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.764 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:28:03.764 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:03.764 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 65.227025955000002, 2023-12-06T18:28:03.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:28:03.764 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.764 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.764 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:03.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:03.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:28:03.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:28:03.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:28:03.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:28:03.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:28:03.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:28:03.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:28:03.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:28:03.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:28:03.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:28:03.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:28:03.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:28:03.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:28:03.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:28:03.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:28:03.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:28:03.767 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.767 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.768 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.768 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:28:03.768 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:03.768 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 64.989523773000002, 2023-12-06T18:28:03.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:28:03.768 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.768 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.768 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.768 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.768 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:03.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.768 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.768 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:03.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:28:03.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:28:03.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:28:03.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:28:03.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:28:03.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:28:03.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:28:03.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:28:03.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:28:03.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:28:03.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:28:03.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:28:03.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:28:03.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:28:03.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:28:03.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:28:03.771 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.771 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.771 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.771 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:28:03.772 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:03.772 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 64.988795773999996, 2023-12-06T18:28:03.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:28:03.772 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.772 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.772 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:03.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:03.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:28:03.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:28:03.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:28:03.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:28:03.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:28:03.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:28:03.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:28:03.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:28:03.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:28:03.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:28:03.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:28:03.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:28:03.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:28:03.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:28:03.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:28:03.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:28:03.775 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.775 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.775 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.776 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:28:03.776 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:03.776 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 64.983783364000004, 2023-12-06T18:28:03.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:28:03.776 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.776 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.776 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:03.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:03.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:28:03.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:28:03.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:28:03.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:28:03.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:28:03.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:28:03.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:28:03.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:28:03.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:28:03.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:28:03.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:28:03.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:28:03.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:28:03.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:28:03.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:28:03.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:28:03.780 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.780 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.780 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.780 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:28:03.780 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:03.780 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 64.978256285000001, 2023-12-06T18:28:03.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:28:03.780 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.780 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.780 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:03.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:03.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:28:03.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:28:03.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:28:03.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:28:03.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:28:03.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:28:03.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:28:03.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:28:03.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:28:03.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:28:03.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:28:03.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:28:03.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:28:03.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:28:03.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:28:03.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:28:03.784 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.784 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.784 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.784 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:28:03.784 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:03.784 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 64.977745447999993, 2023-12-06T18:28:03.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:28:03.784 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.784 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.784 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:03.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:03.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:28:03.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:28:03.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:28:03.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:28:03.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:28:03.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:28:03.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:28:03.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:28:03.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:28:03.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:28:03.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:28:03.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:28:03.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:28:03.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:28:03.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:28:03.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:28:03.787 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.787 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.787 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.788 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:28:03.788 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:03.788 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 64.977286187000004, 2023-12-06T18:28:03.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:28:03.788 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.788 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.788 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:03.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:03.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:28:03.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:28:03.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:28:03.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:28:03.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:28:03.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:28:03.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:28:03.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:28:03.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:28:03.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:28:03.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:28:03.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:28:03.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:28:03.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:28:03.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:28:03.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:28:03.791 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.791 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.791 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.791 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:28:03.791 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:03.792 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 58.397370379999998, 2023-12-06T18:28:03.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:28:03.792 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.792 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.792 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:03.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:03.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:28:03.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:28:03.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:28:03.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:28:03.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:28:03.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:28:03.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:28:03.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:28:03.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:28:03.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:28:03.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:28:03.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:28:03.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:28:03.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:28:03.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:28:03.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:28:03.795 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.795 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.795 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.796 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:28:03.796 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:03.796 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 58.391132575, 2023-12-06T18:28:03.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:28:03.796 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.796 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.796 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:03.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:03.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:28:03.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:28:03.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:28:03.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:28:03.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:28:03.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:28:03.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:28:03.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:28:03.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:28:03.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:28:03.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:28:03.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:28:03.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:28:03.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:28:03.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:28:03.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:28:03.799 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.799 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.800 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.800 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:28:03.800 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:03.800 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 58.385069496, 2023-12-06T18:28:03.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:28:03.800 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.800 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.800 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:03.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:03.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:28:03.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:28:03.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:28:03.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:28:03.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.802 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.802 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:28:03.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:28:03.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.802 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.802 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:28:03.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:28:03.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.802 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.802 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:28:03.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:28:03.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:28:03.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:28:03.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:28:03.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:28:03.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:28:03.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:28:03.803 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.803 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.804 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.804 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:28:03.804 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:03.804 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 58.247682411, 2023-12-06T18:28:03.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:28:03.804 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.804 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.804 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:03.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:03.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:28:03.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:28:03.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:28:03.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:28:03.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:28:03.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:28:03.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:28:03.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:28:03.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:28:03.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:28:03.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:28:03.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:28:03.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:28:03.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:28:03.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:28:03.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:28:03.807 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.807 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.807 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.808 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:28:03.808 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:03.808 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 58.241011684, 2023-12-06T18:28:03.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:28:03.808 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:03.808 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:03.808 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:03.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:03.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:03.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:03.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:03.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:03.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:03.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:28:03.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:28:03.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:03.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:28:03.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:28:03.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:03.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:28:03.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:28:03.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:03.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:28:03.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:28:03.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:03.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:28:03.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:28:03.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:03.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:28:03.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:28:03.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:03.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:28:03.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:28:03.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:03.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:03.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:03.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:28:03.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:28:03.811 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.811 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.811 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.811 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:03.812 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:03.812 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:03.813 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:28:03.814 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:03.810+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:03.914 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:03.911+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:03.980 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:03.981 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:03.981 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:03.981 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:03.989 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:28:04.015 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:04.012+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:04.116 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:04.116 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:04.116 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:04.116 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:04.116 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:04.117 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:04.112+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:04.125 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:28:04.216 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:04.213+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:04.253 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:04.253 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:04.254 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:04.254 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:28:04.254 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:04.263 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:28:04.317 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:04.313+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:04.418 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:04.415+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:04.434 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:04.434 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:04.434 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:04.434 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:04.443 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:28:04.519 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:04.515+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:04.570 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:04.570 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:04.570 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:04.570 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:04.570 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:04.579 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:28:04.620 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:04.616+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:04.720 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:04.717+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:04.740 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:28:04.740 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:28:04.740 INFO:tasks.thrashosds.thrasher:Killing osd 7, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:28:04.740 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-12-06T18:28:04.741 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:28:04.754 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:28:04.738+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 87111) UID: 0 2023-12-06T18:28:04.755 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:28:04.739+0000 b47e640 -1 osd.7 37 *** Got signal Terminated *** 2023-12-06T18:28:04.756 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:04.756 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:04.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:04.756 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:28:04.756 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:04.765 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:28:04.799 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-12-06T18:28:04.799 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi061 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c" 2023-12-06T18:28:04.799 INFO:tasks.ceph.mon.c:Stopped 2023-12-06T18:28:04.799 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2023-12-06T18:28:04.799 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:28:04.821 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:04.818+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:04.915 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:04.915 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:04.915 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:04.915 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:04.922 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:04.919+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:04.925 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:28:05.024 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:05.019+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:05.061 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:05.061 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:05.062 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:05.062 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:05.062 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:05.071 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:28:05.125 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:05.120+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:05.207 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:05.207 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:05.207 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:05.207 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:05.208 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:05.217 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:28:05.261 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:05.257+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:05.326 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:05.322+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:05.353 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:05.353 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:05.354 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:05.354 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:05.363 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:28:05.451 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:05.447+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:05.458 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":16,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":8,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:28:05.528 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:05.524+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:05.545 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:05.545 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:05.545 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:05.545 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:05.546 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:05.554 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:28:05.639 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:05.635+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:05.730 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:05.726+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:05.795 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:05.795 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:05.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:05.795 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:05.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.795 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:28:05.796 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:05.796 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 70.382470429999998, 2023-12-06T18:28:05.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:28:05.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.796 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.796 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.796 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.796 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.796 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:28:05.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.796 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:05.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:05.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:28:05.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:28:05.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:28:05.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:28:05.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:28:05.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:28:05.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:28:05.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:28:05.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:28:05.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:28:05.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:28:05.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:28:05.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:28:05.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:28:05.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:28:05.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:28:05.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:28:05.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:28:05.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:28:05.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:28:05.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:28:05.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:28:05.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:28:05.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:28:05.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:28:05.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:28:05.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:28:05.801 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.801 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.801 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.802 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:05.802 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:05.802 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 68.273897191000003, 2023-12-06T18:28:05.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:28:05.802 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.802 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.802 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.802 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.802 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.802 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:28:05.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.802 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:05.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:05.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:28:05.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:28:05.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:28:05.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:28:05.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:28:05.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:28:05.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:28:05.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:28:05.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:28:05.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:28:05.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:28:05.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:28:05.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:28:05.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:28:05.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:28:05.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:28:05.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:28:05.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:28:05.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:28:05.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:28:05.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:28:05.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:28:05.807 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.807 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.807 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.807 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:05.807 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:05.807 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 68.273151959000003, 2023-12-06T18:28:05.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:28:05.807 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.807 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.808 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.808 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.808 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:28:05.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.808 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.808 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.808 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:05.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.808 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:05.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:28:05.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:28:05.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:28:05.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:28:05.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:28:05.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:28:05.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:28:05.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:28:05.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:28:05.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:28:05.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:28:05.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:28:05.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:28:05.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:28:05.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:28:05.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:28:05.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:28:05.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:28:05.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:28:05.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:28:05.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:28:05.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:28:05.813 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.813 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.813 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.813 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:05.813 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:05.813 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 68.272468204000006, 2023-12-06T18:28:05.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:28:05.813 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.814 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.814 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.814 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.814 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.814 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:28:05.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.814 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.814 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.814 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:05.814 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.815 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.815 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:05.815 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.815 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.815 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.815 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:28:05.815 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:28:05.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.815 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.815 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.815 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:28:05.815 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:28:05.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:28:05.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:28:05.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:28:05.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:28:05.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:28:05.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:28:05.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:28:05.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:28:05.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:28:05.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:28:05.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:28:05.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:28:05.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:28:05.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:28:05.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:28:05.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:28:05.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:28:05.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:28:05.818 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.819 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.819 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.819 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.819 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:05.819 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:05.819 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 68.266894949000005, 2023-12-06T18:28:05.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:28:05.819 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.819 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.819 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.819 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.819 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.819 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:28:05.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.820 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.820 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.820 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.820 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:05.820 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.820 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.820 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.820 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.820 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:05.820 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.820 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.820 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.820 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.820 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:28:05.820 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:28:05.820 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.821 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.821 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.821 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:28:05.821 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:28:05.821 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.821 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.821 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.821 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:28:05.821 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:28:05.821 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.821 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.821 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.821 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:28:05.821 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:28:05.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.822 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.822 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.822 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:28:05.822 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:28:05.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.822 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.822 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.822 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:28:05.822 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:28:05.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.822 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.822 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.822 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:28:05.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:28:05.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:28:05.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:28:05.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:28:05.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:28:05.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:28:05.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:28:05.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.824 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:28:05.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:28:05.824 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.824 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.824 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.824 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:05.824 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:05.824 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 68.266221309000002, 2023-12-06T18:28:05.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:28:05.825 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.825 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.825 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.825 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.825 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.825 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:28:05.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.825 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.825 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.825 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.825 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:05.825 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.825 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.826 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.826 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:05.826 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.826 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.826 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.826 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.826 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:28:05.826 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:28:05.826 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.826 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.826 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.826 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:28:05.826 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:28:05.826 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.826 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.827 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.827 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:28:05.827 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:28:05.827 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.827 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.827 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.827 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:28:05.827 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:28:05.827 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.827 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.827 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.827 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:28:05.827 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:28:05.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.828 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.828 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:28:05.828 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:28:05.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.828 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.828 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:28:05.828 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:28:05.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.828 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.828 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:28:05.828 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:28:05.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.829 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.829 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:28:05.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:28:05.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.829 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.829 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:28:05.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:28:05.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.829 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.829 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:28:05.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:28:05.830 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.830 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.830 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.830 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.830 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.830 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:05.830 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:05.830 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 68.265559109999998, 2023-12-06T18:28:05.830 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:28:05.830 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.830 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.830 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.830 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.830 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.830 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:28:05.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.831 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.831 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.831 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.831 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:05.831 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.831 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.831 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.831 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:05.831 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.831 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.831 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:28:05.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:28:05.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.832 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:28:05.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:28:05.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.832 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:28:05.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:28:05.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.833 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.833 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:28:05.833 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:28:05.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.833 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.833 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.833 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:28:05.833 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:28:05.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.833 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.833 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.833 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:28:05.833 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:28:05.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.833 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.834 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.834 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:28:05.834 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:28:05.834 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.834 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.834 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.834 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:28:05.834 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:28:05.834 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.834 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.834 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.834 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:28:05.834 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:28:05.834 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.835 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.835 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.835 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:28:05.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:28:05.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.835 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.835 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.835 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:28:05.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:28:05.835 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.835 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.835 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.835 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.835 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:05.836 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:05.836 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 68.192849675000005, 2023-12-06T18:28:05.836 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:28:05.836 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.836 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.836 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.836 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.836 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.836 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:28:05.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.836 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.836 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.836 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.836 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:05.836 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.837 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.837 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:05.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.837 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.837 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:28:05.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:28:05.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.837 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.837 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:28:05.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:28:05.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:28:05.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:28:05.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:28:05.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:28:05.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.839 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:28:05.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:28:05.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.839 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:28:05.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:28:05.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.839 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:28:05.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:28:05.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.839 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:28:05.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:28:05.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.840 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:28:05.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:28:05.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.840 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:28:05.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:28:05.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.840 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:28:05.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:28:05.841 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.841 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.841 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.841 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.841 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.841 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:05.841 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:05.841 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 68.192167966, 2023-12-06T18:28:05.841 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:28:05.841 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.841 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.841 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.841 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.841 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.842 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:28:05.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.842 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.842 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.842 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.842 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:05.842 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.842 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.842 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.842 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:05.842 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:28:05.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:28:05.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:28:05.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:28:05.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:28:05.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:28:05.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:28:05.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:28:05.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:28:05.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:28:05.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:28:05.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:28:05.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.845 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:28:05.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:28:05.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.845 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:28:05.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:28:05.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.845 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:28:05.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:28:05.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.846 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.846 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.846 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:28:05.846 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:28:05.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.846 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.846 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.846 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:28:05.846 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:28:05.846 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.846 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.846 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.846 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.847 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:05.847 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:05.847 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 67.113363878000001, 2023-12-06T18:28:05.847 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:28:05.847 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.847 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.847 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.847 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.847 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.847 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:28:05.847 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.847 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.847 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.847 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:05.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:05.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:28:05.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:28:05.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:28:05.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:28:05.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:28:05.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:28:05.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:28:05.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:28:05.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:28:05.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:28:05.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:28:05.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:28:05.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:28:05.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:28:05.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.851 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:28:05.851 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:28:05.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.851 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:28:05.851 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:28:05.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.851 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:28:05.851 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:28:05.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.852 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:28:05.852 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:28:05.852 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.852 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.852 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.852 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.852 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.852 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:05.852 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:05.852 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 67.111947303999997, 2023-12-06T18:28:05.852 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:28:05.852 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.852 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.852 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.853 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.853 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.853 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:28:05.853 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.853 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.853 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.853 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.853 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:05.853 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.853 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.853 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.853 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.853 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:05.853 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.853 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.853 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.854 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.854 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:28:05.854 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:28:05.854 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.854 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.854 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.854 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:28:05.854 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:28:05.854 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.854 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.854 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.854 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:28:05.854 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:28:05.854 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.855 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.855 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.855 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:28:05.855 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:28:05.855 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.855 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.855 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.855 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:28:05.855 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:28:05.855 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.855 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.855 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.855 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:28:05.855 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:28:05.856 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.856 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.856 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.856 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:28:05.856 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:28:05.856 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.856 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.856 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.856 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:28:05.856 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:28:05.856 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.856 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.856 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.856 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:28:05.856 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:28:05.857 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.857 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.857 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.857 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:28:05.857 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:28:05.857 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.857 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.857 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.857 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:28:05.857 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:28:05.857 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.857 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.857 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.857 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.857 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.857 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:05.858 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:05.858 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 67.110862178999994, 2023-12-06T18:28:05.858 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:28:05.858 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.858 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.858 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.858 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.858 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.858 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:28:05.858 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.858 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.859 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:05.859 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.859 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:05.859 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.859 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:28:05.860 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:28:05.860 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.860 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.860 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.860 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:28:05.860 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:28:05.860 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.860 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.860 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.860 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:28:05.860 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:28:05.860 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.860 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.860 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.861 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:28:05.861 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:28:05.861 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.861 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.861 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.861 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:28:05.861 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:28:05.861 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.861 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.861 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.861 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:28:05.861 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:28:05.861 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.861 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.861 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.861 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:28:05.862 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:28:05.862 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.862 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.862 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.862 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:28:05.862 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:28:05.862 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.862 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.862 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.862 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:28:05.862 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:28:05.862 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.862 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.862 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.863 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:28:05.863 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:28:05.863 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.863 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.863 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.863 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:28:05.863 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:28:05.863 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.863 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.863 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.863 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.863 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.863 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:05.864 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:05.864 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 67.075713077000003, 2023-12-06T18:28:05.864 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:28:05.864 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.864 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.864 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.864 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.864 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.864 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:28:05.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.864 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.864 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.864 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.864 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:05.864 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.865 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.865 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.865 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.865 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:05.865 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.865 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.865 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.865 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.865 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:28:05.865 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:28:05.865 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.865 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.865 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.865 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:28:05.865 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:28:05.866 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.866 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.866 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.866 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:28:05.866 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:28:05.866 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.866 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.866 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.866 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:28:05.866 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:28:05.866 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.866 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.866 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.866 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:28:05.866 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:28:05.867 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.867 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.867 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.867 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:28:05.867 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:28:05.867 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.867 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.867 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.867 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:28:05.867 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:28:05.867 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.867 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.867 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.867 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:28:05.868 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:28:05.868 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.868 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.868 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.868 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:28:05.868 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:28:05.868 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.868 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.868 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.868 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:28:05.868 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:28:05.868 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.868 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.869 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.869 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:28:05.869 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:28:05.869 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.869 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.869 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.869 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.869 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.869 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:05.869 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:05.869 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 60.525982552000002, 2023-12-06T18:28:05.869 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:28:05.869 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.870 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.870 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.870 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.870 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:28:05.870 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.870 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.870 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.870 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.870 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:05.870 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.870 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.870 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.870 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.870 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:05.871 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.871 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.871 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.871 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.871 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:28:05.871 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:28:05.871 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.871 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.871 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.871 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:28:05.871 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:28:05.871 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.871 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.871 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.871 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:28:05.871 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:28:05.872 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.872 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.872 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.872 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:28:05.872 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:28:05.872 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.872 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.872 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.872 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:28:05.872 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:28:05.872 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.872 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.872 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.872 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:28:05.873 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:28:05.873 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.873 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.873 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.873 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:28:05.873 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:28:05.873 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.873 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.873 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.873 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:28:05.873 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:28:05.873 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.873 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.873 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.873 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:28:05.874 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:28:05.874 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.874 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.874 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.874 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:28:05.874 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:28:05.874 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.874 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.874 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.874 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:28:05.874 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:28:05.874 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.874 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.875 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.875 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.875 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.875 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:05.875 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:05.875 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 60.525176944000002, 2023-12-06T18:28:05.875 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:28:05.875 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.875 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.875 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.875 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.875 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.875 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:28:05.875 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.875 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.876 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.876 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.876 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:05.876 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.876 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.876 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.876 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.876 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:05.876 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.876 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.876 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.876 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.876 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:28:05.876 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:28:05.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.877 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.877 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.877 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:28:05.877 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:28:05.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.877 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.877 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.877 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:28:05.877 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:28:05.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.877 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.877 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.878 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:28:05.878 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:28:05.878 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.878 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.878 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.878 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:28:05.878 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:28:05.878 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.878 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.878 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.878 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:28:05.878 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:28:05.878 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.879 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.879 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.879 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:28:05.879 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:28:05.879 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.879 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.879 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.879 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:28:05.879 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:28:05.879 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.879 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.879 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.879 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:28:05.879 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:28:05.879 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.880 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.880 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.880 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:28:05.880 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:28:05.880 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.880 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.880 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.880 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:28:05.880 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:28:05.880 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.880 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.880 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.880 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.881 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.881 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:05.881 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:05.881 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 60.524486134999997, 2023-12-06T18:28:05.881 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:28:05.881 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.881 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.881 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.881 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.881 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.881 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:28:05.881 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.881 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.881 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.882 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.882 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:05.882 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.882 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.882 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.882 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.882 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:05.882 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.882 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.882 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.882 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.882 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:28:05.882 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:28:05.882 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.883 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.883 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.883 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:28:05.883 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:28:05.883 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.883 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.883 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.883 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:28:05.883 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:28:05.883 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.883 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.883 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.883 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:28:05.883 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:28:05.884 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.884 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.884 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.884 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:28:05.884 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:28:05.884 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.884 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.884 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.884 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:28:05.884 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:28:05.884 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.884 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.884 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.884 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:28:05.885 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:28:05.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.885 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.885 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.885 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:28:05.885 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:28:05.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.885 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.885 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.885 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:28:05.885 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:28:05.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.885 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.885 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.885 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:28:05.885 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:28:05.886 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.886 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.886 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.886 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:28:05.886 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:28:05.886 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.886 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.886 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.886 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.886 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.886 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:05.886 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:05.886 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 60.520188627000003, 2023-12-06T18:28:05.887 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:28:05.887 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.887 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.887 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.887 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.887 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.887 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:28:05.887 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.887 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.887 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.887 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.887 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:05.888 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.888 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.888 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.888 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.888 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:05.888 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.888 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.888 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.888 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.888 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:28:05.888 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:28:05.888 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.888 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.888 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.889 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:28:05.889 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:28:05.889 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.889 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.889 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.889 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:28:05.889 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:28:05.889 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.889 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.889 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.889 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:28:05.889 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:28:05.890 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.890 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.890 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.890 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:28:05.890 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:28:05.890 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.890 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.890 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.890 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:28:05.890 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:28:05.890 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.890 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.891 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:28:05.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:28:05.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.891 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.891 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:28:05.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:28:05.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.891 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.891 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:28:05.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:28:05.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.892 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.892 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:28:05.892 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:28:05.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.892 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.892 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:28:05.892 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:28:05.892 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.892 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.892 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.893 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:05.893 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:05.893 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 60.519520952000001, 2023-12-06T18:28:05.893 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:28:05.893 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.893 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.893 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.893 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.893 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.893 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:28:05.893 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.893 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.893 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.893 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.893 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:05.894 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.894 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:05.894 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.894 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:28:05.894 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:28:05.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.895 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.895 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:28:05.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:28:05.895 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.895 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.895 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.895 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:28:05.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:28:05.895 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.895 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.895 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.895 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:28:05.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:28:05.895 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.896 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.896 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.896 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:28:05.896 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:28:05.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.896 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.896 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.896 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:28:05.896 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:28:05.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.896 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.896 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.896 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:28:05.896 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:28:05.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.896 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.897 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.897 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:28:05.897 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:28:05.897 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.897 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.897 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.897 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:28:05.897 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:28:05.897 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.897 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.897 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.897 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:28:05.897 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:28:05.898 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.898 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.898 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.898 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:28:05.898 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:28:05.898 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.898 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.898 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.898 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.898 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.898 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:05.898 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:05.898 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 60.518865523999999, 2023-12-06T18:28:05.899 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:28:05.899 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.899 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.899 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.899 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.899 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.899 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:28:05.899 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.899 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.899 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.899 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.899 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:05.899 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.899 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.899 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.900 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.900 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:05.900 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.900 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.900 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.900 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.900 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:28:05.900 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:28:05.900 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.900 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.900 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.900 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:28:05.900 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:28:05.901 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.901 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.901 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.901 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:28:05.901 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:28:05.901 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.901 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.901 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.901 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:28:05.901 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:28:05.901 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.901 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.901 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.901 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:28:05.902 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:28:05.902 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.902 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.902 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.902 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:28:05.902 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:28:05.902 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.902 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.902 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.902 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:28:05.902 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:28:05.902 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.902 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.903 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.903 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:28:05.903 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:28:05.903 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.903 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.903 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.903 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:28:05.903 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:28:05.903 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.903 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.903 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.903 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:28:05.903 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:28:05.903 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.904 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.904 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.904 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:28:05.904 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:28:05.904 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.904 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.904 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.904 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.904 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.904 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:05.904 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:05.904 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 60.478210977000003, 2023-12-06T18:28:05.904 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:28:05.904 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:05.905 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:05.905 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:05.905 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:05.905 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:05.905 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:28:05.905 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.905 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:05.905 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.905 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:05.905 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:05.905 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.905 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.905 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.906 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:05.906 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:05.906 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:05.906 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.906 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.906 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:05.906 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:28:05.906 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:28:05.906 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.906 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.906 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:05.906 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:28:05.906 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:28:05.906 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.907 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.907 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:05.907 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:28:05.907 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:28:05.907 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.907 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.907 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:05.907 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:28:05.907 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:28:05.907 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.907 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.907 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:05.907 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:28:05.907 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:28:05.908 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.908 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.908 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:05.908 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:28:05.908 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:28:05.908 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.908 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.908 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:05.908 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:28:05.908 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:28:05.908 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.908 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.908 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:05.908 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:28:05.908 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:28:05.909 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.909 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.909 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:05.909 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:28:05.909 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:28:05.909 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.909 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.909 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:05.909 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:28:05.909 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:28:05.909 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:05.909 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:05.909 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:05.909 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:28:05.909 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:28:05.910 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.910 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.910 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.910 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:05.910 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:05.910 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:05.911 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:28:05.912 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:05.908+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:06.013 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:06.009+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:06.089 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:06.089 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:06.090 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:06.090 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:06.098 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:28:06.141 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:06.136+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:06.215 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:06.211+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:06.287 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:06.287 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:06.287 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:06.287 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:06.287 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:06.296 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:28:06.316 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:06.313+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:06.418 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:06.414+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:06.477 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:06.477 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:06.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:06.477 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:06.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:06.477 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:28:06.477 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:06.478 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 44.661105649, 2023-12-06T18:28:06.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:28:06.478 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:06.478 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:28:06.478 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:06.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:06.478 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:06.478 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:06.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:06.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:06.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:06.478 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:06.478 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:06.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:06.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:06.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:06.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:06.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:28:06.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:28:06.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:06.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:06.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:06.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:28:06.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:28:06.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:06.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:06.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:06.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:28:06.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:28:06.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:06.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:06.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:06.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:28:06.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:28:06.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:06.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:06.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:06.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:28:06.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:28:06.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:06.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:06.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:06.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:28:06.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:28:06.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:06.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:06.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:06.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:28:06.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:28:06.481 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:06.481 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:06.481 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:06.481 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:06.481 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:06.481 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:06.486 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:28:06.529 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:06.524+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:06.620 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:06.616+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:06.663 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:06.664 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:06.664 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:06.664 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:06.673 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:28:06.721 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:06.717+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:06.836 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:06.833+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:06.850 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:06.851 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:06.851 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:06.851 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:06.851 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:06.860 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:28:06.922 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:06.919+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:07.023 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:07.020+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:07.037 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:07.037 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:07.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:07.037 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:07.037 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:07.046 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:28:07.125 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:07.121+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:07.186 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:07.186 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:07.187 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:07.187 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:07.196 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:28:07.237 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:07.233+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:07.326 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:07.322+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:07.363 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:07.364 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:07.364 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:07.364 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:07.364 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:07.374 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:28:07.458 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:07.454+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:07.528 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:07.525+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:07.540 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:07.540 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:07.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:07.540 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:28:07.540 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:07.550 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:28:07.630 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:07.626+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:07.723 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:07.723 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:07.723 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:07.723 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:07.731 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:07.728+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:07.732 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:28:07.849 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:07.846+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:07.867 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:07.867 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:07.867 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:07.867 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:07.867 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:07.877 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:28:07.883 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-12-06T18:28:07.934 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:07.930+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:08.035 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:08.031+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:08.084 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:08.084 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:08.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:08.084 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:28:08.084 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.085 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:28:08.085 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:08.085 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 71.468491576999995, 2023-12-06T18:28:08.085 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:28:08.085 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.085 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.085 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.085 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.085 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:08.085 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.085 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.085 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.085 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:08.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:28:08.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:28:08.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:28:08.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:28:08.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.087 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:28:08.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:28:08.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.087 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.087 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:28:08.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:28:08.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.087 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.087 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:28:08.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:28:08.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.087 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:28:08.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:28:08.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.088 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.088 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:28:08.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:28:08.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.088 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.088 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:28:08.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:28:08.088 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.088 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.088 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.089 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.089 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:28:08.089 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:08.089 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 70.445664567999998, 2023-12-06T18:28:08.089 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:28:08.089 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.089 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.089 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.089 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.089 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.089 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:08.089 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.089 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.089 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.089 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.090 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:08.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.090 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.090 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.090 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.090 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:28:08.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:28:08.090 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.090 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.090 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.090 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:28:08.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:28:08.090 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.090 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.090 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.091 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:28:08.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:28:08.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.091 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.091 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:28:08.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:28:08.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.091 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.091 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:28:08.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:28:08.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.091 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:28:08.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:28:08.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.092 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:28:08.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:28:08.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.092 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:28:08.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:28:08.092 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.092 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.092 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.093 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:28:08.093 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:08.093 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 70.433617929999997, 2023-12-06T18:28:08.093 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:28:08.093 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.093 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.093 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.093 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.093 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:08.093 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.093 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:08.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:28:08.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:28:08.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:28:08.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:28:08.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:28:08.095 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:28:08.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.095 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:28:08.095 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:28:08.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.095 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:28:08.095 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:28:08.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.095 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:28:08.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:28:08.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.096 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.096 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.096 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:28:08.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:28:08.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.096 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.096 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.096 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:28:08.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:28:08.096 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.096 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.096 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.097 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:28:08.097 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:08.097 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 70.412217322999993, 2023-12-06T18:28:08.097 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:28:08.097 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.097 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.097 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.097 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.097 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:08.097 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.098 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.098 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:08.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.098 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.098 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:28:08.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:28:08.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.098 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.098 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:28:08.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:28:08.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.099 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.099 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.099 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:28:08.099 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:28:08.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.099 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.099 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.099 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:28:08.099 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:28:08.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.099 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.099 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.099 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:28:08.099 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:28:08.100 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.100 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.100 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.100 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:28:08.100 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:28:08.100 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.100 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.100 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.100 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:28:08.100 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:28:08.100 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.100 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.100 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.100 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:28:08.100 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:28:08.101 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.101 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.101 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.101 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.101 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.101 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:28:08.101 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:08.101 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 70.411038000000005, 2023-12-06T18:28:08.101 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:28:08.101 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.101 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.101 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.101 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.101 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.101 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:08.101 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.102 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.102 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.102 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:08.102 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.102 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.102 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.102 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:28:08.102 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:28:08.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.103 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.103 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.103 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:28:08.103 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:28:08.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.103 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.103 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.103 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:28:08.103 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:28:08.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.103 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.103 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.103 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:28:08.103 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:28:08.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.104 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.104 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.104 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:28:08.104 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:28:08.104 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.104 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.104 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.104 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:28:08.104 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:28:08.104 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.104 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.104 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.104 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:28:08.104 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:28:08.104 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.105 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.105 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.105 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:28:08.105 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:28:08.105 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.105 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.105 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.105 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.105 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.105 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:28:08.105 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:08.105 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 70.410466018999998, 2023-12-06T18:28:08.105 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:28:08.105 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.105 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.106 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.106 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.106 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.106 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:08.106 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.106 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.106 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.106 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.106 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:08.106 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.106 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.106 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.106 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.106 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:28:08.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:28:08.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.107 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.107 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.107 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:28:08.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:28:08.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.107 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.107 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.107 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:28:08.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:28:08.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:28:08.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:28:08.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:28:08.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:28:08.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:28:08.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:28:08.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:28:08.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:28:08.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:28:08.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:28:08.109 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.109 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.109 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.109 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:28:08.109 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:08.110 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 69.964630549000006, 2023-12-06T18:28:08.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:28:08.110 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.110 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.110 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.110 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.110 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.110 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:08.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.110 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.110 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.110 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.110 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:08.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.110 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.110 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.111 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.111 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:28:08.111 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:28:08.111 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.111 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.111 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.111 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:28:08.111 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:28:08.111 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.111 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.111 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.111 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:28:08.111 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:28:08.111 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.111 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.112 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.112 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:28:08.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:28:08.112 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.112 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.112 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:28:08.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:28:08.112 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.112 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.112 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:28:08.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:28:08.112 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:28:08.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:28:08.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:28:08.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:28:08.113 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.113 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.113 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.113 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:28:08.113 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:08.113 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 69.963030692999993, 2023-12-06T18:28:08.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:28:08.114 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.114 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.114 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.114 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:08.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.114 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:08.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.115 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.115 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:28:08.115 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:28:08.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.115 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.115 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.115 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:28:08.115 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:28:08.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.115 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.115 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.115 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:28:08.115 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:28:08.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.115 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.115 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.116 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:28:08.116 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:28:08.116 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.116 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.116 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.116 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:28:08.116 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:28:08.116 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.116 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.116 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.116 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:28:08.116 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:28:08.116 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.116 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.116 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.116 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:28:08.117 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:28:08.117 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.117 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.117 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:28:08.117 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:28:08.117 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.117 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.117 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.117 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.117 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:28:08.117 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:08.117 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 69.583825077, 2023-12-06T18:28:08.117 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:28:08.118 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.118 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.118 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:08.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:08.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:28:08.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:28:08.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:28:08.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:28:08.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:28:08.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:28:08.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:28:08.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:28:08.120 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.120 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.120 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:28:08.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:28:08.120 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.120 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.120 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:28:08.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:28:08.120 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.120 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.120 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:28:08.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:28:08.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:28:08.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:28:08.121 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.121 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.121 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.121 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:28:08.121 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:08.121 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 69.346322895, 2023-12-06T18:28:08.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:28:08.121 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.122 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.122 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.122 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.122 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:08.122 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.122 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.122 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:08.122 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.122 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:28:08.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:28:08.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:28:08.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:28:08.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:28:08.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:28:08.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:28:08.124 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:28:08.124 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.124 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.124 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.124 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:28:08.124 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:28:08.124 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.124 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.124 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.124 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:28:08.124 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:28:08.124 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.124 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.124 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.124 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:28:08.124 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:28:08.124 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.125 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.125 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.125 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:28:08.125 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:28:08.125 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.125 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.125 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.125 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.125 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:28:08.125 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:08.125 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 69.345594895999994, 2023-12-06T18:28:08.125 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:28:08.125 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.125 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.126 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.126 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.126 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.126 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:08.126 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.126 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.126 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.126 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.126 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:08.126 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.126 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.126 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.126 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.126 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:28:08.126 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:28:08.127 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.127 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.127 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.127 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:28:08.127 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:28:08.127 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.127 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.127 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.127 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:28:08.127 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:28:08.127 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.127 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.127 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.127 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:28:08.127 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:28:08.128 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.128 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.128 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.128 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:28:08.128 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:28:08.128 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.128 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.128 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.128 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:28:08.128 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:28:08.128 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.128 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.128 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.128 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:28:08.128 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:28:08.129 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.129 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.129 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.129 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:28:08.129 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:28:08.129 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.129 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.129 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.129 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.129 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.129 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:28:08.129 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:08.129 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 69.340582486000002, 2023-12-06T18:28:08.129 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:28:08.129 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.129 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.130 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.130 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.130 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.130 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:08.130 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.130 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.130 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.130 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.130 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:08.130 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.130 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.130 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.130 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.130 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:28:08.130 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:28:08.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.131 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.131 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.131 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:28:08.131 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:28:08.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.131 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.131 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.131 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:28:08.131 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:28:08.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.131 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.131 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.131 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:28:08.131 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:28:08.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.132 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.132 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.132 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:28:08.132 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:28:08.132 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.132 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.132 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.132 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:28:08.132 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:28:08.132 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.132 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.132 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.132 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:28:08.132 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:28:08.132 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.133 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.133 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.133 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:28:08.133 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:28:08.133 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.133 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.133 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.133 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.133 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.133 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:28:08.133 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:08.133 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 69.335055406999999, 2023-12-06T18:28:08.133 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:28:08.133 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.133 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.133 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.134 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.134 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.134 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:08.134 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.134 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.134 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.134 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.134 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:08.134 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.134 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.134 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.134 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.134 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:28:08.134 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:28:08.134 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.135 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.135 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.135 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:28:08.135 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:28:08.135 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.135 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.135 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.135 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:28:08.135 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:28:08.135 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.135 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.135 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.135 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:28:08.135 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:28:08.135 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.135 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.136 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.136 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:28:08.136 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:28:08.136 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.136 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.136 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.136 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:28:08.136 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:28:08.136 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.136 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.136 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.136 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:28:08.136 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:28:08.136 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.136 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.137 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.137 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:28:08.137 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:28:08.137 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.137 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.137 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.137 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.137 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.137 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:28:08.137 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:08.137 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 69.334544570000006, 2023-12-06T18:28:08.137 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:28:08.137 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.137 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.138 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.138 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:08.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.138 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:08.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.138 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:28:08.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:28:08.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:28:08.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:28:08.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:28:08.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:28:08.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.140 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.140 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.140 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:28:08.140 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:28:08.140 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.140 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.140 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.140 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:28:08.140 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:28:08.140 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.140 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.140 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.140 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:28:08.140 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:28:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:28:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:28:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:28:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:28:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:28:08.142 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:08.142 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 69.334085309000002, 2023-12-06T18:28:08.142 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:28:08.142 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.142 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.142 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.142 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.142 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.142 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:08.142 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.142 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.142 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.142 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.143 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:08.143 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.143 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.143 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.143 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.143 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:28:08.143 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:28:08.143 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.143 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.143 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.143 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:28:08.143 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:28:08.143 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.144 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.144 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.144 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:28:08.144 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:28:08.144 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.144 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.144 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.144 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:28:08.144 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:28:08.144 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.144 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.144 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.144 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:28:08.144 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:28:08.144 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.144 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.145 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.145 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:28:08.145 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:28:08.145 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.145 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.145 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.145 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:28:08.145 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:28:08.145 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.145 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.145 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.145 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:28:08.145 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:28:08.145 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.145 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.145 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.146 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.146 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.146 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:28:08.146 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:08.146 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 62.754169502000003, 2023-12-06T18:28:08.146 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:28:08.146 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.146 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.146 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.146 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.146 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.146 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:08.146 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.146 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.147 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.147 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:08.147 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.147 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.147 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.147 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:28:08.147 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:28:08.147 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.147 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.147 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:28:08.147 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:28:08.147 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.148 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:28:08.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:28:08.148 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.148 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.148 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:28:08.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:28:08.148 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.148 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.148 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:28:08.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:28:08.148 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.148 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.149 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:28:08.149 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:28:08.149 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.149 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.149 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.149 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:28:08.149 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:28:08.149 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.149 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.149 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.149 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:28:08.149 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:28:08.149 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.149 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.149 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.150 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:28:08.150 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:08.150 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 62.747931696999999, 2023-12-06T18:28:08.150 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:28:08.150 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.150 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.150 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.150 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.150 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:08.150 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.151 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.151 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:08.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.151 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.151 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.151 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:28:08.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:28:08.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.151 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.151 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.151 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:28:08.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:28:08.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.151 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.152 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.152 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:28:08.152 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:28:08.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.152 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.152 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.152 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:28:08.152 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:28:08.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.152 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.152 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.152 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:28:08.152 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:28:08.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.152 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.153 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.153 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:28:08.153 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:28:08.153 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.153 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.153 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.153 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:28:08.153 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:28:08.153 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.153 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.153 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.153 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:28:08.153 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:28:08.153 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.153 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.153 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.154 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.154 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.154 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:28:08.154 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:08.154 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 62.741868617999998, 2023-12-06T18:28:08.154 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:28:08.154 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.154 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.154 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.154 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.154 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.154 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:08.154 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.154 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.154 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.155 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.155 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:08.155 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.155 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.155 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.155 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:28:08.155 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:28:08.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.155 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.155 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.155 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:28:08.155 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:28:08.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.155 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.155 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.156 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:28:08.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:28:08.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.156 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.156 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.156 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:28:08.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:28:08.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.156 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.156 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.156 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:28:08.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:28:08.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.156 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.156 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.156 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:28:08.157 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:28:08.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.157 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.157 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.157 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:28:08.157 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:28:08.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.157 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.157 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.157 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:28:08.157 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:28:08.157 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.157 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.157 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.158 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.158 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:28:08.158 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:08.158 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 62.604481532999998, 2023-12-06T18:28:08.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:28:08.158 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.158 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.158 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.158 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.158 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.158 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:08.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.158 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.158 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.158 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:08.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:28:08.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:28:08.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:28:08.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:28:08.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:28:08.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:28:08.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.160 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.160 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:28:08.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:28:08.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.160 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.160 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:28:08.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:28:08.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.160 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.160 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:28:08.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:28:08.161 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.161 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.161 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:28:08.161 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:28:08.161 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.161 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.161 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:28:08.161 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:28:08.161 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.161 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.161 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.161 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.162 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:28:08.162 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:08.162 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 62.597810805999998, 2023-12-06T18:28:08.162 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:28:08.162 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:08.162 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:08.162 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:08.162 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.162 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:08.162 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:08.163 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.163 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.163 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.163 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:08.163 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:08.163 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:08.163 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.163 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.163 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:08.163 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:28:08.163 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:28:08.163 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.163 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.163 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:08.163 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:28:08.163 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:28:08.164 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.164 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.164 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:08.164 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:28:08.164 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:28:08.164 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.164 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.164 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:08.164 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:28:08.164 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:28:08.164 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.164 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.164 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:08.164 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:28:08.164 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:28:08.164 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.165 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.165 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:08.165 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:28:08.165 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:28:08.165 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.165 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.165 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:08.165 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:28:08.165 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:28:08.165 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:08.165 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:08.165 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:08.165 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:28:08.165 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:28:08.165 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.165 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.166 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.166 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:08.166 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:08.166 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:08.167 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:28:08.168 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:08.164+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:08.269 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:08.265+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:08.299 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:08.299 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:08.299 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:08.299 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:08.308 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:28:08.369 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:08.366+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:08.440 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:08.440 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:08.440 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:08.440 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:08.440 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:08.450 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:28:08.470 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:08.466+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:08.472 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:28:08.570 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:08.566+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:08.577 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:08.577 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:08.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:08.577 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:28:08.577 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:08.587 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:28:08.671 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:08.667+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:08.728 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:08.728 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:08.728 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:08.728 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:08.738 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:28:08.771 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:08.768+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:08.872 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:08.869+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:08.886 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:08.886 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:08.886 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:08.886 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:08.886 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:08.895 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:28:08.973 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:08.970+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:09.074 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:09.070+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:09.091 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:09.092 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:09.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:09.092 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:09.092 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:09.102 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:28:09.174 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:09.171+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:09.239 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:09.239 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:09.239 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:09.239 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:09.249 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:28:09.291 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:09.287+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:09.376 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:09.373+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:09.427 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:09.427 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:09.427 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:09.428 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:09.428 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:09.437 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:28:09.477 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:09.473+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:09.578 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:09.575+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:09.635 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:09.635 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:09.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:09.636 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:09.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.636 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:28:09.636 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:09.636 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 74.223791234999993, 2023-12-06T18:28:09.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:28:09.636 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.636 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.636 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.636 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.636 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.636 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:28:09.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.636 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:09.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:09.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:28:09.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:28:09.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:28:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:28:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:28:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:28:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:28:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:28:09.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:28:09.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:28:09.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:28:09.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:28:09.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:28:09.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:28:09.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:28:09.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:28:09.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:28:09.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:28:09.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:28:09.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:28:09.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:28:09.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:28:09.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:28:09.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:28:09.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:28:09.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:28:09.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:28:09.641 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.642 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.642 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.642 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.642 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:09.642 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:09.642 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 72.115217995999998, 2023-12-06T18:28:09.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:28:09.642 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.642 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.642 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.642 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.642 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.642 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:28:09.643 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.643 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.643 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.643 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.643 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:09.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.643 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.643 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.643 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.643 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:09.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.643 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.643 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.643 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.643 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:28:09.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:28:09.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:28:09.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:28:09.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:28:09.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:28:09.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.645 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.645 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:28:09.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:28:09.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.645 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.645 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:28:09.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:28:09.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.645 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.645 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:28:09.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:28:09.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:28:09.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:28:09.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:28:09.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:28:09.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:28:09.647 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:28:09.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.647 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:28:09.647 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:28:09.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.647 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:28:09.647 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:28:09.647 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.648 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.648 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.648 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.648 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.648 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:09.648 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:09.648 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 72.114472763999999, 2023-12-06T18:28:09.648 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:28:09.648 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.648 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.648 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.648 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.648 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.648 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:28:09.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.649 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.649 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.649 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.649 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:09.649 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.649 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.649 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.649 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:09.649 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.649 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.649 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.650 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:28:09.650 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:28:09.650 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.650 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.650 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:28:09.650 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:28:09.650 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.650 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.650 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:28:09.650 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:28:09.650 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.651 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.651 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:28:09.651 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:28:09.651 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.651 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.651 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.651 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:28:09.651 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:28:09.651 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.651 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.651 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.651 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:28:09.651 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:28:09.651 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.651 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.651 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.652 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:28:09.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:28:09.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.652 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:28:09.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:28:09.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.652 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:28:09.653 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:28:09.653 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.653 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.653 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.653 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:28:09.653 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:28:09.653 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.653 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.653 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.653 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:28:09.653 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:28:09.653 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.653 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.653 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.653 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.654 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.654 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:09.654 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:09.654 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 72.113789009000001, 2023-12-06T18:28:09.654 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:28:09.654 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.654 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.654 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.654 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.654 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.654 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:28:09.654 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.654 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.655 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.655 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.655 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:09.655 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.655 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.655 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.655 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.655 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:09.655 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.655 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.655 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.655 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.655 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:28:09.655 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:28:09.655 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.656 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.656 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.656 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:28:09.656 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:28:09.656 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.656 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.656 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.656 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:28:09.656 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:28:09.656 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.656 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.656 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.656 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:28:09.656 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:28:09.656 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.657 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.657 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.657 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:28:09.657 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:28:09.657 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.657 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.657 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.657 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:28:09.657 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:28:09.657 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.657 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.657 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.657 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:28:09.657 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:28:09.657 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.658 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:28:09.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:28:09.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.658 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:28:09.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:28:09.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.658 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:28:09.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:28:09.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.659 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:28:09.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:28:09.659 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.659 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.659 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.659 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:09.659 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:09.659 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 72.108215754, 2023-12-06T18:28:09.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:28:09.659 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.660 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.660 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.660 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.660 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.660 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:28:09.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.660 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.660 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.660 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:09.660 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.661 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.661 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:09.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.661 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.661 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.661 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:28:09.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:28:09.661 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.661 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.661 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:28:09.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:28:09.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.662 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.662 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.662 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:28:09.662 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:28:09.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.662 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.662 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.662 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:28:09.662 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:28:09.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.662 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.662 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.663 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:28:09.663 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:28:09.663 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.663 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.663 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.663 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:28:09.663 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:28:09.663 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.663 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.663 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.663 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:28:09.663 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:28:09.663 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.663 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.663 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.664 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:28:09.664 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:28:09.664 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.664 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.664 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.664 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:28:09.664 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:28:09.664 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.664 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.664 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.664 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:28:09.664 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:28:09.664 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.664 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.665 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.665 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:28:09.665 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:28:09.665 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.665 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.665 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.665 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.665 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.665 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:09.665 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:09.665 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 72.107542113999997, 2023-12-06T18:28:09.665 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:28:09.665 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.666 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.666 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.666 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.666 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.666 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:28:09.666 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.666 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.666 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.666 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.666 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:09.666 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.666 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.666 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.667 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.667 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:09.667 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.667 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.667 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.667 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:28:09.667 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:28:09.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.667 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.667 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.667 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:28:09.667 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:28:09.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.667 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.668 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.668 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:28:09.668 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:28:09.668 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.668 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.668 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:28:09.668 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:28:09.668 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.668 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.668 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:28:09.668 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:28:09.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.669 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.669 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:28:09.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:28:09.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.669 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.670 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:28:09.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:28:09.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.670 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.670 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:28:09.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:28:09.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.670 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.670 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:28:09.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:28:09.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:28:09.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:28:09.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:28:09.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:28:09.671 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.671 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.671 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.671 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:09.671 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:09.672 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 72.106879914999993, 2023-12-06T18:28:09.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:28:09.672 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.672 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.672 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.672 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.672 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.672 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:28:09.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.672 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:09.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.673 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.673 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.673 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:09.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.673 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.673 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.673 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.673 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:28:09.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:28:09.673 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.673 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.673 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.673 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:28:09.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:28:09.673 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:28:09.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:28:09.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:28:09.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:28:09.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:28:09.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:28:09.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:28:09.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:28:09.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:28:09.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:28:09.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:28:09.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:28:09.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.676 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.676 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.676 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:28:09.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:28:09.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.676 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.676 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.676 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:28:09.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:28:09.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.676 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.676 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.676 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:28:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:28:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 72.03417048, 2023-12-06T18:28:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:28:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:28:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:28:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:28:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:28:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:28:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:28:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:28:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:28:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:28:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:28:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:28:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:28:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:28:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:28:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:28:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:28:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:28:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:28:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:28:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:28:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:28:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:28:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:28:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 72.033488770999995, 2023-12-06T18:28:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:28:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:28:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:28:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:28:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:28:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:28:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:28:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:28:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:28:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:28:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:28:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:28:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:28:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:28:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:28:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:28:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:28:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:28:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:28:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:28:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:28:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:28:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:28:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:28:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 70.954684682999996, 2023-12-06T18:28:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:28:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:28:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:28:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:28:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:28:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:28:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:28:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:28:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:28:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:28:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:28:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:28:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:28:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:28:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:28:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:28:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:28:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:28:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:28:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:28:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:28:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:28:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:28:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:28:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 70.953268109000007, 2023-12-06T18:28:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:28:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:28:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:28:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:28:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:28:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:28:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:28:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:28:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:28:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:28:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:28:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:28:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:28:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:28:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:28:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:28:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:28:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:28:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:28:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:28:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:28:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:28:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:28:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:28:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 70.952182984000004, 2023-12-06T18:28:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:28:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:28:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:28:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:28:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:28:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:28:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:28:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:28:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:28:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:28:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:28:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:28:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:28:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:28:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:28:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:28:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:28:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:28:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:28:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:28:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:28:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:28:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:28:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:28:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 70.917033881999998, 2023-12-06T18:28:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:28:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:28:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:28:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:28:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:28:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:28:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:28:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:28:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:28:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:28:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:28:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:28:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:28:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:28:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:28:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:28:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:28:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:28:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:28:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:28:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:28:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:28:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:28:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:28:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 64.367303356999997, 2023-12-06T18:28:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:28:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:28:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:28:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:28:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:28:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:28:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:28:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:28:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:28:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:28:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:28:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:28:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:28:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:28:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:28:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:28:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:28:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:28:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:28:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:28:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:28:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:28:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:28:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:28:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 64.366497749000004, 2023-12-06T18:28:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:28:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:28:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:28:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:28:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:28:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:28:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:28:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:28:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:28:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:28:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:28:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:28:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:28:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:28:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:28:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:28:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:28:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:28:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:28:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:28:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:28:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:28:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:28:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:28:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 64.365806939999999, 2023-12-06T18:28:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:28:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:28:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:28:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:28:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:28:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:28:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:28:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:28:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:28:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:28:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:28:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:28:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:28:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:28:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:28:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:28:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:28:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:28:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:28:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:28:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:28:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:28:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:28:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:28:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 64.361509432000005, 2023-12-06T18:28:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:28:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:28:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:28:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:28:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:28:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:28:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:28:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:28:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:28:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:28:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:28:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:28:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:28:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:28:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:28:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:28:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:28:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:28:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:28:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:28:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:28:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:28:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:28:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:28:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 64.360841757000003, 2023-12-06T18:28:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:28:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:28:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:28:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:28:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:28:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:28:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:28:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:28:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:28:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:28:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:28:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:28:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:28:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:28:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:28:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:28:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:28:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:28:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:28:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:28:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:28:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:28:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:28:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:28:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 64.360186329000001, 2023-12-06T18:28:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:28:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:28:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:28:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:28:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:28:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:28:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:28:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:28:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:28:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:28:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:28:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:28:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:28:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:28:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:28:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:28:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:28:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:28:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:28:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:28:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:28:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:28:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:28:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:28:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 64.319531781999999, 2023-12-06T18:28:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:28:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:28:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:28:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:28:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:28:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:28:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:28:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:28:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:28:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:28:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:28:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:28:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:28:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:28:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:28:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:28:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:28:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:28:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:28:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:28:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:28:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:28:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:28:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:28:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:09.748 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:09.749 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:28:09.750 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:09.746+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:09.851 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:09.847+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:09.889 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:09.889 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:09.889 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:09.890 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:09.898 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:28:09.984 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:09.980+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:10.064 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:10.060+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:10.080 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:10.080 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:10.080 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:10.080 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:10.081 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:10.090 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:28:10.157 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:10.152+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:10.255 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:10.252+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:10.272 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:10.272 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:10.273 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:10.273 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:10.273 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:10.273 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:28:10.273 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:10.273 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 48.456909062999998, 2023-12-06T18:28:10.273 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:28:10.273 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:10.273 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:28:10.273 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:10.273 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:10.273 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:10.273 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:10.273 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:10.273 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:10.273 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:10.274 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:10.274 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:10.274 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:10.274 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:10.274 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:10.274 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:10.274 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:28:10.274 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:28:10.274 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:10.274 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:10.274 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:10.274 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:28:10.274 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:28:10.274 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:10.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:10.275 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:10.275 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:28:10.275 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:28:10.275 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:10.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:10.275 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:10.275 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:28:10.275 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:28:10.275 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:10.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:10.275 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:10.275 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:28:10.275 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:28:10.275 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:10.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:10.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:10.276 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:28:10.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:28:10.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:10.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:10.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:10.276 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:28:10.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:28:10.276 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:10.276 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:10.276 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:10.276 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:10.276 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:10.276 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:10.282 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:28:10.367 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:10.363+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:10.457 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:10.453+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:10.464 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:10.464 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:10.464 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:10.464 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:10.473 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:28:10.558 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:10.554+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:10.652 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:10.652 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:10.652 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:10.652 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:10.652 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:10.662 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:28:10.662 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:10.656+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:10.759 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:10.756+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:10.840 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:10.841 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:10.841 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:10.841 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:10.841 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:10.849 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:28:10.894 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:10.890+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:10.986 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:10.982+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:11.001 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:11.001 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:11.001 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:11.001 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:11.010 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:28:11.091 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:11.087+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:11.163 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:11.160+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:11.181 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:11.181 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:11.181 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:11.181 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:11.181 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:11.190 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:28:11.265 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:11.261+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:11.366 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:11.363+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:11.366 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:11.367 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:11.367 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:11.367 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:28:11.367 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:11.375 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:28:11.467 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:11.463+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:11.505 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:11.505 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:11.505 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:11.505 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:11.515 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:28:11.568 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:11.564+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:11.642 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:11.642 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:11.642 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:11.642 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:11.642 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:11.651 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:28:11.668 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:11.665+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:11.769 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:11.765+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:11.862 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:28:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:28:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 75.246566744999996, 2023-12-06T18:28:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:28:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:11.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:28:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:28:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:28:11.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:28:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:28:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:28:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:28:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:28:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:28:11.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:28:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:28:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:28:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:28:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:28:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:28:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:28:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:28:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 74.223739735999999, 2023-12-06T18:28:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:28:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:28:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:28:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:28:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:28:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:28:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:28:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:28:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:28:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:28:11.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:28:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:28:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:28:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:28:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:28:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:28:11.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:28:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:28:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 74.211693097999998, 2023-12-06T18:28:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:28:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.871 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:28:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:28:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:28:11.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:28:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:28:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:28:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:28:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:28:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:28:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:28:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:28:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:28:11.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:28:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:28:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:28:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:28:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.875 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:28:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 74.190292490999994, 2023-12-06T18:28:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:28:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:11.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:28:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:28:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:28:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:28:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:28:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:28:11.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:28:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:28:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:28:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:28:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:28:11.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:28:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:28:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:28:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:28:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:28:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:28:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 74.189113168000006, 2023-12-06T18:28:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:28:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:28:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:28:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:28:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:28:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:28:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:28:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:28:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:28:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:28:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:28:11.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:28:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:28:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:28:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:28:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:28:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:28:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.883 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:28:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 74.188541186999998, 2023-12-06T18:28:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:28:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.884 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:28:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:28:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:28:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:28:11.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:28:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:28:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:28:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:28:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:28:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:28:11.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:28:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:28:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:28:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:28:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:28:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:28:11.887 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:28:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 73.742705717000007, 2023-12-06T18:28:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:28:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:28:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:28:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:28:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:28:11.889 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:28:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:28:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:28:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:28:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:28:11.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:28:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:28:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:28:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:28:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:28:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:28:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:28:11.891 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:28:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 73.741105860999994, 2023-12-06T18:28:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:28:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:28:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:28:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:28:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:28:11.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:28:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:28:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:28:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:28:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:28:11.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:28:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:28:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:28:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:28:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:28:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:28:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:28:11.895 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:28:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 73.361900245000001, 2023-12-06T18:28:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:28:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:11.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:28:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:28:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:28:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:28:11.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:28:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:28:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:28:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:28:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:28:11.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:28:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:28:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:28:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:28:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:28:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:28:11.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:28:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:28:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 73.124398063000001, 2023-12-06T18:28:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:28:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:11.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:28:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:28:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:28:11.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:28:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:28:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:28:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:28:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:28:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:28:11.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:28:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:28:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:28:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:28:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:28:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:28:11.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:28:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:28:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 73.123670063999995, 2023-12-06T18:28:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:28:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:11.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:28:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:28:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:28:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:28:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:28:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:28:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:28:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:28:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:28:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:28:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:28:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:28:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:28:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:28:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:28:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:28:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.907 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:28:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 73.118657654000003, 2023-12-06T18:28:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:28:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:11.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:28:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:28:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:28:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:28:11.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:28:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:28:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:28:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:28:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:28:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:28:11.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:28:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:28:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:28:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:28:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:28:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:28:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.911 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:28:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 73.113130575, 2023-12-06T18:28:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:28:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:28:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:28:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:28:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:28:11.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:28:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:28:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:28:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:28:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:28:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:28:11.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:28:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:28:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:28:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:28:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:28:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:28:11.915 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:28:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 73.112619738000006, 2023-12-06T18:28:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:28:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:28:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:28:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:28:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:28:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:28:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:28:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:28:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:28:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:28:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:28:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:28:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:28:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:28:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:28:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:28:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:28:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.919 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:28:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 73.112160477000003, 2023-12-06T18:28:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:28:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:28:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:28:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:28:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:28:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:28:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:28:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:28:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:28:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:28:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:28:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:28:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:28:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:28:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:28:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:28:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:28:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:28:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 66.532244669999997, 2023-12-06T18:28:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:28:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:28:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:28:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:28:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:28:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:28:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:28:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:28:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:28:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:28:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:28:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:28:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:28:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:28:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:28:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:28:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:28:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:28:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 66.526006864999999, 2023-12-06T18:28:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:28:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:28:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:28:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:28:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:28:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:28:11.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:28:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:28:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:28:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:28:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:28:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:28:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:28:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:28:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:28:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:28:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:28:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:28:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 66.519943785999999, 2023-12-06T18:28:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:28:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:28:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:28:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:28:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:28:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:28:11.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:28:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:28:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:28:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:28:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:28:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:28:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:28:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:28:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:28:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:28:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:28:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.935 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:28:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 66.382556700999999, 2023-12-06T18:28:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:28:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.936 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:28:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:28:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:28:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:28:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:28:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:28:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:28:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:28:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:28:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:28:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:28:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:28:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:28:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:28:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:28:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:28:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:28:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:11.939 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 66.375885973999999, 2023-12-06T18:28:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:28:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:28:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:28:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:28:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:28:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:28:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:28:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:28:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:28:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:28:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:28:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:28:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:28:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:28:11.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:28:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:28:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:28:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:11.943 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:11.943 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:11.945 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:28:11.945 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:11.941+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:12.046 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:12.042+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:12.119 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:12.120 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:12.120 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:12.120 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:12.130 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:28:12.172 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:12.168+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:12.247 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:12.244+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:12.298 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:12.298 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:12.298 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:12.298 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:12.298 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:12.307 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:28:12.348 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:12.345+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:12.449 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:12.446+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:12.477 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:12.477 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:12.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:12.477 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:28:12.477 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:12.486 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:28:12.550 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:12.547+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:12.635 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:12.636 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:12.636 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:12.636 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:12.645 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:28:12.689 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:12.685+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:12.751 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:12.748+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:12.783 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:12.783 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:12.783 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:12.783 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:12.783 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:12.792 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:28:12.853 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:12.849+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:12.954 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:12.950+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:12.968 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:12.968 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:12.968 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:12.968 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:12.968 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:12.979 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:28:13.064 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:13.060+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:13.156 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:13.156 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:13.156 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:13.156 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:13.156 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:13.152+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:13.165 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:28:13.257 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:13.253+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:13.346 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:13.346 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:13.346 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:13.346 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:13.346 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:13.355 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:28:13.397 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:13.393+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:13.458 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:13.455+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:13.559 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:13.556+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:13.594 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:28:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 78.180632504000002, 2023-12-06T18:28:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:28:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:28:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:28:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:28:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:28:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:28:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:28:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:28:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:28:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:28:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:28:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:28:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:28:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:28:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:28:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:28:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:28:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:28:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:28:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:28:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:28:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:28:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:28:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:28:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:28:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.600 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:28:13.600 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:28:13.600 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.600 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.600 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.600 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:28:13.600 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:28:13.600 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.600 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.600 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.600 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.600 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.600 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:13.600 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:13.600 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 76.072059264999993, 2023-12-06T18:28:13.601 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:28:13.601 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.601 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.601 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.601 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.601 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.601 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:28:13.601 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.601 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.601 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.601 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.601 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:13.602 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.602 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.602 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.602 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.602 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:13.602 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.602 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.602 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.602 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.602 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:28:13.602 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:28:13.602 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.602 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.602 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.602 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:28:13.603 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:28:13.603 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.603 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.603 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.603 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:28:13.603 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:28:13.603 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.603 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.603 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.603 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:28:13.603 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:28:13.603 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.603 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.603 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.603 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:28:13.604 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:28:13.604 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.604 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.604 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.604 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:28:13.604 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:28:13.604 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.604 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.604 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.604 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:28:13.604 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:28:13.604 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.604 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.604 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.604 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:28:13.604 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:28:13.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.605 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.605 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.605 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:28:13.605 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:28:13.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.605 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.605 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.605 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:28:13.605 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:28:13.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.605 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.605 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.605 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:28:13.605 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:28:13.605 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.606 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.606 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.606 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.606 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.606 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:13.606 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:13.606 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 76.071314032999993, 2023-12-06T18:28:13.606 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:28:13.606 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.606 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.606 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.606 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.606 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.606 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:28:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:28:13.608 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:28:13.608 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.608 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.608 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.608 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:28:13.608 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:28:13.608 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.608 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.608 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.608 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:28:13.608 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:28:13.608 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.608 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.608 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.608 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:28:13.608 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:28:13.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.609 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.609 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.609 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:28:13.609 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:28:13.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.609 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.609 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.609 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:28:13.609 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:28:13.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.609 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.609 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.609 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:28:13.609 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:28:13.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.610 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.610 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.610 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:28:13.610 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:28:13.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.610 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.610 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.610 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:28:13.610 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:28:13.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.610 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.610 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.610 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:28:13.610 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:28:13.611 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.611 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.611 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.611 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:28:13.611 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:28:13.611 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.611 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.611 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.611 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.611 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.611 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:13.611 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:13.611 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 76.070630277999996, 2023-12-06T18:28:13.611 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:28:13.611 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.612 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.612 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.612 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.612 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.612 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:28:13.612 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.612 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.612 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.612 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.612 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:13.612 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.612 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.612 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.612 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.612 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:13.612 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.613 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.613 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.613 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:28:13.613 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:28:13.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.613 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.613 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.613 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:28:13.613 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:28:13.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.613 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.613 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.613 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:28:13.613 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:28:13.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.614 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:28:13.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:28:13.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.614 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:28:13.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:28:13.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.614 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:28:13.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:28:13.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.615 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:28:13.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:28:13.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.615 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:28:13.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:28:13.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.615 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:28:13.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:28:13.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:28:13.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:28:13.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:28:13.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:28:13.616 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.616 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.616 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.616 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:13.617 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:13.617 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 76.065057022999994, 2023-12-06T18:28:13.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:28:13.617 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.617 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.617 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.617 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.617 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.617 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:28:13.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.617 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:13.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:13.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:28:13.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:28:13.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:28:13.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:28:13.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.619 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.619 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:28:13.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:28:13.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.619 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.619 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:28:13.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:28:13.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.619 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.619 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:28:13.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:28:13.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.620 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.620 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.620 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:28:13.620 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:28:13.620 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.620 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.620 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.620 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:28:13.620 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:28:13.620 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.620 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.620 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.620 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:28:13.620 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:28:13.621 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.621 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.621 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.621 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:28:13.621 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:28:13.621 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.621 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.621 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.621 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:28:13.621 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:28:13.621 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.621 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.621 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.621 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:28:13.621 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:28:13.621 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.622 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.622 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.622 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.622 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.622 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:13.622 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:13.622 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 76.064383383000006, 2023-12-06T18:28:13.622 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:28:13.622 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.622 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.622 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.622 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.622 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.622 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:28:13.623 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.623 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.623 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.623 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.623 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:13.623 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.623 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.623 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.623 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.623 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:13.623 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.623 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.623 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.623 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.623 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:28:13.623 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:28:13.624 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.624 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.624 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.624 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:28:13.624 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:28:13.624 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.624 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.624 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.624 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:28:13.624 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:28:13.624 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.624 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.624 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.624 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:28:13.624 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:28:13.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:28:13.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:28:13.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:28:13.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:28:13.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:28:13.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:28:13.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.626 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.626 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.626 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:28:13.626 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:28:13.626 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.626 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.626 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.626 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:28:13.626 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:28:13.626 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.626 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.626 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.626 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:28:13.626 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:28:13.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.627 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:28:13.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:28:13.627 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.627 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.627 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.627 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:13.627 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:13.627 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 76.063721184000002, 2023-12-06T18:28:13.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:28:13.627 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.627 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.628 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.628 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.628 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.628 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:28:13.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.628 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.628 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.628 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:13.628 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.628 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.628 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:13.628 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.629 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.629 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.629 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.629 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:28:13.629 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:28:13.629 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.629 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.629 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.629 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:28:13.629 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:28:13.629 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.629 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.629 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.629 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:28:13.629 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:28:13.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.630 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.630 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.630 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:28:13.630 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:28:13.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.630 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.630 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.630 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:28:13.630 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:28:13.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.630 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.630 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.630 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:28:13.630 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:28:13.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.631 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.631 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.631 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:28:13.631 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:28:13.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.631 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.631 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.631 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:28:13.631 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:28:13.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.631 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.631 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.631 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:28:13.631 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:28:13.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.632 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.632 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:28:13.632 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:28:13.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.632 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.632 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:28:13.632 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:28:13.632 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.632 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.632 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.633 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:13.633 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:13.633 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 75.991011748999995, 2023-12-06T18:28:13.633 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:28:13.633 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.633 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.633 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.633 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.633 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.633 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:28:13.633 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.633 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.633 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.633 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.633 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:13.633 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.634 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.634 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.634 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.634 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:13.634 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.634 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.634 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.634 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.634 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:28:13.634 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:28:13.634 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.634 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.634 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.634 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:28:13.634 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:28:13.635 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.635 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.635 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.635 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:28:13.635 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:28:13.635 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.635 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.635 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.635 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:28:13.635 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:28:13.635 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.635 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.635 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.635 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:28:13.635 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:28:13.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:28:13.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:28:13.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:28:13.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:28:13.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:28:13.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:28:13.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:28:13.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:28:13.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:28:13.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:28:13.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:28:13.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:28:13.637 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.638 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.638 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.638 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:13.638 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:13.638 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 75.990330040000003, 2023-12-06T18:28:13.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:28:13.638 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.638 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.638 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.638 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.638 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.638 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:28:13.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.639 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:13.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:13.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:28:13.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:28:13.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:28:13.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:28:13.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:28:13.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:28:13.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:28:13.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:28:13.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:28:13.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:28:13.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:28:13.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:28:13.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:28:13.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:28:13.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.642 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.642 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:28:13.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:28:13.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.642 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.642 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.642 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:28:13.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:28:13.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.642 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.642 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.642 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:28:13.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:28:13.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.642 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.643 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.643 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:28:13.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:28:13.643 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.643 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.643 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.643 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.643 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.643 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:13.643 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:13.643 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 74.911525952000005, 2023-12-06T18:28:13.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:28:13.643 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.643 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.643 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.644 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.644 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.644 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:28:13.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.644 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:13.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:13.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.645 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.645 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:28:13.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:28:13.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.645 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.645 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:28:13.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:28:13.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.645 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.645 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:28:13.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:28:13.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:28:13.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:28:13.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:28:13.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:28:13.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:28:13.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:28:13.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.647 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:28:13.647 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:28:13.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.647 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:28:13.647 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:28:13.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.647 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:28:13.647 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:28:13.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.648 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.648 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.648 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:28:13.648 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:28:13.648 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.648 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.648 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.648 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:28:13.648 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:28:13.648 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.648 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.648 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.648 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.648 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.648 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:13.649 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:13.649 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 74.910109378000001, 2023-12-06T18:28:13.649 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:28:13.649 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.649 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.649 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.649 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.649 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.649 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:28:13.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.649 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.649 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.649 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.649 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:13.649 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.650 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.650 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:13.650 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.650 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.650 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.650 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:28:13.650 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:28:13.650 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.650 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.650 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:28:13.650 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:28:13.651 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.651 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.651 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.651 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:28:13.651 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:28:13.651 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.651 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.651 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.651 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:28:13.651 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:28:13.651 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.651 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.651 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.651 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:28:13.651 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:28:13.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.652 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:28:13.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:28:13.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.652 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:28:13.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:28:13.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.652 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:28:13.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:28:13.653 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.653 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.653 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.653 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:28:13.653 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:28:13.653 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.653 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.653 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.653 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:28:13.653 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:28:13.653 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.653 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.653 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.653 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:28:13.653 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:28:13.654 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.654 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.654 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.654 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.654 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.654 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:13.654 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:13.654 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 74.909024252999998, 2023-12-06T18:28:13.654 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:28:13.654 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.654 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.654 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.654 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.654 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.654 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:28:13.655 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.655 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.655 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.655 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.655 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:13.655 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.655 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.655 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.655 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.655 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:13.655 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.655 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.655 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.655 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.655 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:28:13.656 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:28:13.656 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.656 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.656 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.656 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:28:13.656 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:28:13.656 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.656 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.656 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.656 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:28:13.656 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:28:13.656 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.656 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.656 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.656 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:28:13.657 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:28:13.657 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.657 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.657 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.657 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:28:13.657 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:28:13.657 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.657 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.657 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.657 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:28:13.657 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:28:13.657 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.657 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.657 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.657 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:28:13.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:28:13.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.658 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:28:13.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:28:13.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.658 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:28:13.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:28:13.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.658 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:28:13.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:28:13.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.659 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:28:13.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:28:13.659 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.659 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.659 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.659 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:13.659 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:13.659 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 74.873875150999993, 2023-12-06T18:28:13.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:28:13.660 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.660 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.660 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.660 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.660 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.660 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:28:13.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.660 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.660 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.660 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:13.660 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.660 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:13.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.661 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.661 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.661 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:28:13.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:28:13.661 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.661 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.661 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:28:13.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:28:13.661 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.661 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.661 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:28:13.662 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:28:13.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.662 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.662 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.662 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:28:13.662 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:28:13.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.662 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.662 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.662 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:28:13.662 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:28:13.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.662 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.662 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.662 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:28:13.663 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:28:13.663 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.663 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.663 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.663 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:28:13.663 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:28:13.663 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.663 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.663 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.663 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:28:13.663 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:28:13.663 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.663 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.663 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.663 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:28:13.664 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:28:13.664 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.664 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.664 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.664 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:28:13.664 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:28:13.664 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.664 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.664 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.664 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:28:13.664 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:28:13.664 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.664 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.664 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.664 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.664 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.665 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:13.665 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:13.665 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 68.324144626000006, 2023-12-06T18:28:13.665 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:28:13.665 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.665 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.665 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.665 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.665 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.665 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:28:13.665 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.665 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.665 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.665 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.665 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:13.666 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.666 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.666 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.666 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.666 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:13.666 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.666 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.666 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.666 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.666 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:28:13.666 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:28:13.666 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.666 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.666 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.666 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:28:13.667 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:28:13.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.667 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.667 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.667 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:28:13.667 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:28:13.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.667 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.667 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.667 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:28:13.667 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:28:13.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.667 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.667 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.667 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:28:13.667 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:28:13.668 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.668 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.668 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:28:13.668 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:28:13.668 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.668 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.668 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:28:13.668 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:28:13.668 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.668 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.668 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:28:13.668 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:28:13.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.669 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.669 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:28:13.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:28:13.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.669 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.669 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:28:13.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:28:13.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.669 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.669 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:28:13.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:28:13.670 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.670 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.670 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.670 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:13.670 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:13.670 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 68.323339017999999, 2023-12-06T18:28:13.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:28:13.670 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.670 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.670 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.670 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.670 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.670 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:28:13.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.671 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:13.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:13.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:28:13.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:28:13.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:28:13.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:28:13.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:28:13.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:28:13.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:28:13.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:28:13.673 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.673 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.673 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.673 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:28:13.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:28:13.673 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.673 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.673 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.673 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:28:13.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:28:13.673 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.673 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.673 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.673 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:28:13.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:28:13.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:28:13.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:28:13.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:28:13.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:28:13.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:28:13.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:28:13.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:28:13.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:28:13.675 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.675 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.675 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.675 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:13.675 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:13.675 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 68.322648208999993, 2023-12-06T18:28:13.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:28:13.675 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.675 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.676 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.676 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.676 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.676 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:28:13.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.676 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.676 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.676 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.676 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:13.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.676 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.676 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.676 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:13.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:28:13.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:28:13.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:28:13.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:28:13.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:28:13.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:28:13.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:28:13.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:28:13.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:28:13.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:28:13.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:28:13.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:28:13.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:28:13.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:28:13.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:28:13.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:28:13.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:28:13.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:28:13.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:28:13.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:28:13.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:28:13.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:28:13.680 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.680 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.680 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.681 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:13.681 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:13.681 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 68.318350701, 2023-12-06T18:28:13.681 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:28:13.681 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.681 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.681 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.681 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.681 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.681 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:28:13.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.681 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.681 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:13.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.682 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.682 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:13.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.682 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.682 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:28:13.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:28:13.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.682 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.682 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.683 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:28:13.683 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:28:13.683 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.683 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.683 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.683 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:28:13.683 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:28:13.683 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.683 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.683 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.683 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:28:13.683 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:28:13.683 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.683 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.683 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.684 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:28:13.684 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:28:13.684 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.684 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.684 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.684 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:28:13.684 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:28:13.684 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.684 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.684 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.684 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:28:13.684 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:28:13.684 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.684 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.684 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:28:13.685 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:28:13.685 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:28:13.685 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:28:13.685 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:28:13.685 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:28:13.685 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:28:13.686 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:28:13.686 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.686 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.686 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.686 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.686 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.686 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:13.686 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:13.686 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 68.317683025999997, 2023-12-06T18:28:13.686 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:28:13.686 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.686 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.686 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.686 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.686 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.687 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:28:13.687 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.687 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.687 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.687 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.687 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:13.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.687 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.687 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.687 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.687 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:13.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.687 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.687 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.687 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.687 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:28:13.688 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:28:13.688 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.688 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.688 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:28:13.688 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:28:13.688 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.688 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.688 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:28:13.688 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:28:13.688 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.688 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.688 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:28:13.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:28:13.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.689 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.689 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:28:13.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:28:13.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.689 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.689 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:28:13.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:28:13.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.689 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.689 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:28:13.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:28:13.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:28:13.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:28:13.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:28:13.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:28:13.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:28:13.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:28:13.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:28:13.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:28:13.691 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.691 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.691 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.691 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:13.691 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:13.691 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 68.317027597999996, 2023-12-06T18:28:13.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:28:13.691 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.692 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.692 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.692 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.692 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.692 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:28:13.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.692 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:13.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:13.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:28:13.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:28:13.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:28:13.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:28:13.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:28:13.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:28:13.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:28:13.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:28:13.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:28:13.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:28:13.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:28:13.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:28:13.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:28:13.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:28:13.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:28:13.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:28:13.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:28:13.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:28:13.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:28:13.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:28:13.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:28:13.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:28:13.696 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.696 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.696 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.697 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:13.697 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:13.697 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 68.276373050999993, 2023-12-06T18:28:13.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:28:13.697 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:13.697 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:13.697 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:13.697 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:13.697 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:13.697 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:28:13.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.697 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:13.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:13.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:13.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:13.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:13.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:13.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:13.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:28:13.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:28:13.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:13.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:28:13.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:28:13.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:13.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:28:13.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:28:13.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:13.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:28:13.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:28:13.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:13.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:28:13.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:28:13.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:13.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:28:13.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:28:13.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:13.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:28:13.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:28:13.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:13.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:28:13.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:28:13.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:13.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:28:13.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:28:13.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:13.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:28:13.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:28:13.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:13.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:13.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:13.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:28:13.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:28:13.702 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.702 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.702 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.702 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:13.702 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:13.702 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:13.703 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:28:13.704 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:13.700+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:13.805 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:13.801+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:13.881 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:13.881 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:13.881 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:13.881 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:13.890 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:28:13.906 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:13.903+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:13.979 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":18,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:28:13.995 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2023-12-06T18:28:13.995 INFO:tasks.ceph.mon.c:Restarting daemon 2023-12-06T18:28:13.995 DEBUG:teuthology.orchestra.run.smithi061:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2023-12-06T18:28:13.997 INFO:tasks.ceph.mon.c:Started 2023-12-06T18:28:13.997 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2023-12-06T18:28:13.998 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:28:14.006 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-12-06T18:28:14.081 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:14.077+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:14.095 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:14.095 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:14.095 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:14.095 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:14.095 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:14.105 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:28:14.156 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:14.152+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:14.221 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:28:14.222 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:28:14.222 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:14.218+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:14.310 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:14.306+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:14.310 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:14.310 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:28:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 52.492362995999997, 2023-12-06T18:28:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:28:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:28:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:28:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:28:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:28:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:28:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:28:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:28:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:28:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:28:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:28:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:28:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:28:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:28:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:28:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:28:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:14.315 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:14.319 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:28:14.339 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:28:14.339 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:28:14.413 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:14.407+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:14.435 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:28:14.435 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:28:14.512 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:14.508+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:14.513 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:14.513 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:14.513 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:14.513 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:14.526 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:28:14.584 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:28:14.584 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:28:14.612 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:14.609+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:14.685 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:28:14.686 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:28:14.713 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:14.710+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:14.714 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:14.714 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:14.725 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:28:14.774 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":18,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:28:14.788 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:28:14.788 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:28:14.815 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:14.811+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:14.888 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:28:14.888 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:28:14.905 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:14.905 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:14.917 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:14.912+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:14.918 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:28:15.018 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:15.014+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:15.046 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:15.046 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:15.057 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:28:15.118 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:15.115+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:15.187 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:15.188 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:15.197 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:28:15.238 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:15.235+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:15.251 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is unset 2023-12-06T18:28:15.321 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:15.317+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:15.369 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:15.370 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:15.379 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:28:15.422 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:15.418+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:15.508 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:15.508 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:15.518 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:28:15.563 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:15.559+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:15.624 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:15.620+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:15.648 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:15.648 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:15.657 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:28:15.725 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:15.721+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:15.825 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:15.825 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:15.826 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:15.822+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:15.834 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:28:15.927 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:15.923+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:15.961 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:15.962 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:15.971 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:28:16.027 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:16.024+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:16.100 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:16.100 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:16.110 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:28:16.128 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:16.125+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:16.230 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:16.226+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:16.279 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:16.279 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:16.288 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:28:16.330 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:16.327+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:16.431 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:16.427+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:16.442 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:16.442 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:16.451 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:28:16.532 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:16.528+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:16.630 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:16.630 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:16.633 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:16.629+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:16.639 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:28:16.734 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:16.730+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:16.819 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:16.819 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:16.830 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:28:16.874 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:16.870+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:16.935 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:16.931+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:16.943 INFO:tasks.ceph.osd.7:Stopped 2023-12-06T18:28:16.943 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.7 2023-12-06T18:28:16.943 INFO:tasks.thrashosds.thrasher:remote for osd 7 is ubuntu@smithi152.front.sepia.ceph.com 2023-12-06T18:28:16.944 DEBUG:teuthology.orchestra.run.smithi152:> type ceph-objectstore-tool 2023-12-06T18:28:16.958 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2023-12-06T18:28:16.958 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --op list-pgs 2023-12-06T18:28:16.993 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:16.993 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:17.002 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:28:17.044 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:17.040+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:17.137 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:17.133+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:17.198 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:17.198 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:17.208 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:28:17.238 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:17.234+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:17.268 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-12-06T18:28:17.339 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:17.335+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:17.358 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:17.358 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:17.368 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:28:17.453 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:17.449+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:17.546 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:17.542+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:17.557 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:17.557 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:17.567 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:28:17.652 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:17.648+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:17.743 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:17.739+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:17.761 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:17.761 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:17.770 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:28:17.790 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:28:17.857 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:17.853+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:17.945 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:17.940+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:17.962 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:17.963 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:17.973 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:28:18.058 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:18.054+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:18.092 INFO:teuthology.orchestra.run.smithi152.stdout:2.1 2023-12-06T18:28:18.146 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:18.142+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:18.151 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:18.151 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:18.160 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:28:18.247 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:18.244+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:18.344 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:18.344 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:18.348 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:18.344+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:18.353 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:28:18.437 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is unset 2023-12-06T18:28:18.450 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:18.446+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:18.531 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":18,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"b","quorum_age":4,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:28:18.542 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:18.542 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:18.551 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:18.547+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:18.552 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:28:18.617 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --op export --pgid 2.1 --file /var/log/ceph/exp.2.1.7 2023-12-06T18:28:18.682 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:18.679+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:18.697 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:18.697 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:18.707 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:28:18.725 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:28:18.729 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:28:18.730 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:28:18.752 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:18.748+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:18.853 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:18.849+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:18.876 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:18.876 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:18.886 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:28:18.954 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:18.951+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:19.031 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:19.031 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:19.041 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:28:19.087 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:19.082+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:19.156 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:19.152+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:19.174 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:19.174 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:19.183 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:28:19.257 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:19.253+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:19.358 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:19.354+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:19.368 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:19.368 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:19.378 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:28:19.459 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:19.455+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:19.508 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:19.509 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:19.518 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:28:19.560 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:19.556+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:19.660 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:19.657+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:19.687 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:19.687 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:19.696 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:28:19.735 INFO:teuthology.orchestra.run.smithi152.stderr:Exporting 2.1 info 2.1( empty local-lis/les=25/26 n=0 ec=21/21 lis/c=25/25 les/c/f=26/26/0 sis=25) 2023-12-06T18:28:19.735 INFO:teuthology.orchestra.run.smithi152.stderr:Export successful 2023-12-06T18:28:19.780 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:19.775+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:19.861 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:19.861 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:19.862 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:19.858+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:19.871 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:28:19.963 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:19.959+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:20.039 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:20.039 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:20.048 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:28:20.089 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:20.086+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:20.172 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:20.169+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:20.187 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:20.187 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:20.196 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:28:20.217 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-12-06T18:28:20.262 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --force --op remove --pgid 2.1 2023-12-06T18:28:20.281 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:20.277+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:20.366 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:20.363+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:20.375 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:20.375 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:20.384 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:28:20.454 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-12-06T18:28:20.470 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:20.465+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:20.565 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:20.565 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:20.568 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:20.564+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:20.575 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:28:20.669 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:20.666+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:20.731 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:28:20.756 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:20.756 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:20.766 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:28:20.808 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:20.804+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:20.871 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:20.867+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:20.955 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:20.955 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:20.964 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:28:20.972 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:20.969+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:21.019 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:28:21.020 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:28:21.155 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:21.091+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:21.155 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:28:21.155 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:28:21.200 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:21.196+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:21.224 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:21.224 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:21.224 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:21.224 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:21.224 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.224 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:28:21.225 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:21.225 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 85.804411579999993, 2023-12-06T18:28:21.225 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:28:21.225 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.225 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.225 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.225 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.225 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.225 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:28:21.225 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.225 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.225 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.225 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.225 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:21.225 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.225 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.226 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.226 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.226 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:21.226 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.226 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.226 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.226 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.226 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:28:21.226 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:28:21.226 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.226 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.226 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.226 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:28:21.226 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:28:21.227 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.227 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.227 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.227 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:28:21.227 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:28:21.227 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.227 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.227 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.227 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:28:21.227 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:28:21.227 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.227 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.227 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.228 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:28:21.228 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:28:21.228 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.228 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.228 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:28:21.228 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:28:21.228 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:28:21.228 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.228 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.228 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.228 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:28:21.228 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:28:21.228 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.228 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.228 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.228 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:28:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:28:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:28:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:28:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:28:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:28:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.229 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:28:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:28:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:28:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:28:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:28:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:28:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.230 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.231 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.231 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:21.231 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:21.231 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 83.695838340999998, 2023-12-06T18:28:21.231 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:28:21.231 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.231 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.231 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.231 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.231 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.231 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:28:21.231 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.231 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.231 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.231 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.232 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:21.232 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.232 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.232 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.232 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.232 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:21.232 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.232 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.232 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.232 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.232 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:28:21.232 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:28:21.232 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.232 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.232 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.232 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:28:21.233 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:28:21.233 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.233 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.233 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.233 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:28:21.233 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:28:21.233 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.233 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.233 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.233 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:28:21.233 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:28:21.233 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.233 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.233 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.233 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:28:21.234 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:28:21.234 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.234 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.234 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.234 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:28:21.234 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:28:21.234 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.234 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.234 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.234 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:28:21.234 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:28:21.234 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.234 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.234 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.234 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:28:21.234 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:28:21.235 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.235 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.235 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.235 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:28:21.235 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:28:21.235 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.235 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.235 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.235 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:28:21.235 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:28:21.235 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.235 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.235 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.235 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:28:21.235 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:28:21.236 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.236 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.236 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.236 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.236 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.236 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:21.236 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:21.236 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 83.695093108999998, 2023-12-06T18:28:21.236 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:28:21.236 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.236 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.236 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.236 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.236 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.236 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:28:21.236 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.237 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.237 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.237 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.237 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:21.237 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.237 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.237 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.237 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.237 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:21.237 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.237 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.237 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.237 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.237 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:28:21.238 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:28:21.238 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.238 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.238 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.238 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:28:21.238 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:28:21.238 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.238 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.238 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.238 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:28:21.238 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:28:21.238 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.238 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.238 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.238 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:28:21.239 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:28:21.239 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.239 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.239 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.239 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:28:21.239 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:28:21.239 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.239 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.239 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.239 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:28:21.239 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:28:21.239 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.239 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.239 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.240 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:28:21.240 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:28:21.240 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.240 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.240 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.240 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:28:21.240 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:28:21.240 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.240 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.240 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.240 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:28:21.240 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:28:21.240 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.240 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.240 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.241 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:28:21.241 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:28:21.241 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.241 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.241 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.241 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:28:21.241 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:28:21.241 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.241 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.241 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.241 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.241 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.241 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:21.241 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:21.241 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 83.694409354000001, 2023-12-06T18:28:21.241 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:28:21.241 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.242 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.242 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.242 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.242 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.242 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:28:21.242 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.242 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.242 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.242 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.242 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:21.242 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.242 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.242 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.242 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.242 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:21.243 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.243 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.243 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.243 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.243 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:28:21.243 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:28:21.243 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.243 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.243 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.243 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:28:21.243 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:28:21.243 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.243 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.243 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.243 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:28:21.244 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:28:21.244 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.244 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.244 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.244 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:28:21.244 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:28:21.244 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.244 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.244 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.244 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:28:21.244 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:28:21.244 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.244 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.244 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.244 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:28:21.244 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:28:21.244 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.245 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.245 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.245 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:28:21.245 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:28:21.245 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.245 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.245 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.245 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:28:21.245 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:28:21.245 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.245 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.245 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.245 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:28:21.245 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:28:21.245 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.245 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.245 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.246 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:28:21.246 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:28:21.246 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.246 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.246 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.246 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:28:21.246 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:28:21.246 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.246 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.246 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.246 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.246 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.246 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:21.246 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:21.246 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 83.688836099, 2023-12-06T18:28:21.246 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:28:21.247 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.247 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.247 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.247 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.247 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.247 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:28:21.247 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.247 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.247 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.247 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.247 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:21.247 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.247 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.247 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.247 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.247 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:21.247 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.248 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.248 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.248 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.248 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:28:21.248 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:28:21.248 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.248 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.248 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.248 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:28:21.248 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:28:21.248 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.248 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.248 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.248 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:28:21.248 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:28:21.248 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.248 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.249 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.249 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:28:21.249 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:28:21.249 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.249 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.249 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.249 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:28:21.249 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:28:21.249 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.249 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.249 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.249 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:28:21.249 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:28:21.249 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.249 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.249 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.249 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:28:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:28:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:28:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:28:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:28:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:28:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.250 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:28:21.251 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:28:21.251 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.251 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.251 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.251 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:28:21.251 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:28:21.251 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.251 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.251 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.251 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.251 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.251 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:21.251 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:21.251 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 83.688162458999997, 2023-12-06T18:28:21.251 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:28:21.251 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.252 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.252 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.252 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.252 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.252 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:28:21.252 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.252 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.252 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.252 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.252 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:21.252 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.252 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.252 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.252 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.252 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:21.252 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.252 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.253 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.253 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.253 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:28:21.253 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:28:21.253 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.253 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.253 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.253 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:28:21.253 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:28:21.253 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.253 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.253 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.253 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:28:21.253 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:28:21.253 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.253 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.253 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.254 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:28:21.254 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:28:21.254 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.254 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.254 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.254 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:28:21.254 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:28:21.254 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.254 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.254 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.254 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:28:21.254 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:28:21.254 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.254 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.254 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.254 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:28:21.255 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:28:21.255 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.255 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.255 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.255 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:28:21.255 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:28:21.255 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.255 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.255 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.255 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:28:21.255 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:28:21.255 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.255 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.255 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.255 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:28:21.255 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:28:21.255 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.256 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.256 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.256 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:28:21.256 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:28:21.256 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.256 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.256 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.256 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.256 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:21.256 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:21.256 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 83.687500259999993, 2023-12-06T18:28:21.256 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:28:21.256 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.256 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.256 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.257 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.257 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.257 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:28:21.257 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.257 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.257 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.257 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.257 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:21.257 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.257 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.257 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.257 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.257 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:21.257 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.257 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.258 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.258 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.258 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:28:21.258 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:28:21.258 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.258 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.258 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.258 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:28:21.258 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:28:21.258 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.258 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.258 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.258 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:28:21.258 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:28:21.258 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.259 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.259 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.259 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:28:21.259 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:28:21.259 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.259 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.259 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.259 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:28:21.259 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:28:21.259 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.259 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.259 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.259 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:28:21.259 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:28:21.259 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.260 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.260 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.260 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:28:21.260 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:28:21.260 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.260 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.260 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.260 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:28:21.260 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:28:21.260 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.260 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.260 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.260 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:28:21.260 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:28:21.260 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.260 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.261 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.261 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:28:21.261 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:28:21.261 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.261 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.261 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.261 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:28:21.261 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:28:21.261 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.261 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.261 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.261 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.261 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.261 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:21.261 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:21.262 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 83.614790825, 2023-12-06T18:28:21.262 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:28:21.262 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.262 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.262 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.262 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.262 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.262 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:28:21.262 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.262 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.262 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.262 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.262 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:21.262 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.262 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.263 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.263 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.263 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:21.263 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.263 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.263 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.263 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.263 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:28:21.263 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:28:21.263 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.263 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.263 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.263 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:28:21.263 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:28:21.263 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.263 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.263 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.264 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:28:21.264 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:28:21.264 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.264 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.264 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.264 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:28:21.264 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:28:21.264 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.264 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.264 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.264 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:28:21.264 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:28:21.264 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.264 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.265 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.265 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:28:21.265 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:28:21.265 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.265 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.265 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.265 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:28:21.265 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:28:21.265 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.265 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.265 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.265 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:28:21.265 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:28:21.265 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.265 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.265 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.266 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:28:21.266 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:28:21.266 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.266 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.266 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.266 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:28:21.266 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:28:21.266 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.266 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.266 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.266 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:28:21.266 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:28:21.266 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.267 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.267 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.267 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.267 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.267 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:21.267 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:21.267 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 83.614109115999995, 2023-12-06T18:28:21.267 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:28:21.267 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.267 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.267 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.267 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.267 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.267 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:28:21.268 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.268 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.268 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.268 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.268 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:21.268 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.268 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.268 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.268 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.268 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:21.268 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.268 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.268 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.268 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.268 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:28:21.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:28:21.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.269 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.269 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:28:21.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:28:21.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.269 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.269 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:28:21.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:28:21.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.269 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.269 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:28:21.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:28:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:28:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:28:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:28:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:28:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.270 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.271 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:28:21.271 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:28:21.271 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.271 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.271 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.271 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:28:21.271 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:28:21.271 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.271 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.271 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.271 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:28:21.271 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:28:21.271 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.271 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.271 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.272 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:28:21.272 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:28:21.272 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.272 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.272 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.272 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:28:21.272 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:28:21.272 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.272 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.272 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.272 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.272 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.272 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:21.272 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:21.272 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 82.535305027999996, 2023-12-06T18:28:21.273 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:28:21.273 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.273 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.273 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.273 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.273 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.273 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:28:21.273 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.273 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.273 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.273 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.273 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:21.273 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.273 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.273 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.273 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.274 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:21.274 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.274 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.274 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.274 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.274 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:28:21.274 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:28:21.274 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.274 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.274 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.274 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:28:21.274 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:28:21.274 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.274 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.274 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.275 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:28:21.275 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:28:21.275 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.275 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.275 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:28:21.275 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:28:21.275 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.275 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.275 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:28:21.275 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:28:21.275 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.276 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:28:21.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:28:21.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.276 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:28:21.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:28:21.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.276 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:28:21.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:28:21.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.277 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.277 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:28:21.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:28:21.277 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.277 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.277 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.277 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:28:21.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:28:21.277 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.277 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.277 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.277 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:28:21.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:28:21.277 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.277 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.277 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.278 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.278 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.278 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:21.278 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:21.278 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 82.533888454000007, 2023-12-06T18:28:21.278 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:28:21.278 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.278 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.278 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.278 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.278 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.278 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:28:21.278 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.278 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.278 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:21.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.279 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:21.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.279 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:28:21.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:28:21.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:28:21.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:28:21.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:28:21.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:28:21.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:28:21.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:28:21.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.281 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:28:21.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:28:21.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.281 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:28:21.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:28:21.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.281 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:28:21.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:28:21.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:28:21.282 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:28:21.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:28:21.282 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:28:21.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:28:21.282 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:28:21.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.283 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:28:21.283 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:28:21.283 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.283 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.283 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.283 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.283 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.283 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:21.283 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:21.283 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 82.532803329000004, 2023-12-06T18:28:21.283 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:28:21.283 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.283 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.283 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.283 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.283 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.284 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:28:21.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.284 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.284 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.284 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:21.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.284 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.284 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:21.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.284 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.284 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:28:21.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:28:21.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.285 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.285 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:28:21.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:28:21.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.285 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.285 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:28:21.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:28:21.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.285 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.285 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:28:21.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:28:21.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.286 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:28:21.286 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:28:21.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.286 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:28:21.286 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:28:21.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.286 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:28:21.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:28:21.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:28:21.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:28:21.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:28:21.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:28:21.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.288 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:28:21.288 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:28:21.288 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.288 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.288 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.288 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:28:21.288 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:28:21.288 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.288 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.288 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.288 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.288 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.288 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:21.288 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:21.288 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 82.497654226999998, 2023-12-06T18:28:21.289 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:28:21.289 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.289 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.289 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.289 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.289 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.289 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:28:21.289 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.289 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.289 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.289 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.289 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:21.289 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.289 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.289 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.289 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.290 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:21.290 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.290 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.290 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.290 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:28:21.290 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:28:21.290 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.290 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.290 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:28:21.290 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:28:21.290 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.290 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:28:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:28:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:28:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:28:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:28:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:28:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.291 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.292 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:28:21.292 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:28:21.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.292 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.292 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:28:21.292 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:28:21.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.292 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.292 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:28:21.292 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:28:21.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.292 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.293 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:28:21.293 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:28:21.293 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.293 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.293 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.293 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:28:21.293 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:28:21.293 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.293 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.293 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.293 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:28:21.293 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:28:21.293 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.293 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.293 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.293 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.293 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.294 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:21.294 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:21.294 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 75.947923701999997, 2023-12-06T18:28:21.294 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:28:21.294 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.294 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.294 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.294 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.294 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.294 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:28:21.294 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.294 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.294 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.294 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.295 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:21.295 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.295 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.295 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.295 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:21.295 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.295 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.295 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.295 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:28:21.295 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:28:21.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.295 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.295 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.295 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:28:21.295 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:28:21.296 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.296 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.296 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.296 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:28:21.296 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:28:21.296 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.296 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.296 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.296 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:28:21.296 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:28:21.296 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.296 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.296 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.296 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:28:21.296 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:28:21.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.297 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.297 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.297 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:28:21.297 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:28:21.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.297 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.297 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.297 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:28:21.297 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:28:21.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.297 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.297 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.297 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:28:21.297 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:28:21.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.297 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.298 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.298 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:28:21.298 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:28:21.298 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.298 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.298 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.298 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:28:21.298 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:28:21.298 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.298 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.298 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.298 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:28:21.298 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:28:21.298 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.299 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.299 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.299 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.299 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.299 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:21.299 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:21.299 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 75.947118094000004, 2023-12-06T18:28:21.299 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:28:21.299 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.299 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.299 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.299 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.299 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.299 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:28:21.299 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.300 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.300 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.300 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.300 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:21.300 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.300 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.300 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.300 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.300 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:21.300 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.300 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.300 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.300 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.300 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:28:21.300 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:28:21.301 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.301 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.301 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.301 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:28:21.301 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:28:21.301 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.301 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.301 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.301 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:28:21.301 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:28:21.301 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.301 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.301 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.301 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:28:21.301 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:28:21.301 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.302 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.302 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.302 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:28:21.302 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:28:21.302 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.302 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.302 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.302 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:28:21.302 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:28:21.302 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.302 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.302 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.302 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:28:21.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:28:21.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.303 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:28:21.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:28:21.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.303 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:28:21.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:28:21.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.303 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:28:21.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:28:21.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:28:21.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:28:21.304 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.304 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.304 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.304 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:21.304 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:21.304 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 75.946427284999999, 2023-12-06T18:28:21.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:28:21.304 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.305 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.305 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.305 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.305 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.305 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:28:21.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.305 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:21.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:21.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.306 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.306 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:28:21.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:28:21.306 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.306 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:28:21.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:28:21.306 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.306 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:28:21.307 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:28:21.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.307 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.307 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.307 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:28:21.307 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:28:21.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.307 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.307 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.307 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:28:21.307 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:28:21.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.307 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.307 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.307 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:28:21.307 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:28:21.308 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.308 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.308 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.308 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:28:21.308 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:28:21.308 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.308 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.308 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.308 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:28:21.308 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:28:21.308 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.308 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.308 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.308 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:28:21.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:28:21.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.309 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.309 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:28:21.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:28:21.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.309 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.309 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:28:21.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:28:21.309 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.309 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.309 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.309 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:21.309 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:21.310 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 75.942129777000005, 2023-12-06T18:28:21.310 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:28:21.310 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.310 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.310 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.310 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.310 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.310 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:28:21.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.310 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.310 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.310 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.310 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:21.310 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:28:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:28:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:28:21.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:28:21.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:28:21.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:28:21.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:28:21.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:28:21.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:28:21.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:28:21.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.313 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:28:21.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:28:21.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.313 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:28:21.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:28:21.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.313 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:28:21.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:28:21.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:28:21.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:28:21.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:28:21.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:28:21.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:28:21.315 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:28:21.315 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.315 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.315 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.315 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.315 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.315 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:21.315 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:21.315 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 75.941462102000003, 2023-12-06T18:28:21.315 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:28:21.315 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.315 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.315 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.315 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.315 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.315 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:28:21.316 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.316 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.316 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.316 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.316 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:21.316 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.316 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.316 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.316 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.316 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:21.316 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.316 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.316 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.316 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.316 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:28:21.316 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:28:21.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.317 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.317 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:28:21.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:28:21.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.317 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.317 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:28:21.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:28:21.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.317 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.317 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:28:21.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:28:21.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:28:21.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:28:21.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:28:21.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:28:21.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:28:21.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:28:21.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:28:21.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:28:21.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:28:21.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:28:21.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:28:21.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:28:21.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:28:21.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:28:21.320 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.320 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.320 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.320 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:21.320 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:21.320 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 75.940806674000001, 2023-12-06T18:28:21.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:28:21.320 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.320 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.320 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.320 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.320 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.321 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:28:21.321 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.321 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.321 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.321 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.321 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:21.321 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.321 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.321 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.321 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.321 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:21.321 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.321 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.321 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.321 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.321 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:28:21.321 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:28:21.322 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.322 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.322 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.322 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:28:21.322 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:28:21.322 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.322 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.322 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.322 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:28:21.322 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:28:21.322 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.322 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.322 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.322 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:28:21.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:28:21.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.323 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.323 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.323 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:28:21.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:28:21.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.323 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.323 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.323 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:28:21.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:28:21.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.323 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.323 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.324 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:28:21.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:28:21.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.324 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:28:21.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:28:21.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.324 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:28:21.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:28:21.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.324 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:28:21.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:28:21.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:28:21.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:28:21.325 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.325 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.325 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.325 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:21.325 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:21.325 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 75.900152126999998, 2023-12-06T18:28:21.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:28:21.325 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.325 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:21.326 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:21.326 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:21.326 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:21.326 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:28:21.326 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.326 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.326 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.326 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.326 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:21.326 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.326 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.326 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.326 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.326 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:21.327 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.327 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.327 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.327 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.327 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:28:21.327 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:28:21.327 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.327 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.327 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.327 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:28:21.327 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:28:21.327 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.327 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.327 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.327 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:28:21.327 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:28:21.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.328 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.328 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.328 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:28:21.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:28:21.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.328 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.328 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.328 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:28:21.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:28:21.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.328 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.328 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.328 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:28:21.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:28:21.329 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:21.329 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:28:21.329 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:28:21.329 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:21.329 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:28:21.329 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:28:21.329 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:21.329 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:28:21.329 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:28:21.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:21.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:28:21.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:28:21.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:28:21.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:28:21.330 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.330 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.330 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.330 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.330 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.330 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:21.332 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:28:21.332 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:28:21.333 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:28:21.333 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:21.328+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:21.347 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is set 2023-12-06T18:28:21.381 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:28:21.381 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:28:21.390 INFO:teuthology.orchestra.run.smithi152.stdout: marking collection for removal 2023-12-06T18:28:21.390 INFO:teuthology.orchestra.run.smithi152.stdout:setting '_remove' omap key 2023-12-06T18:28:21.391 INFO:teuthology.orchestra.run.smithi152.stdout:finish_remove_pgs 2.1_head removing 2.1 2023-12-06T18:28:21.391 INFO:teuthology.orchestra.run.smithi152.stdout:Remove successful 2023-12-06T18:28:21.432 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:21.429+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:21.484 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:28:21.484 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:28:21.489 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:21.489 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:21.489 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:21.489 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:21.499 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:28:21.539 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:21.536+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:21.545 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:28:21.578 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:28:21.579 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:28:21.635 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:21.631+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:21.683 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:28:21.683 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:28:21.687 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:28:21.692 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:28:21.692 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:28:21.692 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:21.693 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:21.693 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:21.693 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:21.693 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:21.702 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:28:21.745 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:21.740+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:21.836 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:21.832+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:21.882 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:28:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 60.066222025999998, 2023-12-06T18:28:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:28:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:28:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:28:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:28:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:28:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:28:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:28:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:28:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:28:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:28:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:28:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:28:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:28:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:28:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:28:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:28:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:21.886 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:21.891 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:28:21.904 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.2.1.7 2023-12-06T18:28:21.945 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:21.942+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:22.037 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:22.035+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:22.046 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:22.046 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:22.046 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:22.046 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:22.056 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:28:22.138 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:22.134+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:22.236 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:22.236 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:22.236 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:22.236 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:22.236 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:22.239 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:22.236+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:22.245 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:28:22.252 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":18,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"b","quorum_age":8,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:28:22.340 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:22.337+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:22.431 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:22.432 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:22.432 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:22.432 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:22.432 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:22.440 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:22.437+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:22.441 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:28:22.541 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:22.539+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:22.572 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:22.572 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:22.572 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:22.572 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:22.582 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:28:22.642 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:22.639+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:22.710 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:22.711 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:22.711 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:22.711 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:22.711 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:22.719 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:28:22.743 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:22.741+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:22.804 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:08.737 98294== Warning: unimplemented fcntl command: 1036 2023-12-06T18:28:22.845 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:22.841+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:22.866 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:22.866 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:22.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:22.866 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:28:22.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:22.867 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:28:22.867 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:22.867 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 1.363220264, 2023-12-06T18:28:22.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:28:22.867 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:22.867 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:28:22.867 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:28:22.867 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:28:22.867 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:28:22.867 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:28:22.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:22.868 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:22.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:22.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:22.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:22.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:22.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:22.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:22.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:22.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:22.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:22.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:22.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:22.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:22.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:28:22.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:28:22.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:22.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:22.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:22.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:28:22.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:28:22.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:22.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:22.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:22.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:28:22.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:28:22.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:22.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:22.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:22.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:28:22.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:28:22.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:22.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:22.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:22.871 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:28:22.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:28:22.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:22.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:22.871 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:22.871 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:28:22.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:28:22.871 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:22.871 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:22.871 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:22.871 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:22.871 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:22.871 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:22.876 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:28:22.945 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:22.943+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:23.046 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:23.044+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:23.048 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:23.048 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:23.048 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:23.048 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:23.048 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:08.980 98294== Warning: unimplemented fcntl command: 1036 2023-12-06T18:28:23.053 INFO:teuthology.orchestra.run.smithi152.stdout:get_pg_num_history pg_num_history pg_num_history(e37 pg_nums {1={15=1},2={21=8}} deleted_pools ) 2023-12-06T18:28:23.053 INFO:teuthology.orchestra.run.smithi152.stdout:Importing pgid 2.1 2023-12-06T18:28:23.055 INFO:teuthology.orchestra.run.smithi152.stdout:write_pg epoch 26 info 2.1( empty local-lis/les=25/26 n=0 ec=21/21 lis/c=25/25 les/c/f=26/26/0 sis=25) 2023-12-06T18:28:23.056 INFO:teuthology.orchestra.run.smithi152.stdout:Import successful 2023-12-06T18:28:23.057 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:28:23.147 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:23.144+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:23.185 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:23.186 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:23.186 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:23.186 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:23.186 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:23.195 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:28:23.248 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:23.245+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:23.349 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:23.346+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:23.364 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-12-06T18:28:23.368 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:23.368 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:23.368 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:23.368 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:28:23.368 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.368 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:28:23.368 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:23.368 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 86.748990925000001, 2023-12-06T18:28:23.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:28:23.369 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.369 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.369 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:23.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:23.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.370 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.370 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.370 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:28:23.370 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:28:23.370 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.370 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.370 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.370 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:28:23.370 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:28:23.370 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.370 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.370 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.370 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:28:23.371 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:28:23.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.371 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.371 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.371 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:28:23.371 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:28:23.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.371 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.371 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.371 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:28:23.371 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:28:23.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.371 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.371 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.372 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:28:23.372 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:28:23.372 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.372 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.372 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.372 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:28:23.372 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:28:23.372 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.372 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.372 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.372 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:28:23.372 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:28:23.372 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.372 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.372 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.373 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.373 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.373 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:28:23.373 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:23.373 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 85.726163916000004, 2023-12-06T18:28:23.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:28:23.373 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.373 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.373 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.373 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.373 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.373 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:23.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.373 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.373 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.374 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.374 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:23.374 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.374 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.374 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:28:23.374 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:28:23.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.374 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.374 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:28:23.374 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:28:23.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:28:23.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:28:23.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:28:23.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:28:23.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:28:23.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:28:23.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.376 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:28:23.376 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:28:23.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.376 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.376 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:28:23.376 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:28:23.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.376 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.376 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:28:23.376 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:28:23.376 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.376 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:28:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 85.714117278000003, 2023-12-06T18:28:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:28:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:23.377 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.378 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.378 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:23.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.378 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.378 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:28:23.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:28:23.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.379 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.379 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:28:23.379 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:28:23.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.379 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.379 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.379 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:28:23.379 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:28:23.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.380 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:28:23.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:28:23.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.380 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:28:23.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:28:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:28:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:28:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:28:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:28:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:28:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:28:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:28:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 85.692716670999999, 2023-12-06T18:28:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:28:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:23.385 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.385 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.385 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.385 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.385 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:23.385 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.385 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.385 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.386 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.386 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:28:23.386 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:28:23.386 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.386 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.386 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.386 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:28:23.386 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:28:23.386 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.386 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.386 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.386 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:28:23.387 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:28:23.387 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.387 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.387 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.387 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:28:23.387 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:28:23.387 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.387 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.387 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.387 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:28:23.388 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:28:23.388 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.388 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.388 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.388 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:28:23.388 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:28:23.388 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.388 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.388 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.388 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:28:23.388 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:28:23.388 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:28:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:28:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:28:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 85.691537347999997, 2023-12-06T18:28:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:28:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.389 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.390 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.390 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.390 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:23.390 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.390 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.390 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.390 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.390 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:23.390 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.390 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.390 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.390 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.390 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:28:23.390 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:28:23.391 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.391 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.391 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.391 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:28:23.391 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:28:23.391 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.391 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.391 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.391 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:28:23.391 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:28:23.391 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.392 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.392 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.392 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:28:23.392 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:28:23.392 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.392 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.392 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.392 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:28:23.392 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:28:23.392 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.392 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.392 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.392 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:28:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:28:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:28:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:28:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:28:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:28:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.394 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:28:23.394 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:23.394 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 85.690965367000004, 2023-12-06T18:28:23.394 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:28:23.394 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.394 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.394 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.394 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.394 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.394 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:23.394 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.394 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.395 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.395 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:23.395 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.395 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.395 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.395 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.395 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:28:23.395 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:28:23.395 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.395 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.395 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.395 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:28:23.395 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:28:23.395 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:28:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:28:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:28:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:28:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:28:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:28:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.396 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.397 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.397 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:28:23.397 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:28:23.397 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.397 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.397 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:28:23.397 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:28:23.397 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.397 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.398 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:28:23.398 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:28:23.398 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.398 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.398 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.398 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.398 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.398 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:28:23.398 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:23.398 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 85.245129896999998, 2023-12-06T18:28:23.398 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:28:23.398 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.399 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.399 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.399 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.399 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:23.399 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.399 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.399 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:23.399 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.399 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:28:23.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:28:23.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:28:23.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:28:23.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:28:23.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:28:23.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:28:23.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:28:23.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:28:23.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:28:23.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.402 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:28:23.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:28:23.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.402 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.402 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:28:23.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:28:23.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.402 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.402 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:28:23.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:28:23.402 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.403 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.403 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.403 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:28:23.403 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:23.403 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 85.243530041, 2023-12-06T18:28:23.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:28:23.403 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.403 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.403 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.403 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:28:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:28:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.404 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:28:23.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:28:23.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:28:23.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:28:23.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:28:23.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:28:23.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:28:23.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:28:23.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:28:23.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:28:23.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:28:23.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:28:23.407 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.407 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.407 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.407 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:28:23.407 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:28:23.407 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.407 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.407 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.407 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.407 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.407 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:28:23.408 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:23.408 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 84.864324425000007, 2023-12-06T18:28:23.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:28:23.408 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.408 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.408 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.408 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.408 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:23.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:23.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:28:23.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:28:23.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:28:23.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:28:23.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:28:23.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:28:23.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:28:23.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:28:23.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:28:23.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:28:23.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:28:23.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:28:23.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.412 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.412 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:28:23.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:28:23.412 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.412 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.412 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:28:23.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:28:23.412 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.412 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.412 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.412 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.412 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:28:23.413 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:23.413 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 84.626822243000007, 2023-12-06T18:28:23.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:28:23.413 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.413 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.413 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.413 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:23.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.413 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.413 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:23.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:28:23.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:28:23.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:28:23.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:28:23.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:28:23.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:28:23.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:28:23.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:28:23.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:28:23.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:28:23.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:28:23.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:28:23.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:28:23.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:28:23.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:28:23.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:28:23.416 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.417 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.417 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.417 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:28:23.417 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:23.417 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 84.626094244000001, 2023-12-06T18:28:23.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:28:23.417 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.417 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.418 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:23.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:23.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:28:23.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:28:23.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:28:23.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:28:23.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:28:23.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:28:23.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:28:23.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:28:23.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:28:23.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:28:23.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:28:23.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:28:23.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:28:23.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:28:23.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:28:23.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:28:23.422 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.422 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.422 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.422 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:28:23.422 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:23.422 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 84.621081833999995, 2023-12-06T18:28:23.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:28:23.422 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.423 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.423 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:23.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.424 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.424 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.424 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:23.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:28:23.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:28:23.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.426 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.426 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:28:23.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:28:23.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.427 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.427 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:28:23.427 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:28:23.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.428 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.428 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:28:23.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:28:23.428 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.428 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:28:23.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:28:23.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:28:23.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:28:23.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:28:23.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:28:23.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:28:23.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:28:23.432 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.432 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.432 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.432 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:28:23.433 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:23.433 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 84.615554755000005, 2023-12-06T18:28:23.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:28:23.433 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.433 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.433 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.434 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.434 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:23.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:23.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.436 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:28:23.436 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:28:23.436 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.436 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:28:23.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:28:23.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:28:23.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:28:23.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:28:23.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:28:23.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:28:23.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:28:23.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:28:23.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:28:23.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:28:23.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:28:23.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:28:23.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:28:23.442 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.442 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.442 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.443 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:28:23.443 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:23.444 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 84.615043917999998, 2023-12-06T18:28:23.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:28:23.444 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.444 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.444 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:23.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:23.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:28:23.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:28:23.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:28:23.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:28:23.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.448 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:28:23.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:28:23.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:28:23.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:28:23.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:28:23.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:28:23.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:28:23.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:28:23.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:28:23.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:28:23.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:28:23.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:28:23.453 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.454 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.454 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.454 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:28:23.454 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:23.455 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 84.614584656999995, 2023-12-06T18:28:23.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:28:23.455 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.455 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.455 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:23.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:23.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:28:23.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:28:23.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:28:23.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:28:23.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:28:23.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:28:23.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:28:23.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:28:23.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:28:23.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:28:23.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:28:23.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:28:23.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:28:23.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:28:23.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:28:23.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:28:23.461 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.461 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.461 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.461 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:28:23.461 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:23.461 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 78.034668850000003, 2023-12-06T18:28:23.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:28:23.461 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.461 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.462 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:23.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:23.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:28:23.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:28:23.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:28:23.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:28:23.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:28:23.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:28:23.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:28:23.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:28:23.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:28:23.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:28:23.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:28:23.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:28:23.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:28:23.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:28:23.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:28:23.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:28:23.465 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.465 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.465 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.465 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:28:23.465 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:23.465 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 78.028431045000005, 2023-12-06T18:28:23.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:28:23.465 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.466 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.466 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:23.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:23.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:28:23.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:28:23.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:28:23.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:28:23.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:28:23.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:28:23.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:28:23.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:28:23.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:28:23.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:28:23.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:28:23.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:28:23.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:28:23.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:28:23.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:28:23.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:28:23.469 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.469 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.469 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.469 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:28:23.469 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:23.469 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 78.022367966000004, 2023-12-06T18:28:23.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:28:23.469 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.469 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.469 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.470 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.470 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:23.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.470 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.470 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:23.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.470 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.470 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:28:23.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:28:23.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:28:23.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:28:23.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:28:23.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:28:23.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:28:23.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:28:23.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:28:23.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:28:23.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:28:23.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:28:23.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:28:23.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:28:23.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:28:23.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:28:23.473 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.473 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.473 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.473 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:28:23.473 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:23.473 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 77.884980881000004, 2023-12-06T18:28:23.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:28:23.473 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.473 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.473 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:23.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:23.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:28:23.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:28:23.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:28:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:28:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:28:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:28:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:28:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:28:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:28:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:28:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:28:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:28:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:28:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:28:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:28:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:28:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:28:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 77.878310154000005, 2023-12-06T18:28:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:28:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:23.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:28:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:28:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:28:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:28:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:28:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:28:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:28:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:28:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:23.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:28:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:28:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:28:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:28:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:28:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:28:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:28:23.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:28:23.481 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.481 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.481 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.481 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:23.481 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:23.481 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:23.482 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:28:23.483 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:23.480+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:23.577 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f /var/log/ceph/exp.2.1.7 2023-12-06T18:28:23.603 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:23.599+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:23.614 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:23.614 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:23.614 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:23.614 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:23.624 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:28:23.684 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:23.681+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:23.693 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:28:23.785 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:23.782+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:23.793 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:23.793 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:23.793 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:23.793 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:23.793 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:23.803 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:28:23.886 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:23.882+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:23.934 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:23.934 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:23.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:23.934 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:28:23.934 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:23.945 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:28:23.950 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:28:23.950 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:28:23.987 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:23.984+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:24.048 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:28:24.048 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:28:24.086 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:24.086 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:24.088 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:24.084+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:24.095 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:28:24.177 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:28:24.177 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:28:24.189 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:24.185+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:24.272 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:28:24.272 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:28:24.277 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:24.277 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:24.286 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:28:24.290 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:24.287+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:24.359 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:28:24.359 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:28:24.391 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:24.388+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:24.446 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:28:24.447 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:28:24.483 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:24.483 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:24.492 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:28:24.492 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:24.489+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:24.593 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:24.590+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:24.672 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:24.672 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:24.683 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:28:24.694 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:24.691+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:24.741 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:28:24.741 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:28:24.744 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:28:24.747 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:28:24.748 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:28:24.796 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:24.792+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:24.870 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:24.870 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:24.880 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:28:24.896 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:24.893+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:25.006 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:25.003+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:25.022 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:25.022 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:25.032 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:28:25.087 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is set 2023-12-06T18:28:25.098 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:25.095+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:25.200 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:25.197+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:25.216 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:25.216 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:25.225 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:28:25.266 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:28:25.311 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:25.308+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:25.402 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:25.399+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:25.412 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:25.412 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:25.422 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:28:25.506 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:25.503+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:25.598 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:25.599 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:25.604 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:25.602+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:25.608 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:28:25.705 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:25.703+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:25.790 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:25.790 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:25.799 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:28:25.806 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:25.802+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:25.907 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:25.904+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:25.931 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":18,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"b","quorum_age":11,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:28:25.983 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:25.983 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:25.992 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:28:26.034 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:26.031+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:26.109 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:26.106+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:26.170 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:26.170 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:26.179 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:28:26.224 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:26.220+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:26.310 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:26.307+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:26.311 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:26.312 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:26.321 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:28:26.410 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:26.407+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:26.488 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:26.488 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:26.497 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:28:26.538 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:26.535+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:26.612 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:26.609+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:26.667 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:26.667 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:26.677 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:28:26.714 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:26.711+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:26.748 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:28:26.807 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:26.807 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:26.814 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:26.811+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:26.816 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:28:26.915 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:26.913+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:26.990 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:26.990 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:26.999 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:28:27.045 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:27.041+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:27.104 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-12-06T18:28:27.117 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:27.113+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:27.146 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:27.146 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:27.157 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:28:27.242 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:27.238+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:27.340 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:27.336+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:27.349 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:27.350 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:27.359 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:28:27.385 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:28:27.385 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:28:27.444 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:27.440+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:27.495 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:28:27.495 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:28:27.520 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:27.517+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:27.530 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:27.530 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:27.540 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:28:27.590 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:28:27.591 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:28:27.624 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:27.621+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:27.682 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:28:27.682 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:28:27.711 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:27.711 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:27.721 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:28:27.722 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:27.718+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:27.772 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:28:27.773 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:28:27.822 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:27.819+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:27.858 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:28:27.858 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:28:27.859 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:27.859 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:27.859 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:27.859 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:27.868 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:28:27.944 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:28:27.944 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:28:27.944 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:27.940+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:27.947 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:28:27.951 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:28:27.951 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:28:28.005 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:28.005 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:28.005 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:28.005 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:28.005 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:28.014 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:28:28.059 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:28.056+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:28.125 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:28.122+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:28.153 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:28.154 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:28.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:28.154 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:28.154 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:28.162 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:28:28.247 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:28.244+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:28.259 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is unset 2023-12-06T18:28:28.326 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:28.323+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:28.350 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:28.350 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:28.350 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:28.350 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:28.360 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:28:28.428 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:28.424+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:28.497 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:28.497 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:28.497 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:28.497 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:28.497 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:28.507 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:28:28.552 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:28.549+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:28.603 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7] out_osds: [5] dead_osds: [7] live_osds: [0, 1, 2, 3, 4, 5, 6] 2023-12-06T18:28:28.603 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:28:28.604 INFO:tasks.thrashosds.thrasher:Adding osd 5 2023-12-06T18:28:28.604 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2023-12-06T18:28:28.648 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:28.645+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:28.705 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:28.705 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:28.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:28.705 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:28.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.705 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:28:28.705 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:28.706 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 93.291545080999995, 2023-12-06T18:28:28.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:28:28.706 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.706 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.706 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.706 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.706 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.706 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:28:28.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.706 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:28.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:28.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:28:28.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:28:28.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:28:28.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:28:28.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:28:28.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:28:28.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:28:28.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:28:28.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:28:28.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:28:28.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:28:28.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:28:28.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:28:28.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:28:28.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:28:28.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:28:28.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:28:28.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.710 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:28:28.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:28:28.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.710 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:28:28.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:28:28.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.710 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:28:28.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:28:28.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:28:28.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:28:28.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:28:28.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:28:28.711 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.711 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.712 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.712 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:28.712 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:28.712 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 91.182971842000001, 2023-12-06T18:28:28.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:28:28.712 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.712 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.712 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.712 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.712 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.712 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:28:28.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.712 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:28.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:28.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:28:28.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:28:28.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:28:28.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:28:28.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:28:28.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:28:28.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:28:28.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:28:28.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:28:28.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:28:28.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:28:28.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:28:28.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:28:28.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:28:28.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:28:28.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:28:28.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:28:28.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:28:28.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:28:28.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:28:28.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:28:28.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:28:28.717 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.717 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.717 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.717 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:28.717 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:28.717 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 91.182226610000001, 2023-12-06T18:28:28.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:28:28.717 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.717 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.717 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.718 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.718 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.718 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:28:28.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.718 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:28.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:28.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:28:28.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:28:28.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:28:28.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:28:28.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:28:28.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:28:28.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:28:28.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:28:28.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:28:28.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:28:28.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:28:28.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:28:28.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:28:28.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:28:28.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:28:28.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:28:28.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:28:28.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:28:28.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:28:28.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:28:28.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:28:28.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:28:28.722 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.722 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.722 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.722 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:28.723 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:28.723 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 91.181542855000004, 2023-12-06T18:28:28.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:28:28.723 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.723 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.723 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.723 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.723 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.723 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:28:28.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.723 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:28.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:28.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:28:28.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:28:28.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:28:28.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:28:28.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:28:28.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:28:28.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:28:28.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:28:28.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:28:28.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:28:28.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.726 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:28:28.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:28:28.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.726 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:28:28.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:28:28.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.726 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:28:28.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:28:28.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:28:28.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:28:28.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:28:28.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:28:28.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:28:28.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:28:28.728 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.728 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.728 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.728 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:28.728 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:28.728 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 91.175969600000002, 2023-12-06T18:28:28.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:28:28.728 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.728 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.728 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.728 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.728 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.728 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:28:28.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.728 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:28.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:28.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:28:28.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:28:28.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:28:28.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:28:28.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:28:28.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:28:28.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:28:28.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:28:28.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:28:28.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:28:28.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:28:28.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:28:28.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:28:28.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:28:28.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:28:28.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:28:28.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:28:28.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:28:28.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:28:28.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:28:28.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:28:28.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:28:28.733 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.733 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.733 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.733 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:28.733 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:28.733 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 91.17529596, 2023-12-06T18:28:28.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:28:28.733 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.733 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.733 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.733 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.733 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.734 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:28:28.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.734 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:28.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:28.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:28:28.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:28:28.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:28:28.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:28:28.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:28:28.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:28:28.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:28:28.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:28:28.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:28:28.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:28:28.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:28:28.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:28:28.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:28:28.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:28:28.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:28:28.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:28:28.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:28:28.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:28:28.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:28:28.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:28:28.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:28:28.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:28:28.738 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.738 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.738 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.738 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:28.738 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:28.738 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 91.174633760999996, 2023-12-06T18:28:28.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:28:28.739 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.739 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.739 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.739 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.739 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.739 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:28:28.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.739 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:28.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:28.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:28:28.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:28:28.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:28:28.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:28:28.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:28:28.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:28:28.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:28:28.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:28:28.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:28:28.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:28:28.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:28:28.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:28:28.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:28:28.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:28:28.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:28:28.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:28:28.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:28:28.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:28:28.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:28:28.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:28:28.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:28:28.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:28:28.743 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.743 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.743 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.744 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:28.744 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:28.744 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 91.101924326000002, 2023-12-06T18:28:28.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:28:28.744 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.744 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.744 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.744 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.744 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.744 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:28:28.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.744 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:28.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:28.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:28:28.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:28:28.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:28:28.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:28:28.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:28:28.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:28:28.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:28:28.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:28:28.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:28:28.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:28:28.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:28:28.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:28:28.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:28:28.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:28:28.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.748 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.748 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.748 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:28:28.748 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:28:28.748 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.748 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.748 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.748 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:28:28.748 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:28:28.748 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.748 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.748 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.748 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:28:28.748 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:28:28.748 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.749 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.749 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.749 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:28:28.749 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:28:28.749 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.749 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.749 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.749 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.749 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:28.749 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:28.749 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 91.101242616999997, 2023-12-06T18:28:28.749 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:28:28.749 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.749 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.749 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.750 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.750 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.750 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:28:28.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.750 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.750 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.750 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.750 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:28.750 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.750 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.750 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.750 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:28.750 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.751 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.751 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:28:28.751 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:28:28.751 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.751 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.751 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:28:28.751 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:28:28.751 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.751 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.751 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:28:28.751 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:28:28.751 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:28:28.752 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:28:28.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:28:28.752 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:28:28.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:28:28.752 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:28:28.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.753 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.753 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:28:28.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:28:28.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.753 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.753 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.753 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:28:28.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:28:28.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.753 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.753 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.753 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:28:28.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:28:28.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.753 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.753 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.754 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:28:28.754 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:28:28.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.754 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.754 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.754 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:28:28.754 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:28:28.754 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.754 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.754 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.754 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.754 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:28.754 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:28.755 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 90.022438528999999, 2023-12-06T18:28:28.755 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:28:28.755 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.755 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.755 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.755 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.755 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.755 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:28:28.755 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.755 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.755 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.755 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.755 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:28.755 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.755 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.756 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.756 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.756 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:28.756 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.756 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.756 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.756 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:28:28.756 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:28:28.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.756 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.756 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.756 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:28:28.756 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:28:28.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:28:28.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:28:28.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:28:28.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:28:28.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:28:28.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:28:28.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.758 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.758 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.758 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:28:28.758 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:28:28.758 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.758 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.758 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.758 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:28:28.758 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:28:28.758 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.758 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.758 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.758 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:28:28.758 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:28:28.758 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:28:28.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:28:28.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:28:28.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:28:28.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:28:28.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:28:28.759 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.760 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.760 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.760 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.760 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:28.760 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:28.760 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 90.021021954999995, 2023-12-06T18:28:28.760 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:28:28.760 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.760 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.760 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.760 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.760 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:28:28.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.761 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.761 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.761 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.761 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:28.761 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.761 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.761 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.761 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:28.761 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.761 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.761 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.761 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:28:28.761 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:28:28.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.762 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.762 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:28:28.762 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:28:28.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.762 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.762 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:28:28.762 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:28:28.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.762 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.762 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:28:28.762 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:28:28.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.763 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:28:28.763 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:28:28.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.763 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:28:28.763 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:28:28.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.763 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:28:28.763 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:28:28.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.764 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:28:28.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:28:28.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.764 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:28:28.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:28:28.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.764 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:28:28.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:28:28.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.765 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:28:28.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:28:28.765 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.765 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.765 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.765 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:28.765 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:28.765 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 90.019936830000006, 2023-12-06T18:28:28.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:28:28.765 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.766 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.766 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.766 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.766 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.766 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:28:28.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.766 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.766 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.766 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.766 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:28.766 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.766 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.766 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.766 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:28.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.767 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:28:28.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:28:28.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.767 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:28:28.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:28:28.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.767 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:28:28.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:28:28.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:28:28.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:28:28.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:28:28.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:28:28.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:28:28.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:28:28.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:28:28.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:28:28.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:28:28.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:28:28.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:28:28.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:28:28.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:28:28.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:28:28.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:28:28.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:28:28.770 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.770 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.770 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.771 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:28.771 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:28.771 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 89.984787728000001, 2023-12-06T18:28:28.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:28:28.771 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.771 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.771 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.771 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.771 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.771 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:28:28.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.771 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:28.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.772 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:28.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.772 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:28:28.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:28:28.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.772 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:28:28.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:28:28.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:28:28.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:28:28.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:28:28.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:28:28.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:28:28.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:28:28.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:28:28.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:28:28.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:28:28.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:28:28.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:28:28.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:28:28.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:28:28.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:28:28.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:28:28.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:28:28.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:28:28.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:28:28.776 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.776 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.776 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.776 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:28.776 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:28.776 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 83.435057203, 2023-12-06T18:28:28.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:28:28.776 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.776 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.776 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.776 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.776 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.776 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:28:28.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.777 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:28.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:28.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:28:28.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:28:28.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:28:28.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:28:28.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:28:28.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:28:28.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:28:28.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:28:28.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:28:28.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:28:28.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:28:28.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:28:28.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:28:28.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:28:28.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:28:28.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:28:28.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:28:28.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:28:28.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:28:28.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:28:28.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:28:28.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:28:28.781 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.781 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.781 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.781 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:28.781 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:28.781 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 83.434251595000006, 2023-12-06T18:28:28.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:28:28.782 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.782 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.782 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.782 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.782 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.782 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:28:28.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.782 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.782 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.782 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:28.782 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.782 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:28.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:28:28.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:28:28.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:28:28.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:28:28.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:28:28.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:28:28.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:28:28.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:28:28.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:28:28.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:28:28.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.785 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:28:28.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:28:28.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.785 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:28:28.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:28:28.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.785 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:28:28.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:28:28.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:28:28.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:28:28.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:28:28.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:28:28.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:28:28.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:28:28.786 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.786 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.786 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.787 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:28.787 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:28.787 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 83.433560786000001, 2023-12-06T18:28:28.787 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:28:28.787 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.787 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.787 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.787 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.787 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.787 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:28:28.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.787 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.788 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:28.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.788 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:28.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.788 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:28:28.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:28:28.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:28:28.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:28:28.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:28:28.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:28:28.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:28:28.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:28:28.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.790 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.790 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:28:28.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:28:28.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.790 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.790 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:28:28.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:28:28.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.790 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.790 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:28:28.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:28:28.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:28:28.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:28:28.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:28:28.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:28:28.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:28:28.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:28:28.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.792 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:28:28.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:28:28.792 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.792 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.792 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.792 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:28.792 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:28.792 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 83.429263277999993, 2023-12-06T18:28:28.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:28:28.792 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.792 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.792 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.792 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.792 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.792 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:28:28.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.793 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:28.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:28.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:28:28.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:28:28.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:28:28.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:28:28.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:28:28.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:28:28.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:28:28.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:28:28.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:28:28.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:28:28.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:28:28.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:28:28.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:28:28.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:28:28.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:28:28.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:28:28.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:28:28.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:28:28.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:28:28.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:28:28.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:28:28.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:28:28.797 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.797 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.797 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.797 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:28.797 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:28.797 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 83.428595603000005, 2023-12-06T18:28:28.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:28:28.797 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.797 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.797 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.797 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.798 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.798 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:28:28.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.798 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:28.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:28.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:28:28.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:28:28.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:28:28.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:28:28.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:28:28.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:28:28.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:28:28.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:28:28.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:28:28.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:28:28.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:28:28.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:28:28.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:28:28.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:28:28.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:28:28.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:28:28.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:28:28.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:28:28.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:28:28.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:28:28.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.802 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:28:28.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:28:28.802 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.802 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.802 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.802 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:28.802 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:28.802 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 83.427940175000003, 2023-12-06T18:28:28.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:28:28.803 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.803 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.803 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.803 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.803 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.803 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:28:28.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.803 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:28.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:28.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:28:28.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:28:28.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:28:28.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:28:28.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:28:28.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:28:28.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:28:28.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:28:28.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:28:28.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:28:28.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:28:28.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:28:28.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:28:28.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:28:28.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:28:28.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:28:28.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:28:28.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:28:28.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:28:28.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:28:28.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:28:28.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:28:28.807 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.807 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.807 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.807 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:28.807 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:28.807 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 83.387285628000001, 2023-12-06T18:28:28.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:28:28.808 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:28.808 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:28.808 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:28.808 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:28.808 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:28.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:28:28.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.808 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:28.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.808 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:28.808 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:28.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:28.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:28.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:28.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:28.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:28:28.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:28:28.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:28.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:28:28.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:28:28.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:28.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:28:28.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:28:28.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:28.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:28:28.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:28:28.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:28.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:28:28.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:28:28.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:28.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:28:28.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:28:28.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:28.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:28:28.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:28:28.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:28.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:28:28.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:28:28.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:28.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:28:28.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:28:28.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:28.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:28:28.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:28:28.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:28.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:28.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:28.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:28:28.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:28:28.812 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.812 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.812 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.813 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:28.813 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:28.813 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:28.814 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:28:28.815 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:28.811+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:28.916 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:28.913+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:28.946 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:28:28.993 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:28.993 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:28.993 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:28.993 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:29.003 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:28:29.047 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:29.044+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:29.117 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:29.114+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:29.156 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:29.156 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:29.156 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:29.156 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:29.156 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:29.165 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:28:29.217 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:29.214+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:29.319 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:29.315+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:29.349 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:29.349 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:29.349 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:29.350 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:29.350 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:29.350 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:28:29.350 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:29.350 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 67.533443390000002, 2023-12-06T18:28:29.350 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:28:29.350 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:29.350 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:28:29.350 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:29.350 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:29.350 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:29.350 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:29.350 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:29.350 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:29.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:29.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:29.351 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:29.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:29.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:29.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:29.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:29.351 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:28:29.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:28:29.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:29.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:29.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:29.351 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:28:29.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:28:29.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:29.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:29.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:29.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:28:29.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:28:29.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:29.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:29.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:29.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:28:29.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:28:29.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:29.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:29.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:29.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:28:29.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:28:29.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:29.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:29.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:29.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:28:29.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:28:29.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:29.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:29.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:29.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:28:29.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:28:29.353 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:29.353 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:29.353 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:29.353 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:29.353 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:29.353 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:29.359 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:28:29.401 INFO:tasks.ceph.mon.b.smithi152.stderr:2023-12-06T18:28:29.388+0000 ff5f640 -1 mon.b@1(leader).osd e43 definitely_dead 0 2023-12-06T18:28:29.419 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:29.416+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:29.520 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:29.517+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:29.533 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:29.533 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:29.533 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:29.533 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:29.542 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:28:29.621 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:29.618+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:29.637 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:28:29.650 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2023-12-06T18:28:29.650 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-12-06T18:28:29.650 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:28:29.664 INFO:tasks.ceph.mon.b.smithi152.stderr:2023-12-06T18:28:29.651+0000 9f54640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 86576) UID: 0 2023-12-06T18:28:29.665 INFO:tasks.ceph.mon.b.smithi152.stderr:2023-12-06T18:28:29.652+0000 9f54640 -1 mon.b@1(leader) e1 *** Got Signal Terminated *** 2023-12-06T18:28:29.682 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:29.682 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:29.682 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:29.682 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:29.682 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:29.691 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:28:29.734 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:29.730+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:29.823 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:29.820+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:29.888 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:29.888 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:29.888 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:29.888 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:29.888 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:29.900 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:28:29.945 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:29.941+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:29.952 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:28:30.025 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:30.022+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:30.053 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:30.053 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:30.053 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:30.053 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:30.066 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:28:30.126 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:30.124+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:30.159 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:02:41.480 86580== 2023-12-06T18:28:30.159 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:02:41.480 86580== Exit program on first error (--exit-on-first-error=yes) 2023-12-06T18:28:30.170 INFO:tasks.ceph.mon.b.smithi152.stderr:daemon-helper: command failed with exit status 42 2023-12-06T18:28:30.237 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:30.234+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:30.269 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:30.269 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:30.269 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:30.269 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:30.269 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:30.278 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:28:30.279 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-12-06T18:28:30.305 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:28:30.305 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:28:30.328 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:30.325+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:30.400 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:28:30.400 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:28:30.408 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:30.408 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:30.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:30.408 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:28:30.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:30.409 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:28:30.409 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:30.409 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 8.9148299340000001, 2023-12-06T18:28:30.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:28:30.409 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:30.409 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:28:30.409 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:28:30.409 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:28:30.409 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:28:30.409 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:28:30.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:30.409 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:30.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:30.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:30.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:30.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:30.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:30.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:30.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:30.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:30.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:30.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:30.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:30.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:30.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:28:30.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:28:30.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:30.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:30.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:30.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:28:30.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:28:30.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:30.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:30.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:30.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:28:30.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:28:30.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:30.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:30.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:30.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:28:30.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:28:30.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:30.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:30.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:30.412 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:28:30.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:28:30.412 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:30.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:30.412 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:30.412 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:28:30.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:28:30.412 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:30.412 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:30.412 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:30.412 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:30.412 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:30.412 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:30.418 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:28:30.428 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:30.426+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:30.496 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:28:30.496 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:28:30.529 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:30.525+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:30.547 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:30.547 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:30.547 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:30.547 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:30.556 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:28:30.589 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:28:30.589 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:28:30.641 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:30.638+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:30.680 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:28:30.681 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:28:30.730 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:30.727+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:30.784 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:28:30.784 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:28:30.788 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:30.788 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:30.797 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:28:30.831 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:30.829+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:30.882 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:28:30.883 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:28:30.891 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:28:30.894 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:28:30.895 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:28:30.932 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:30.929+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:30.969 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:30.969 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:30.979 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:28:31.034 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:31.031+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:31.134 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:31.131+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:31.149 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:31.149 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:31.159 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:28:31.236 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:31.233+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:31.287 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:31.288 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:31.298 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:28:31.337 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:31.334+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:31.426 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:31.426 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:31.435 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:28:31.437 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:31.433+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:31.538 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:31.535+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:31.583 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:31.583 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:31.592 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:28:31.676 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:31.672+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:31.740 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:31.736+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:31.771 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:31.771 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:31.780 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:28:31.865 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:31.861+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:31.942 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:31.939+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:31.957 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:31.957 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:31.968 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:28:32.043 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:32.039+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:32.144 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:32.144 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:32.145 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:32.141+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:32.153 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:28:32.245 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:32.241+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:32.330 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:32.330 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:32.340 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:28:32.345 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:32.342+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:32.446 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:32.442+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:32.519 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:32.520 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:32.529 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:28:32.532 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-12-06T18:28:32.572 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:32.567+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:32.648 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:32.645+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:32.706 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:32.706 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:32.716 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:28:32.762 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:32.758+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:32.851 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:32.847+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:32.853 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:32.853 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:32.863 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:28:32.895 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:28:32.951 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:32.948+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:33.045 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:33.045 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:33.052 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:33.048+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:33.054 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:28:33.152 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:33.149+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:33.171 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:28:33.171 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:28:33.195 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:33.195 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:33.205 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:28:33.258 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:33.255+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:33.268 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:28:33.268 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:28:33.354 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:33.350+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:33.358 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:33.358 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:33.367 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:28:33.375 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:28:33.375 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:28:33.455 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:33.451+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:33.467 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:28:33.468 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:28:33.506 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:33.506 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:33.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:33.506 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:33.506 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:33.515 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:28:33.559 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:28:33.559 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:28:33.559 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:33.555+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:33.651 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:28:33.652 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:28:33.655 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:33.655 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:33.656 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:33.656 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:33.656 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:33.652+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:33.665 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:28:33.740 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:28:33.740 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:28:33.743 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:28:33.746 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:28:33.746 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:28:33.757 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:33.754+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:33.834 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:33.834 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:33.834 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:33.834 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:33.834 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:33.844 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:28:33.858 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:33.854+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:33.959 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:33.955+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:33.974 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:28:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:28:33.974 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 12.48076745, 2023-12-06T18:28:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:28:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:28:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:28:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:28:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:28:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:28:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:33.975 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:28:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:28:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:28:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:28:33.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:28:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:28:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:28:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:28:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:28:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:28:33.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:28:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:28:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:33.978 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:33.978 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:33.983 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:28:34.060 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:34.056+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:34.112 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:34.112 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:34.112 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:34.112 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:34.121 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:28:34.161 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:34.157+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:34.261 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:34.258+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:34.290 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:34.290 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:34.290 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:34.290 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:34.290 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:34.299 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:28:34.363 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:34.359+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:34.464 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:34.460+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:34.470 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:34.471 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:34.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:34.471 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:28:34.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.471 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:28:34.471 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:34.471 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 97.855811224000007, 2023-12-06T18:28:34.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:28:34.471 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.471 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.471 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:34.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:34.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:28:34.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:28:34.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:28:34.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:28:34.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:28:34.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:28:34.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:28:34.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:28:34.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:28:34.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:28:34.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:28:34.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:28:34.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:28:34.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:28:34.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:28:34.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:28:34.475 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.475 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.475 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.475 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:28:34.475 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:34.475 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 96.832984214999996, 2023-12-06T18:28:34.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:28:34.475 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.475 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.475 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:34.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:34.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:28:34.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:28:34.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:28:34.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:28:34.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:28:34.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:28:34.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:28:34.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:28:34.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:28:34.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:28:34.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:28:34.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:28:34.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:28:34.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:28:34.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:28:34.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:28:34.478 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.479 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.479 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.479 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:28:34.479 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:34.479 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 96.820937576999995, 2023-12-06T18:28:34.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:28:34.479 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.479 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.479 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:34.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:34.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:28:34.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:28:34.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:28:34.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:28:34.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:28:34.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:28:34.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:28:34.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:28:34.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:28:34.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:28:34.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:28:34.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:28:34.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:28:34.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:28:34.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:28:34.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:28:34.482 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.483 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.483 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.483 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:28:34.483 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:34.483 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 96.799536970000005, 2023-12-06T18:28:34.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:28:34.483 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.483 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.483 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:34.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:34.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:28:34.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:28:34.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:28:34.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:28:34.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:28:34.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:28:34.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:28:34.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:28:34.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:28:34.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:28:34.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:28:34.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:28:34.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:28:34.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:28:34.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:28:34.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:28:34.486 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.487 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.487 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.487 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:28:34.487 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:34.487 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 96.798357647000003, 2023-12-06T18:28:34.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:28:34.487 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.487 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.487 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.487 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.487 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:34.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:34.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:28:34.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:28:34.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:28:34.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:28:34.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:28:34.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:28:34.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:28:34.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:28:34.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:28:34.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:28:34.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:28:34.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:28:34.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:28:34.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:28:34.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:28:34.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:28:34.490 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.490 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.491 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.491 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.491 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:28:34.491 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:34.491 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 96.797785665999996, 2023-12-06T18:28:34.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:28:34.491 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.491 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.491 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.491 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.491 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:34.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.491 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:34.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:28:34.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:28:34.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:28:34.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:28:34.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:28:34.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:28:34.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:28:34.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:28:34.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:28:34.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:28:34.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:28:34.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:28:34.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:28:34.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:28:34.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:28:34.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:28:34.494 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.494 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.494 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.495 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:28:34.495 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:34.495 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 96.351950196000004, 2023-12-06T18:28:34.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:28:34.495 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.495 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.495 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.495 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:34.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:34.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:28:34.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:28:34.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:28:34.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:28:34.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:28:34.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:28:34.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:28:34.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:28:34.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:28:34.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:28:34.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:28:34.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:28:34.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:28:34.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:28:34.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:28:34.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:28:34.498 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.498 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.498 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.499 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:28:34.499 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:34.499 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 96.350350340000006, 2023-12-06T18:28:34.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:28:34.499 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.499 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.499 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.499 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.499 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:34.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.499 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:34.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:28:34.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:28:34.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:28:34.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:28:34.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:28:34.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:28:34.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:28:34.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:28:34.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:28:34.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:28:34.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:28:34.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:28:34.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:28:34.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:28:34.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:28:34.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:28:34.502 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.502 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.502 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.503 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:28:34.503 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:34.503 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 95.971144723999998, 2023-12-06T18:28:34.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:28:34.503 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.503 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.503 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:34.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:34.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:28:34.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:28:34.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:28:34.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:28:34.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:28:34.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:28:34.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:28:34.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:28:34.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:28:34.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:28:34.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:28:34.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:28:34.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:28:34.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:28:34.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:28:34.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:28:34.506 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.506 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.506 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.507 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:28:34.507 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:34.507 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 95.733642541999998, 2023-12-06T18:28:34.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:28:34.507 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.507 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.507 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:34.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:34.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:28:34.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:28:34.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:28:34.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:28:34.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:28:34.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:28:34.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:28:34.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:28:34.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:28:34.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:28:34.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:28:34.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:28:34.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:28:34.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:28:34.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:28:34.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:28:34.510 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.510 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.510 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.510 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:28:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 95.732914543000007, 2023-12-06T18:28:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:28:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:28:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:28:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:28:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:28:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:28:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:28:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:28:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:28:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:28:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:28:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:28:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:28:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:28:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:28:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:28:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:28:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:28:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 95.727902133000001, 2023-12-06T18:28:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:28:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:28:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:28:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:28:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:28:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:28:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:28:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:28:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:28:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:28:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:28:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:28:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:28:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:28:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:28:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:28:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:28:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:28:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 95.722375053999997, 2023-12-06T18:28:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:28:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:28:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:28:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:28:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:28:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:28:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:28:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:28:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:28:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:28:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:28:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:28:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:28:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:28:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:28:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:28:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:28:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:28:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 95.721864217000004, 2023-12-06T18:28:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:28:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:28:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:28:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:28:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:28:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:28:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:28:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:28:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:28:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:28:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:28:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:28:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:28:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:28:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:28:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:28:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:28:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:28:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 95.721404956000001, 2023-12-06T18:28:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:28:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:28:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:28:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:28:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:28:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:28:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:28:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:28:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:28:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:28:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:28:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:28:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:28:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:28:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:28:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:28:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:28:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:28:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 89.141489148999995, 2023-12-06T18:28:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:28:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:28:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:28:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:28:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:28:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:28:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:28:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:28:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:28:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:28:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:28:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:28:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:28:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:28:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:28:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:28:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:28:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:28:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 89.135251343999997, 2023-12-06T18:28:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:28:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:28:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:28:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:28:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:28:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:28:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:28:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:28:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:28:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:28:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:28:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:28:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:28:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:28:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:28:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:28:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:28:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:28:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 89.129188264999996, 2023-12-06T18:28:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:28:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:28:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:28:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:28:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:28:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:28:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:28:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:28:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:28:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:28:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:28:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:28:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:28:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:28:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:28:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:28:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:28:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:28:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 88.991801179999996, 2023-12-06T18:28:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:28:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:28:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:28:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:28:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:28:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:28:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:28:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:28:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:28:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:28:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:28:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:28:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:28:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:28:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:28:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:28:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:28:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:28:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 88.985130452999996, 2023-12-06T18:28:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:28:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:28:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:28:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:28:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:28:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:28:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:28:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:28:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:28:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:28:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:28:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:28:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:28:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:28:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:28:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:28:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:28:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.550 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.550 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:34.550 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:34.550 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:34.551 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:28:34.564 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:34.561+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:34.665 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:34.662+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:34.719 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:34.719 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:34.719 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:34.719 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:34.728 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:28:34.766 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:34.762+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:34.855 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:34.855 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:34.856 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:34.856 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:34.856 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:34.864 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:28:34.866 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:34.862+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:34.967 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:34.963+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:34.998 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:34.998 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:34.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:34.999 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:28:34.999 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:35.008 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:28:35.067 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:35.063+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:35.154 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:35.155 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:35.155 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:35.155 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:35.164 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:28:35.167 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:35.164+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:35.268 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:35.265+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:35.339 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:35.339 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:35.339 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:35.340 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:35.340 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:35.348 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:28:35.369 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:35.365+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:35.470 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:35.466+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:35.525 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:35.525 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:35.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:35.525 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:35.525 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:35.535 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:28:35.577 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:35.573+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:35.671 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:35.668+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:35.714 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:35.714 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:35.714 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:35.714 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:35.723 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:28:35.748 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:28:35.751 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-12-06T18:28:35.751 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi152 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b" 2023-12-06T18:28:35.752 INFO:tasks.ceph.mon.b:Stopped 2023-12-06T18:28:35.752 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2023-12-06T18:28:35.752 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:28:35.811 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:35.807+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:35.862 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:35.862 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:35.862 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:35.862 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:35.862 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:35.871 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:28:35.913 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:35.909+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:35.974 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:35.971+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:36.075 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:36.072+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:36.126 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:36.127 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:36.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:36.127 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:36.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.127 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:28:36.127 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:36.127 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 100.716122908, 2023-12-06T18:28:36.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:28:36.127 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.129 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.129 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.129 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.129 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:28:36.129 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.129 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.129 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.129 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.129 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:36.129 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:28:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:28:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:28:36.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:28:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:28:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:28:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:28:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:28:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:28:36.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:28:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:28:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:28:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:28:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:28:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:28:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:28:36.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:28:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:28:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:28:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:28:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:28:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:28:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:28:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:28:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:28:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:28:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:28:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 98.607549668999994, 2023-12-06T18:28:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:28:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:28:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:28:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:28:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:28:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:28:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:28:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:28:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:28:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:28:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:28:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:28:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:28:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:28:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:28:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:28:36.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:28:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:28:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:28:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:28:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:28:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:28:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:28:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:28:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 98.606804436999994, 2023-12-06T18:28:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:28:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.140 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:28:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:28:36.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:28:36.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:28:36.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:28:36.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:28:36.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:28:36.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:28:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:28:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:28:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:28:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:28:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:28:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:28:36.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:28:36.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:28:36.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:28:36.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:28:36.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:28:36.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:28:36.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:28:36.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:28:36.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:28:36.145 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.145 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.145 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.145 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:36.145 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:36.145 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 98.606120681999997, 2023-12-06T18:28:36.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:28:36.146 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.146 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.146 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.146 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.146 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.146 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:28:36.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.146 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:36.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:28:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:28:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:28:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:28:36.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:28:36.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:28:36.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:28:36.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:28:36.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:28:36.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:28:36.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:28:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:28:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:28:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:28:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:28:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:28:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:28:36.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:28:36.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:28:36.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:28:36.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:28:36.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:28:36.150 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.150 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 98.600547426999995, 2023-12-06T18:28:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:28:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:28:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:28:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:28:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:28:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:28:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:28:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:28:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:28:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:28:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.153 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:28:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:28:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:28:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:28:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:28:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:28:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:28:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:28:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:28:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:28:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:28:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:28:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:28:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:28:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 98.599873787000007, 2023-12-06T18:28:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:28:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.156 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:28:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:28:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:28:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:28:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:28:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:28:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:28:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.158 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:28:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:28:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:28:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:28:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:28:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:28:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.159 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:28:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:28:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:28:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:28:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:28:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:28:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:28:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:28:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:28:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:28:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 98.599211588000003, 2023-12-06T18:28:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:28:36.161 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:28:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:28:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:28:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:28:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:28:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.163 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:28:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:28:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:28:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:28:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:28:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:28:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:28:36.164 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:28:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:28:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:28:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:28:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:28:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:28:36.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:28:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:28:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:28:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:28:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:28:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.166 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 98.526502152999996, 2023-12-06T18:28:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:28:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:28:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.167 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:28:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:28:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.168 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:28:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:28:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:28:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:28:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:28:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:28:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:28:36.169 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:28:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:28:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:28:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:28:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:28:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:28:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:28:36.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:28:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:28:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:28:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:28:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:28:36.171 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:28:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 98.525820444000004, 2023-12-06T18:28:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:28:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:28:36.172 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:28:36.173 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:28:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:28:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:28:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:28:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:28:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:28:36.174 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:28:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:28:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:28:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:28:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:28:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:28:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:28:36.175 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:28:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:28:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:28:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:28:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:28:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:28:36.176 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:28:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:28:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 97.447016356000006, 2023-12-06T18:28:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:28:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.177 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:28:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:36.178 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:28:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:28:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:28:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:28:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:28:36.179 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:28:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:28:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:28:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:28:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:28:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:28:36.180 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:28:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:28:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:28:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:28:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:28:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:28:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:28:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:28:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:28:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:28:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:28:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.182 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 97.445599782000002, 2023-12-06T18:28:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:28:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:28:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:36.183 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:28:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:28:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:28:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:28:36.184 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:28:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:28:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:28:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:28:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:28:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:28:36.185 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:28:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:28:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:28:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:28:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:28:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:28:36.186 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:28:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:28:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:28:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:28:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:28:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:28:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.187 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 97.444514656999999, 2023-12-06T18:28:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:28:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:28:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.188 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:28:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:28:36.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:28:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:28:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:28:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:28:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:28:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:28:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:28:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:28:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:28:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:28:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:28:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:28:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:28:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:28:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:28:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:28:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:28:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:28:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:28:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:28:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 97.409365554999994, 2023-12-06T18:28:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:28:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:28:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:28:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:28:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:28:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:28:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:28:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:28:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:28:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:28:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:28:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:28:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:28:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:28:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:28:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:28:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:28:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:28:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:28:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:28:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:28:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:28:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:28:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:28:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 90.859635030000007, 2023-12-06T18:28:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:28:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:28:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:36.199 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:28:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:28:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:28:36.200 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:28:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:28:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:28:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:28:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:28:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:28:36.201 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:28:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:28:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:28:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:28:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:28:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.202 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:28:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:28:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:28:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:28:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:28:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:28:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.203 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:28:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:28:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 90.858829421999999, 2023-12-06T18:28:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:28:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.204 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:28:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.205 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:28:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:28:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:28:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:28:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:28:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:28:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.206 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:28:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:28:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:28:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:28:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:28:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:28:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.207 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:28:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:28:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:28:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:28:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:28:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:28:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.208 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:28:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:28:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:28:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:28:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 90.858138612999994, 2023-12-06T18:28:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:28:36.209 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:28:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.210 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:28:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:28:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:28:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:28:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:28:36.211 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:28:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:28:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:28:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:28:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:28:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:28:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:28:36.212 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:28:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:28:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:28:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:28:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:28:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:28:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.213 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:28:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:28:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:28:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:28:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.214 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 90.853841105000001, 2023-12-06T18:28:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:28:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:28:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.215 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:28:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:28:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:28:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:28:36.216 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:28:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:28:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:28:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:28:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:28:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:28:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.217 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:28:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:28:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:28:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:28:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:28:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:28:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.218 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:28:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:28:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:28:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:28:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:28:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:28:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.219 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 90.853173429999998, 2023-12-06T18:28:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:28:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:28:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.220 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:28:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:28:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.221 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:28:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:28:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:28:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:28:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:28:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:28:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.222 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:28:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:28:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:28:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:28:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:28:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:28:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:28:36.223 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:28:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:28:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:28:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:28:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:28:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:28:36.224 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:28:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 90.852518001999996, 2023-12-06T18:28:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:28:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:28:36.225 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:28:36.226 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:28:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:28:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:28:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:28:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:28:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:28:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:28:36.227 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:28:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:28:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:28:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:28:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:28:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:28:36.228 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:28:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:28:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:28:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:28:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:28:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:28:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.229 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:28:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:28:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 90.811863454999994, 2023-12-06T18:28:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:28:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:36.230 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:28:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.231 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:28:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:28:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:28:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:28:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:28:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:28:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.232 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:28:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:28:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:28:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:28:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:28:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:28:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:28:36.233 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:28:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:28:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:28:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:28:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:28:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:28:36.234 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:28:36.235 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.235 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.235 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.235 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:28:36.235 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:28:36.235 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.235 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.235 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.235 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.235 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.235 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:36.237 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:28:36.237 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:36.233+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:36.338 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:36.334+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:36.426 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:36.426 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:36.426 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:36.426 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:36.436 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:28:36.442 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:36.435+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:36.509 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":9,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:28:36.539 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:36.535+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:36.587 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:36.587 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:36.587 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:36.587 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:36.587 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:36.596 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:28:36.640 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:36.637+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:36.736 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:36.736 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:36.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:36.736 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:36.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.736 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:28:36.736 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:36.736 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 74.919918444999993, 2023-12-06T18:28:36.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:28:36.736 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:36.737 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:28:36.737 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:36.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:36.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:36.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:36.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:36.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:28:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:28:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:28:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:28:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:28:36.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:28:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:28:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:28:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:28:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:28:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:28:36.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:28:36.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:36.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:36.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:36.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:28:36.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:28:36.740 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.740 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.740 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.740 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:36.740 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:36.740 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:36.742 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:36.738+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:36.745 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:28:36.843 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:36.839+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:36.923 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:36.923 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:36.923 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:36.923 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:36.933 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:28:36.977 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:36.973+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:37.045 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:37.041+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:37.080 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:37.080 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:37.080 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:37.080 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:37.080 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:37.090 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:28:37.146 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:37.142+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:37.247 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:37.243+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:37.269 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:37.270 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:37.270 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:37.270 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:37.270 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:37.279 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:28:37.348 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:37.345+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:37.417 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:37.417 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:37.417 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:37.418 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:37.426 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:28:37.449 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:37.445+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:37.550 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:37.546+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:37.553 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:37.553 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:37.554 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:37.554 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:37.554 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:37.563 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:28:37.650 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:37.647+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:37.695 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:37.695 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:37.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:37.696 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:28:37.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:37.696 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:28:37.696 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:37.696 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.202528704999999, 2023-12-06T18:28:37.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:28:37.696 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:37.696 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:28:37.696 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:28:37.696 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:28:37.696 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:28:37.696 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:28:37.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:37.696 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:37.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:37.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:37.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:37.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:37.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:37.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:37.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:37.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:37.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:37.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:37.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:37.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:37.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:28:37.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:28:37.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:37.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:37.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:37.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:28:37.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:28:37.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:37.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:37.698 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:37.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:28:37.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:28:37.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:37.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:37.698 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:37.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:28:37.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:28:37.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:37.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:37.698 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:37.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:28:37.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:28:37.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:37.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:37.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:37.699 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:28:37.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:28:37.699 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:37.699 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:37.699 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:37.699 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:37.699 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:37.699 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:37.706 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:28:37.751 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:37.748+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:37.834 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:37.834 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:37.834 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:37.834 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:37.843 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:28:37.852 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:37.849+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:37.953 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:37.949+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:38.012 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:38.012 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:38.012 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:38.013 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:38.013 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:38.022 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:28:38.065 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:38.060+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:38.154 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:38.150+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:38.234 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:28:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:28:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 101.616070106, 2023-12-06T18:28:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:28:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.234 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:28:38.235 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:28:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:28:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:28:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:28:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:28:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:28:38.236 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:28:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:28:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:28:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:28:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:28:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:28:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:28:38.237 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:28:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:28:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:28:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 100.593243097, 2023-12-06T18:28:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:28:38.238 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.239 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:28:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:28:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:28:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:28:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:28:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:28:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.240 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:28:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:28:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:28:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:28:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:28:38.241 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:28:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:28:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:28:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:28:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:28:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.242 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:28:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 100.581196459, 2023-12-06T18:28:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:28:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.243 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:28:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:28:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:28:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:28:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.244 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:28:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:28:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:28:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:28:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:28:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:28:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.245 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:28:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:28:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:28:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:28:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:28:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:28:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.246 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:28:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 100.55979585199999, 2023-12-06T18:28:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:28:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.247 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:28:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:28:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:28:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:28:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.248 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:28:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:28:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:28:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:28:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:28:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:28:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.249 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:28:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:28:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:28:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:28:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:28:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:28:38.250 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:28:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 100.55861652900001, 2023-12-06T18:28:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:28:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:38.251 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:28:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:28:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:28:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:28:38.252 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:28:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:28:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:28:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:28:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:28:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:28:38.253 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:28:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:28:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:28:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:28:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:28:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:28:38.254 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:28:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 100.558044548, 2023-12-06T18:28:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:28:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:38.255 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:28:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:28:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:28:38.256 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:28:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:28:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:28:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:28:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:28:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:28:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:28:38.257 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:28:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:28:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:28:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:28:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:28:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:28:38.258 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:28:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 100.11220907800001, 2023-12-06T18:28:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:28:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:38.259 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:28:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:28:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:28:38.260 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:28:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:28:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:28:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:28:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:28:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:28:38.261 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:28:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:28:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:28:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:28:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:28:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.262 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:28:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:28:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:28:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 100.11060922199999, 2023-12-06T18:28:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:28:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.263 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:28:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:28:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.264 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:28:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:28:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:28:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:28:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:28:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:28:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.265 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:28:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:28:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:28:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:28:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:28:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:28:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.266 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:28:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:28:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:28:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 99.731403606000001, 2023-12-06T18:28:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:28:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.267 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:28:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:28:38.268 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:28:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:28:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:28:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:28:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:28:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:28:38.269 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:28:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:28:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:28:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:28:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:28:38.270 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:28:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:28:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:28:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:28:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 99.493901424000001, 2023-12-06T18:28:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:28:38.271 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.272 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:28:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:28:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:28:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:28:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:28:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:28:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.273 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:28:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:28:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:28:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:28:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:28:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:28:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.274 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:28:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:28:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:28:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:28:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:28:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:38.275 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 99.493173424999995, 2023-12-06T18:28:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:28:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.276 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:28:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:28:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:28:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:28:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:28:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:28:38.277 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:28:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:28:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:28:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:28:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.278 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:28:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:28:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:28:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:28:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:28:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:28:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.279 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:28:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 99.488161015000003, 2023-12-06T18:28:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:28:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.280 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:28:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:28:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:28:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:28:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:28:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:28:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:28:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:28:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:28:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:28:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.282 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:28:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:28:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:28:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:28:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:28:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:28:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.283 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:28:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 99.482633935999999, 2023-12-06T18:28:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:28:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.284 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:28:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:28:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:28:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:28:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:28:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:28:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:28:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:28:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:28:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:28:38.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:28:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:28:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:28:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:28:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:28:38.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:28:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:28:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 99.482123099000006, 2023-12-06T18:28:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:28:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.288 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:28:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:28:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.289 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:28:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:28:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:28:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:28:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:28:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:28:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:28:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:28:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:28:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:28:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:28:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:28:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:28:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:28:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:28:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 99.481663838000003, 2023-12-06T18:28:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:28:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.292 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:28:38.293 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:28:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:28:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:28:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:28:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:28:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.294 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:28:38.295 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:28:38.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.295 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.295 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.295 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:28:38.295 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:28:38.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.295 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.295 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.295 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:28:38.295 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:28:38.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.295 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.295 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.295 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:28:38.296 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:28:38.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.296 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.296 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.296 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:28:38.296 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:28:38.296 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.296 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.296 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.296 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.296 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:28:38.296 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:38.296 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 92.901748030999997, 2023-12-06T18:28:38.296 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:28:38.296 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.297 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.297 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.297 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.297 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.297 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:38.297 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.297 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.297 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.297 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:38.297 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.297 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.297 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.298 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:28:38.298 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:28:38.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.298 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.298 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.298 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:28:38.298 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:28:38.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.298 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.298 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.298 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:28:38.298 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:28:38.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.298 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.299 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.299 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:28:38.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:28:38.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.299 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.299 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:28:38.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:28:38.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.299 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.299 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:28:38.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:28:38.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:28:38.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:28:38.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:28:38.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:28:38.300 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.300 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.300 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.300 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:28:38.300 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:38.301 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 92.895510225999999, 2023-12-06T18:28:38.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:28:38.301 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.301 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.301 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.301 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.301 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:38.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.301 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.301 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:38.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.302 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.302 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:28:38.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:28:38.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.302 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.302 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:28:38.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:28:38.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.302 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.302 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:28:38.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:28:38.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:28:38.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:28:38.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:28:38.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:28:38.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:28:38.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:28:38.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.304 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:28:38.304 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:28:38.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.304 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:28:38.304 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:28:38.304 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.304 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.304 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.305 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.305 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:28:38.305 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:38.305 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 92.889447146999998, 2023-12-06T18:28:38.305 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:28:38.305 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.305 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.305 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.305 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.305 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.305 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:38.305 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.305 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.305 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.305 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:38.305 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.306 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.306 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.306 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:28:38.306 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:28:38.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.306 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.306 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.306 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:28:38.306 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:28:38.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.306 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.306 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.306 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:28:38.306 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:28:38.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.307 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.307 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:28:38.307 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:28:38.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.307 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.307 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:28:38.307 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:28:38.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.307 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.307 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:28:38.307 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:28:38.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.308 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.308 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:28:38.308 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:28:38.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.308 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.308 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:28:38.308 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:28:38.308 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.308 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.308 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.308 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:28:38.309 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:38.309 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 92.752060061999998, 2023-12-06T18:28:38.309 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:28:38.309 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.309 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.309 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.309 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.309 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.309 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:38.309 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.309 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.309 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.309 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:38.310 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.310 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.310 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.310 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:28:38.310 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:28:38.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.310 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.310 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.310 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:28:38.310 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:28:38.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.310 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.310 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.311 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:28:38.311 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:28:38.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.311 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.311 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:28:38.311 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:28:38.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.311 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.311 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:28:38.311 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:28:38.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.311 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.312 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:28:38.312 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:28:38.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.312 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.312 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.312 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:28:38.312 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:28:38.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.312 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.312 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.312 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:28:38.312 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:28:38.312 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.312 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.312 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.312 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.312 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:28:38.313 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:38.313 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 92.745389334999999, 2023-12-06T18:28:38.313 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:28:38.313 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:38.313 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:38.313 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:38.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.313 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:38.313 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:38.313 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.313 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:38.313 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:38.313 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:38.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.314 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:38.314 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:28:38.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:28:38.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.314 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:38.314 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:28:38.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:28:38.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.314 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:38.314 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:28:38.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:28:38.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:38.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:28:38.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:28:38.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:38.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:28:38.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:28:38.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:38.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:28:38.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:28:38.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.316 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:38.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:28:38.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:28:38.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:38.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:38.316 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:38.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:28:38.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:28:38.316 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.316 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.316 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.316 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:38.316 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:38.316 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:38.318 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:28:38.318 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:38.315+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:38.419 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:38.415+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:38.503 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:38.504 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:38.504 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:38.504 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:38.514 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:28:38.520 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:38.516+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:38.621 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:38.617+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:38.641 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:38.641 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:38.641 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:38.641 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:38.641 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:38.651 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:28:38.720 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-12-06T18:28:38.721 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:38.718+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:38.782 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:38.783 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:38.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:38.783 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:28:38.783 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:38.794 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:28:38.839 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:38.835+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:38.930 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:38.926+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:38.945 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:38.945 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:38.945 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:38.945 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:38.955 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:28:39.041 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:39.037+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:39.126 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:39.122+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:39.133 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:39.133 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:39.133 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:39.133 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:39.133 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:39.145 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:28:39.226 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:39.223+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:39.321 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:39.321 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:39.321 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:39.321 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:39.322 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:39.327 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:39.323+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:39.330 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:28:39.428 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:39.424+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:39.509 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:39.509 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:39.509 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:39.509 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:39.518 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:28:39.523 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:28:39.560 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:39.556+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:39.630 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:39.627+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:39.705 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:39.728 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:39.728 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:39.728 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:39.728 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:39.730 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:28:39.731 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:39.728+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:39.833 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:39.829+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:39.964 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:39.964 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:39.964 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:39.964 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:39.964 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.964 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:28:39.964 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:39.964 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 104.553810493, 2023-12-06T18:28:39.964 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:28:39.965 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:39.965 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:39.965 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:39.965 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:39.965 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:39.965 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:28:39.965 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.965 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:39.965 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.965 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:39.965 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:39.965 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:39.965 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.965 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.965 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:39.966 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:39.966 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:39.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.966 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.966 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:39.966 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:28:39.966 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:28:39.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.966 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.966 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:39.966 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:28:39.966 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:28:39.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.966 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.967 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:39.967 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:28:39.967 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:28:39.967 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.967 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.967 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:39.967 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:28:39.967 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:28:39.967 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.967 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.967 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:39.967 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:28:39.967 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:28:39.967 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.968 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.968 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:28:39.968 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:28:39.968 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:28:39.968 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.968 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.968 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:39.968 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:28:39.968 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:28:39.968 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.968 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.968 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:39.968 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:28:39.968 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:28:39.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.969 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.969 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:39.969 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:28:39.969 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:28:39.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.969 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.969 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:39.969 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:28:39.969 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:28:39.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.969 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.969 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:39.969 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:28:39.969 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:28:39.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.970 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.970 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:39.970 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:28:39.970 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:28:39.970 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.970 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.970 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:39.970 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:28:39.970 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:28:39.970 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:39.970 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:39.970 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:39.970 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.971 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.971 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:39.971 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:39.971 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 102.44523725400001, 2023-12-06T18:28:39.971 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:28:39.971 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:39.971 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:39.971 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:39.971 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:39.971 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:39.971 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:28:39.971 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.971 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:39.971 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.971 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:39.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:39.972 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:39.972 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.972 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.972 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:39.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:39.972 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:39.972 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.972 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.972 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:39.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:28:39.972 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:28:39.972 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.972 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.972 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:39.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:28:39.973 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:28:39.973 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.973 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.973 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:39.973 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:28:39.973 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:28:39.973 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.973 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.973 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:39.973 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:28:39.973 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:28:39.973 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.973 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.973 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:39.974 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:28:39.974 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:28:39.974 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.974 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.974 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:39.974 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:28:39.974 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:28:39.974 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.974 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.974 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:39.974 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:28:39.974 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:28:39.974 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.974 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.974 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:39.974 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:28:39.975 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:28:39.975 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.975 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.975 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:39.975 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:28:39.975 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:28:39.975 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.975 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.975 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:39.975 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:28:39.975 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:28:39.975 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.975 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.975 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:39.975 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:28:39.975 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:28:39.975 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:39.976 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:39.976 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:39.976 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.976 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.976 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:39.976 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:39.976 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 102.44449202200001, 2023-12-06T18:28:39.976 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:28:39.976 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:39.976 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:39.976 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:39.976 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:39.976 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:39.976 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:28:39.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.977 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:39.977 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.977 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:39.977 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:39.977 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:39.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.977 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.977 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:39.977 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:39.977 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:39.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.977 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.977 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:39.977 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:28:39.977 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:28:39.978 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.978 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.978 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:39.978 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:28:39.978 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:28:39.978 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.978 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.978 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:39.978 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:28:39.978 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:28:39.978 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.978 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.978 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:39.978 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:28:39.979 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:28:39.979 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.979 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.979 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:39.979 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:28:39.979 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:28:39.979 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.979 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.979 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:39.979 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:28:39.979 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:28:39.979 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.979 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.979 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:39.979 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:28:39.979 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:28:39.979 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.980 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:39.980 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:28:39.980 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:28:39.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.980 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:39.980 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:28:39.980 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:28:39.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.980 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:39.980 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:28:39.980 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:28:39.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.981 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:39.981 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:28:39.981 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:28:39.981 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:39.981 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:39.981 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:39.981 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.981 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.981 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:39.981 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:39.981 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 102.44380826699999, 2023-12-06T18:28:39.981 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:28:39.981 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:39.981 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:39.981 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:39.981 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:39.982 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:39.982 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:28:39.982 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.982 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:39.982 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.982 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:39.982 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:39.982 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:39.982 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.982 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.982 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:39.982 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:39.982 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:39.982 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.982 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.983 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:39.983 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:28:39.983 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:28:39.983 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.983 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.983 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:39.983 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:28:39.983 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:28:39.983 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.983 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.983 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:39.983 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:28:39.983 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:28:39.983 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.983 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.984 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:39.984 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:28:39.984 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:28:39.984 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.984 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.984 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:39.984 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:28:39.984 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:28:39.984 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.984 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.984 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:39.984 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:28:39.984 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:28:39.984 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.984 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.984 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:39.984 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:28:39.985 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:28:39.985 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.985 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.985 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:39.985 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:28:39.985 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:28:39.985 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.985 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.985 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:39.985 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:28:39.985 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:28:39.985 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.985 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.985 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:39.985 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:28:39.985 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:28:39.985 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.986 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.986 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:39.986 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:28:39.986 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:28:39.986 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:39.986 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:39.986 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:39.986 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.986 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.986 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:39.986 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:39.986 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 102.43823501200001, 2023-12-06T18:28:39.986 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:28:39.986 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:39.987 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:39.987 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:39.987 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:39.987 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:39.987 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:28:39.987 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.987 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:39.987 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.987 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:39.987 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:39.987 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:39.987 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.987 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.987 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:39.987 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:39.988 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:39.988 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.988 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.988 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:39.988 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:28:39.988 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:28:39.988 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.988 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.988 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:39.988 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:28:39.988 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:28:39.988 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.988 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.988 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:39.988 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:28:39.989 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:28:39.989 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.989 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.989 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:39.989 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:28:39.989 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:28:39.989 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.989 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.989 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:39.989 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:28:39.989 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:28:39.989 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.989 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.989 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:39.989 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:28:39.990 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:28:39.990 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.990 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.990 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:39.990 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:28:39.990 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:28:39.990 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.990 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.990 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:39.990 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:28:39.990 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:28:39.990 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.990 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.990 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:39.991 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:28:39.991 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:28:39.991 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.991 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.991 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:39.991 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:28:39.991 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:28:39.991 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.991 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.991 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:39.991 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:28:39.991 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:28:39.991 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:39.991 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:39.991 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:39.991 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.992 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.992 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:39.992 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:39.992 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 102.437561372, 2023-12-06T18:28:39.992 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:28:39.992 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:39.992 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:39.992 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:39.992 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:39.992 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:39.992 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:28:39.992 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.992 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:39.992 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.992 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:39.992 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:39.993 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:39.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.993 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.993 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:39.993 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:39.993 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:39.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.993 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.993 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:39.993 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:28:39.993 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:28:39.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.993 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.993 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:39.993 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:28:39.993 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:28:39.994 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.994 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.994 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:39.994 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:28:39.994 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:28:39.994 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.994 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.994 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:39.994 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:28:39.994 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:28:39.994 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.994 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.994 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:39.994 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:28:39.994 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:28:39.995 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.995 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.995 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:39.995 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:28:39.995 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:28:39.995 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.995 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.995 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:39.995 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:28:39.995 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:28:39.995 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.995 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.995 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:39.995 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:28:39.995 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:28:39.995 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.996 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.996 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:39.996 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:28:39.996 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:28:39.996 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.996 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.996 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:39.996 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:28:39.996 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:28:39.996 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.996 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.996 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:39.996 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:28:39.996 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:28:39.996 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:39.997 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:39.997 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:39.997 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.997 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.997 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:39.997 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:39.997 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 102.436899173, 2023-12-06T18:28:39.997 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:28:39.997 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:39.997 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:39.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:39.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:39.997 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:39.997 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:28:39.998 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.998 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:39.998 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.998 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:39.998 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:39.998 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:39.998 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.998 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.998 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:39.998 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:39.998 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:39.998 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.998 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.998 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:39.999 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:28:39.999 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:28:39.999 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.999 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.999 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:39.999 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:28:39.999 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:28:39.999 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.999 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.999 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:39.999 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:28:39.999 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:28:39.999 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:39.999 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:39.999 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:39.999 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:28:39.999 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:28:40.000 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.000 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.000 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:40.000 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:28:40.000 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:28:40.000 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.000 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.000 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:40.000 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:28:40.000 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:28:40.000 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.000 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.000 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:40.000 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:28:40.000 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:28:40.001 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.001 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.001 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:40.001 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:28:40.001 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:28:40.001 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.001 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.001 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:40.001 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:28:40.001 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:28:40.001 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.001 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.001 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:40.001 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:28:40.001 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:28:40.002 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.002 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.002 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:40.002 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:28:40.002 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:28:40.002 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.002 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:40.002 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.002 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.002 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.002 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:40.002 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:40.002 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 102.36418973799999, 2023-12-06T18:28:40.003 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:28:40.003 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:40.003 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:40.003 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:40.003 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:40.003 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:40.003 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:28:40.003 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.003 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:40.003 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.003 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:40.003 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:40.003 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.003 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.003 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.004 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:40.004 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:40.004 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.004 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.004 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.004 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:40.004 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:28:40.004 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:28:40.004 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.004 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.004 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:40.004 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:28:40.004 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:28:40.004 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.004 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.005 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:40.005 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:28:40.005 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:28:40.005 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.005 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.005 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:40.005 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:28:40.005 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:28:40.005 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.005 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.005 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:40.005 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:28:40.005 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:28:40.005 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.005 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.006 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:40.006 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:28:40.006 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:28:40.006 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.006 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.006 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:40.006 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:28:40.006 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:28:40.006 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.006 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.006 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:40.006 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:28:40.006 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:28:40.006 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.007 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.007 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:40.007 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:28:40.007 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:28:40.007 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.007 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.007 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:40.007 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:28:40.007 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:28:40.007 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.007 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.007 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:40.007 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:28:40.007 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:28:40.008 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.008 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:40.008 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.008 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.008 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.008 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:40.008 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:40.008 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 102.363508029, 2023-12-06T18:28:40.008 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:28:40.008 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:40.008 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:40.008 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:40.008 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:40.008 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:40.008 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:28:40.008 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.009 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:40.009 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.009 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:40.009 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:40.009 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.009 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.009 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.009 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:40.009 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:40.009 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.009 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.009 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.009 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:40.009 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:28:40.009 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:28:40.009 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.010 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.010 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:40.010 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:28:40.010 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:28:40.010 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.010 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.010 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:40.010 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:28:40.010 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:28:40.010 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.010 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.010 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:40.010 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:28:40.010 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:28:40.010 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.011 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.011 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:40.011 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:28:40.011 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:28:40.011 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.011 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.011 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:40.011 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:28:40.011 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:28:40.011 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.011 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.011 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:40.011 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:28:40.011 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:28:40.012 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.012 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.012 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:40.012 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:28:40.012 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:28:40.012 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.012 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.012 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:40.012 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:28:40.012 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:28:40.012 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.012 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.012 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:40.012 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:28:40.012 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:28:40.013 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.013 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.013 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:40.013 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:28:40.013 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:28:40.013 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.013 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:40.013 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.013 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.013 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.013 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:40.013 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:40.013 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 101.284703941, 2023-12-06T18:28:40.013 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:28:40.013 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:40.014 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:40.014 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:40.014 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:40.014 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:40.014 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:28:40.014 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.014 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:40.014 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.014 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:40.014 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:40.014 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.014 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.014 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.014 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:40.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:40.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:40.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:28:40.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:28:40.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:40.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:28:40.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:28:40.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:40.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:28:40.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:28:40.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:40.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:28:40.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:28:40.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:40.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:28:40.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:28:40.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:40.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:28:40.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:28:40.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:40.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:28:40.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:28:40.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:40.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:28:40.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:28:40.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:40.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:28:40.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:28:40.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:40.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:28:40.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:28:40.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:40.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:28:40.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:28:40.019 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.019 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:40.019 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.019 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:40.019 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:40.019 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 101.283287367, 2023-12-06T18:28:40.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:28:40.019 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:40.019 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:40.019 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:40.019 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:40.019 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:40.019 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:28:40.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.020 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:40.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:40.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:40.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:40.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:40.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:40.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:28:40.021 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:28:40.021 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.021 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.021 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:40.021 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:28:40.021 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:28:40.021 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.021 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.021 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:40.021 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:28:40.021 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:28:40.021 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.021 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.021 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:40.021 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:28:40.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:28:40.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:40.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:28:40.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:28:40.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:40.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:28:40.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:28:40.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:40.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:28:40.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:28:40.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:40.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:28:40.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:28:40.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:40.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:28:40.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:28:40.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:40.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:28:40.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:28:40.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:40.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:28:40.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:28:40.024 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.024 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:40.024 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.024 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:40.024 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:40.024 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 101.282202242, 2023-12-06T18:28:40.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:28:40.025 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:40.025 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:40.025 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:40.025 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:40.025 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:40.025 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:28:40.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.025 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:40.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:40.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:40.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.026 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:40.026 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:40.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.026 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.026 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.026 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:40.026 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:28:40.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:28:40.026 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.026 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.026 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:40.026 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:28:40.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:28:40.026 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.026 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.026 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:40.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:28:40.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:28:40.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.027 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:40.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:28:40.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:28:40.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.027 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:40.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:28:40.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:28:40.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:40.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:28:40.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:28:40.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:40.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:28:40.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:28:40.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:40.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:28:40.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:28:40.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:40.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:28:40.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:28:40.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:40.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:28:40.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:28:40.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:40.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:28:40.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:28:40.029 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.029 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:40.029 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.030 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:40.030 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:40.030 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 101.24705314000001, 2023-12-06T18:28:40.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:28:40.030 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:40.030 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:40.030 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:40.030 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:40.030 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:40.030 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:28:40.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.030 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:40.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:40.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:40.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:40.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:40.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:40.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:28:40.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:28:40.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:40.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:28:40.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:28:40.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:40.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:28:40.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:28:40.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:40.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:28:40.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:28:40.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:40.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:28:40.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:28:40.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:40.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:28:40.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:28:40.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:40.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:28:40.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:28:40.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:40.034 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:28:40.034 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:28:40.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.034 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:40.034 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:28:40.034 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:28:40.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.034 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:40.034 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:28:40.034 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:28:40.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:40.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:28:40.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:28:40.035 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.035 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:40.035 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.035 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:40.035 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:40.035 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 94.697322615000004, 2023-12-06T18:28:40.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:28:40.035 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:40.035 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:40.035 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:40.036 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:40.036 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:40.036 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:28:40.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.036 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:40.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:40.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:40.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:40.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:40.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.037 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:40.037 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:28:40.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:28:40.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.037 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:40.037 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:28:40.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:28:40.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.037 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:40.037 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:28:40.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:28:40.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:40.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:28:40.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:28:40.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:40.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:28:40.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:28:40.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:40.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:28:40.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:28:40.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:40.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:28:40.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:28:40.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:40.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:28:40.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:28:40.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:40.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:28:40.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:28:40.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:40.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:28:40.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:28:40.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:40.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:28:40.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:28:40.040 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.040 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:40.040 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.041 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:40.041 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:40.041 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 94.696517006999997, 2023-12-06T18:28:40.041 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:28:40.041 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:40.041 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:40.041 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:40.041 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:40.041 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:40.041 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:28:40.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.041 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:40.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.041 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:40.042 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:40.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.042 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.042 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.042 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:40.042 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:40.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.042 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.042 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.042 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:40.042 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:28:40.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:28:40.042 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.042 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:40.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:28:40.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:28:40.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:40.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:28:40.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:28:40.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:40.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:28:40.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:28:40.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:40.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:28:40.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:28:40.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:40.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:28:40.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:28:40.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:40.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:28:40.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:28:40.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:40.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:28:40.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:28:40.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:40.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:28:40.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:28:40.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:40.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:28:40.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:28:40.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.046 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:40.046 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:28:40.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:28:40.046 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.046 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:40.046 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.046 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:40.046 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:40.046 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 94.695826198000006, 2023-12-06T18:28:40.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:28:40.046 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:40.047 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:40.047 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:40.047 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:40.047 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:40.047 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:28:40.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.047 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:40.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:40.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:40.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:40.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:40.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:40.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:28:40.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:28:40.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:40.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:28:40.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:28:40.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:40.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:28:40.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:28:40.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:40.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:28:40.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:28:40.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:40.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:28:40.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:28:40.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:40.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:28:40.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:28:40.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:40.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:28:40.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:28:40.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:40.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:28:40.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:28:40.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:40.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:28:40.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:28:40.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:40.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:28:40.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:28:40.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:40.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:28:40.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:28:40.051 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.051 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:40.051 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 94.691528689999998, 2023-12-06T18:28:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:28:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:28:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:40.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:40.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:40.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:40.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:28:40.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:28:40.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:40.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:28:40.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:28:40.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.054 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:40.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:28:40.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:28:40.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.054 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:40.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:28:40.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:28:40.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.054 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:40.055 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:28:40.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:28:40.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.055 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:40.055 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:28:40.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:28:40.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.055 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:40.055 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:28:40.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:28:40.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.055 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:40.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:28:40.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:28:40.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:40.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:28:40.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:28:40.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:40.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:28:40.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:28:40.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:40.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:28:40.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:28:40.057 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.057 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:40.057 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.057 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:40.057 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:40.057 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 94.690861014999996, 2023-12-06T18:28:40.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:28:40.057 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:40.057 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:40.057 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:40.057 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:40.058 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:40.058 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:28:40.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.058 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:40.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:40.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:40.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:40.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:40.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:40.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:28:40.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:28:40.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:40.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:28:40.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:28:40.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:40.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:28:40.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:28:40.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:40.060 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:28:40.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:28:40.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:40.060 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:28:40.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:28:40.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:40.060 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:28:40.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:28:40.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:40.061 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:28:40.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:28:40.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.061 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:40.061 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:28:40.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:28:40.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.061 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:40.061 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:28:40.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:28:40.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.061 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:40.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:28:40.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:28:40.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:40.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:28:40.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:28:40.062 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.062 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:40.062 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.062 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:40.062 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:40.063 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 94.690205586999994, 2023-12-06T18:28:40.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:28:40.063 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:40.063 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:40.063 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:40.063 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:40.063 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:40.063 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:28:40.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.063 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:40.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:40.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:40.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:40.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:40.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:40.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:28:40.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:28:40.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:40.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:28:40.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:28:40.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:40.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:28:40.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:28:40.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:40.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:28:40.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:28:40.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:40.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:28:40.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:28:40.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:40.066 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:28:40.066 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:28:40.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.066 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.066 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:40.066 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:28:40.066 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:28:40.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.066 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.066 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:40.066 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:28:40.066 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:28:40.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.066 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.067 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:40.067 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:28:40.067 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:28:40.067 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.067 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.067 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:40.067 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:28:40.067 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:28:40.067 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.067 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.067 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:40.067 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:28:40.067 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:28:40.067 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.067 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:40.068 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.068 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:40.068 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:40.068 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 94.649551040000006, 2023-12-06T18:28:40.068 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:28:40.068 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:40.068 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:40.068 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:40.068 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:40.068 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:40.068 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:28:40.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.069 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:40.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:40.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:40.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:40.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:40.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:40.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:28:40.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:28:40.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.070 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:40.070 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:28:40.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:28:40.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.070 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:40.070 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:28:40.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:28:40.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.070 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:40.070 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:28:40.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:28:40.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.071 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:40.071 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:28:40.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:28:40.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.071 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:40.071 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:28:40.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:28:40.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.071 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:40.071 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:28:40.072 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:28:40.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.072 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.072 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:40.072 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:28:40.072 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:28:40.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.072 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.072 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:40.072 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:28:40.072 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:28:40.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.072 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.072 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:40.072 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:28:40.072 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:28:40.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:40.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:28:40.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:28:40.073 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.073 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:40.073 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.073 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.073 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:40.073 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:40.074 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:39.960+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:40.074 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:28:40.116 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:40.112+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:40.178 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:40.174+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:40.252 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:40.253 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:40.253 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:40.253 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:40.262 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:28:40.279 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:40.275+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:40.380 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:40.376+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:40.402 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:40.402 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:40.402 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:40.402 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:40.402 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:40.411 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:28:40.480 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:40.477+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:40.581 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:40.578+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:40.590 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:40.590 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:40.590 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:40.590 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:40.590 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.591 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:28:40.591 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:40.591 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 78.774591224000005, 2023-12-06T18:28:40.591 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:28:40.591 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:40.591 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:28:40.591 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:40.591 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.591 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:40.591 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:40.591 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.591 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.591 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.591 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:40.591 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:40.592 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:40.592 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.592 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.592 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:40.592 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:28:40.592 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:28:40.592 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.592 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.592 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:40.592 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:28:40.592 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:28:40.592 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.592 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.592 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:40.592 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:28:40.592 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:28:40.593 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.593 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.593 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:40.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:28:40.593 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:28:40.593 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.593 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.593 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:40.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:28:40.593 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:28:40.593 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.593 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.593 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:40.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:28:40.593 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:28:40.594 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:40.594 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:40.594 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:40.594 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:28:40.594 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:28:40.594 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.594 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:40.594 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.594 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:40.594 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:40.594 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:40.599 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:28:40.683 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:40.679+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:40.779 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:40.780 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:40.780 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:40.780 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:40.783 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:40.780+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:40.788 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:28:40.885 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:40.881+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:40.966 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:40.966 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:40.966 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:40.967 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:40.967 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:40.975 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:28:40.986 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:40.982+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:41.087 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:41.083+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:41.114 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:41.114 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:41.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:41.114 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:41.114 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:41.123 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:28:41.187 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:41.183+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:41.264 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:41.264 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:41.264 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:41.264 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:41.276 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:28:41.289 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:41.284+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:41.389 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:41.386+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:41.407 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:41.407 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:41.407 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:41.407 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:41.407 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:41.416 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:28:41.491 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:41.487+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:41.547 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:41.547 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:41.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:41.547 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:28:41.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:41.548 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:28:41.548 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:41.548 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.054010863999999, 2023-12-06T18:28:41.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:28:41.548 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:41.548 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:28:41.548 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:28:41.548 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:28:41.548 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:28:41.548 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:28:41.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:41.548 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:41.548 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:41.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:41.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:41.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:41.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:41.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:41.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:41.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:41.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:41.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:41.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:41.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:41.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:28:41.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:28:41.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:41.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:41.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:41.550 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:28:41.550 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:28:41.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:41.550 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:41.550 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:41.550 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:28:41.550 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:28:41.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:41.550 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:41.550 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:41.550 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:28:41.550 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:28:41.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:41.551 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:41.551 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:41.551 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:28:41.551 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:28:41.551 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:41.551 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:41.551 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:41.551 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:28:41.551 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:28:41.551 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:41.551 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:41.551 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:41.551 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:41.551 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:41.551 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:41.557 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:28:41.599 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:41.594+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:41.693 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:41.689+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:41.725 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:41.725 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:41.725 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:41.725 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:41.735 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:28:41.794 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:41.790+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:41.863 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:41.863 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:41.863 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:41.863 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:41.863 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:41.874 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:28:41.919 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:41.915+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:41.996 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:41.992+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:42.042 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:42.043 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:42.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:42.043 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:28:42.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.043 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:28:42.043 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:42.043 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 105.427587368, 2023-12-06T18:28:42.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:28:42.043 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.043 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.043 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:42.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:42.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:28:42.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:28:42.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:28:42.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:28:42.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:28:42.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:28:42.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:28:42.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:28:42.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:28:42.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:28:42.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:28:42.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:28:42.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:28:42.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:28:42.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:28:42.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:28:42.047 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.047 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.047 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.047 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:28:42.047 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:42.047 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 104.40476035899999, 2023-12-06T18:28:42.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:28:42.047 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.047 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.048 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:42.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:42.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:28:42.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:28:42.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:28:42.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:28:42.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:28:42.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:28:42.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:28:42.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:28:42.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:28:42.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:28:42.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:28:42.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:28:42.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:28:42.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:28:42.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:28:42.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:28:42.051 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.051 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.051 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.051 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:28:42.051 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:42.051 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 104.39271372100001, 2023-12-06T18:28:42.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:28:42.052 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.052 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.052 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:42.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:42.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:28:42.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:28:42.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:28:42.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:28:42.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:28:42.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:28:42.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:28:42.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:28:42.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:28:42.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:28:42.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:28:42.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:28:42.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:28:42.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:28:42.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:28:42.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:28:42.055 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.055 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.055 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.055 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:28:42.055 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:42.055 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 104.371313114, 2023-12-06T18:28:42.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:28:42.056 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.056 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.056 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:42.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:42.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:28:42.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:28:42.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:28:42.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:28:42.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:28:42.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:28:42.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:28:42.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:28:42.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:28:42.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:28:42.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:28:42.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:28:42.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:28:42.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:28:42.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:28:42.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:28:42.059 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.059 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.059 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.059 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:28:42.059 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:42.059 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 104.370133791, 2023-12-06T18:28:42.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:28:42.059 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.059 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.060 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:42.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:42.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:28:42.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:28:42.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:28:42.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:28:42.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:28:42.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:28:42.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:28:42.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:28:42.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:28:42.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:28:42.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:28:42.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:28:42.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:28:42.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:28:42.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:28:42.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:28:42.063 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.063 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.063 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.063 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:28:42.063 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:42.063 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 104.36956180999999, 2023-12-06T18:28:42.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:28:42.063 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.064 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.064 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:42.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:42.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:28:42.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:28:42.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:28:42.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:28:42.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:28:42.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:28:42.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:28:42.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:28:42.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:28:42.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:28:42.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:28:42.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:28:42.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:28:42.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:28:42.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:28:42.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:28:42.067 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.067 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.067 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.067 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:28:42.067 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:42.067 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 103.92372634, 2023-12-06T18:28:42.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:28:42.068 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.068 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.068 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:42.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:42.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:28:42.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:28:42.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:28:42.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:28:42.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:28:42.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:28:42.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:28:42.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:28:42.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:28:42.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:28:42.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:28:42.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:28:42.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:28:42.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:28:42.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:28:42.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:28:42.073 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.073 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.073 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.073 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.073 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.073 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:28:42.073 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:42.073 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 103.922126484, 2023-12-06T18:28:42.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:28:42.073 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.073 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.073 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.074 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.074 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:42.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.074 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.074 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:42.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.074 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.074 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:28:42.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:28:42.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:28:42.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:28:42.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:28:42.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:28:42.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:28:42.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:28:42.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:28:42.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:28:42.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:28:42.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:28:42.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:28:42.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:28:42.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:28:42.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:28:42.077 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.077 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.077 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.077 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:28:42.077 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:42.077 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 103.542920868, 2023-12-06T18:28:42.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:28:42.077 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.077 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.077 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.078 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:42.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.078 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.078 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.078 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:42.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.078 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.078 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.078 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:28:42.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:28:42.078 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.078 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.079 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:28:42.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:28:42.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.079 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.079 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:28:42.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:28:42.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.079 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.079 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:28:42.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:28:42.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.079 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.079 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:28:42.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:28:42.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:28:42.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:28:42.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:28:42.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:28:42.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:28:42.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:28:42.081 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.081 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.081 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.081 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:28:42.081 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:42.081 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 103.305418686, 2023-12-06T18:28:42.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:28:42.081 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.081 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.081 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:42.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:42.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:28:42.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:28:42.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:28:42.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:28:42.083 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.083 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.083 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:28:42.083 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:28:42.083 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.083 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.083 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:28:42.083 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:28:42.083 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.083 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.083 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:28:42.083 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:28:42.083 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.084 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.084 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.084 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:28:42.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:28:42.084 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.084 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.084 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.084 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:28:42.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:28:42.084 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.084 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.084 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.084 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:28:42.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:28:42.084 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.084 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.085 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.085 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.085 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:28:42.085 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:42.085 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 103.304690687, 2023-12-06T18:28:42.085 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:28:42.085 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.085 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.085 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.085 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.085 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:42.085 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.085 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:42.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:28:42.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:28:42.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:28:42.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:28:42.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.087 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.087 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:28:42.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:28:42.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.087 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.087 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:28:42.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:28:42.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.087 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.087 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:28:42.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:28:42.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.087 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:28:42.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:28:42.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.088 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.088 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:28:42.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:28:42.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.088 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.088 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:28:42.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:28:42.088 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.088 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.088 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.088 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.089 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:28:42.089 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:42.089 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 103.299678277, 2023-12-06T18:28:42.089 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:28:42.089 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.089 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.089 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.089 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.089 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.089 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:42.089 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.089 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.089 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.089 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.089 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:42.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.090 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.090 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.090 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.090 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:28:42.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:28:42.090 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.090 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.090 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.090 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:28:42.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:28:42.090 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.090 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.090 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.090 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:28:42.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:28:42.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.091 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.091 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:28:42.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:28:42.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.091 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.091 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:28:42.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:28:42.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.091 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.091 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:28:42.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:28:42.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.092 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:28:42.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:28:42.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.092 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:28:42.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:28:42.092 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.092 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.092 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.092 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:28:42.092 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:42.092 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 103.29415119799999, 2023-12-06T18:28:42.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:28:42.093 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.093 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.093 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.093 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.093 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:42.093 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.093 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.093 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:42.093 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.093 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.093 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:28:42.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:28:42.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:28:42.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:28:42.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:28:42.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:28:42.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:28:42.095 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:28:42.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.095 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:28:42.095 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:28:42.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.095 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:28:42.095 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:28:42.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:28:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:28:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:28:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:28:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:28:42.096 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 103.293640361, 2023-12-06T18:28:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:28:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:28:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:28:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:28:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:28:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:28:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:28:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.098 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:28:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:28:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:28:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:28:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:28:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:28:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.099 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:28:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:28:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:28:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:28:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:28:42.100 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 103.2931811, 2023-12-06T18:28:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:28:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.101 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:28:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:28:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:28:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:28:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:28:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:28:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.102 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:28:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:28:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:28:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:28:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:28:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:28:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.103 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:28:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:28:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:28:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:28:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:28:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 96.713265293000006, 2023-12-06T18:28:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:28:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.104 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:28:42.105 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:28:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:28:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:28:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:28:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:28:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:28:42.106 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:28:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:28:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:28:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:28:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:28:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:28:42.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:28:42.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:28:42.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:28:42.108 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.108 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.108 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.108 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:28:42.108 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:42.108 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 96.707027487999994, 2023-12-06T18:28:42.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:28:42.108 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.108 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.109 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:42.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:42.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:28:42.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:28:42.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.110 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.110 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:28:42.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:28:42.110 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.110 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.110 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.110 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:28:42.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:28:42.110 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.110 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.110 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.110 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:28:42.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:28:42.111 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.111 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.111 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.111 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:28:42.111 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:28:42.111 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.111 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.111 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.111 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:28:42.111 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:28:42.111 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.111 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.111 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.111 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:28:42.111 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:28:42.111 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.112 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.112 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:28:42.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:28:42.112 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.112 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.112 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.112 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.112 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:28:42.112 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:42.112 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 96.700964408999994, 2023-12-06T18:28:42.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:28:42.112 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.112 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.112 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:42.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:42.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:28:42.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:28:42.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.114 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:28:42.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:28:42.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.114 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:28:42.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:28:42.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.114 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:28:42.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:28:42.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.115 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.115 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.115 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:28:42.115 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:28:42.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.115 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.115 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.115 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:28:42.115 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:28:42.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.115 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.115 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.115 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:28:42.115 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:28:42.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.116 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.116 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.116 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:28:42.116 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:28:42.116 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.116 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.116 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.116 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.116 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.116 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:28:42.116 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:42.116 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 96.563577323999993, 2023-12-06T18:28:42.116 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:28:42.116 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.117 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.117 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.117 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.117 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:42.117 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.117 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.117 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.117 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:42.117 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.117 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.117 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.117 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:28:42.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:28:42.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:28:42.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:28:42.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:28:42.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:28:42.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:28:42.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:28:42.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:28:42.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:28:42.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:28:42.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:28:42.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:28:42.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:28:42.120 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.120 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.120 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:28:42.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:28:42.120 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.120 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.120 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.120 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.120 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:28:42.120 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:42.120 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 96.556906596999994, 2023-12-06T18:28:42.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:28:42.120 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:42.121 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:42.121 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:42.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:42.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:42.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:42.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:42.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:42.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:42.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:28:42.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:28:42.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.122 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:42.122 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:28:42.122 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:28:42.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.122 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:42.122 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:28:42.122 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:28:42.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.122 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:42.122 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:28:42.122 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:28:42.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:42.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:28:42.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:28:42.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:42.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:28:42.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:28:42.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:42.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:28:42.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:28:42.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:42.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:42.124 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:42.124 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:28:42.124 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:28:42.124 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.124 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.124 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.124 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:42.124 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:42.124 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:42.125 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:28:42.126 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:42.122+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:42.227 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:42.223+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:42.296 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:42.296 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:42.296 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:42.296 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:42.306 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:28:42.350 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:42.347+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:42.429 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:42.426+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:42.479 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:42.479 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:42.479 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:42.479 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:42.479 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:42.490 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:28:42.533 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:42.528+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:42.632 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:42.628+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:42.658 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:42.658 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:42.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:42.658 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:28:42.658 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:42.666 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:28:42.732 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:42.729+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:42.814 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:42.814 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:42.814 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:42.814 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:42.824 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:28:42.868 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:42.864+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:42.934 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:42.930+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:42.960 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:42.960 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:42.960 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:42.960 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:42.961 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:42.969 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:28:43.035 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:43.031+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:43.136 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:43.133+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:43.146 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:43.146 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:43.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:43.146 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:43.146 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:43.155 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:28:43.237 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:43.234+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:43.291 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:43.292 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:43.292 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:43.292 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:43.302 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:28:43.345 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:43.340+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:43.439 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:43.435+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:43.480 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:43.480 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:43.480 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:43.480 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:43.480 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:43.489 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:28:43.540 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:43.536+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:43.641 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:43.637+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:43.689 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:43.689 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:43.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:43.689 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:43.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.689 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:28:43.689 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:43.689 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 108.276053672, 2023-12-06T18:28:43.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:28:43.689 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.689 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.689 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.690 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.690 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.690 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:28:43.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.690 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:43.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:43.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:28:43.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:28:43.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:28:43.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:28:43.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:28:43.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:28:43.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:28:43.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:28:43.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:28:43.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:28:43.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:28:43.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:28:43.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:28:43.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:28:43.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:28:43.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:28:43.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:28:43.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:28:43.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:28:43.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:28:43.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:28:43.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:28:43.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:28:43.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:28:43.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:28:43.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:28:43.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:28:43.695 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.695 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.695 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.695 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:43.695 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:43.695 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 106.16748043299999, 2023-12-06T18:28:43.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:28:43.695 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.695 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.696 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.696 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.696 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.696 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:28:43.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.696 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:43.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:43.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:28:43.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:28:43.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:28:43.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:28:43.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:28:43.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:28:43.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:28:43.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:28:43.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:28:43.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:28:43.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:28:43.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:28:43.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:28:43.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:28:43.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:28:43.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:28:43.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:28:43.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:28:43.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:28:43.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:28:43.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:28:43.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:28:43.700 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.700 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.700 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.700 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:43.700 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:43.701 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 106.16673520099999, 2023-12-06T18:28:43.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:28:43.701 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.701 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.701 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.701 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.701 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.701 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:28:43.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.701 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:43.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:43.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:28:43.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:28:43.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:28:43.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:28:43.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:28:43.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:28:43.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:28:43.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:28:43.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:28:43.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:28:43.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:28:43.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:28:43.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:28:43.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:28:43.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:28:43.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:28:43.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.705 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.705 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:28:43.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:28:43.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.705 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.705 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:28:43.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:28:43.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.705 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.705 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:28:43.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:28:43.706 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.706 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.706 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.706 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:43.706 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:43.706 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 106.166051446, 2023-12-06T18:28:43.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:28:43.706 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.706 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.706 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.706 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.706 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.707 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:28:43.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.707 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:43.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:43.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:28:43.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:28:43.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:28:43.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:28:43.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:28:43.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:28:43.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:28:43.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:28:43.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:28:43.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:28:43.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:28:43.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:28:43.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:28:43.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:28:43.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.710 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:28:43.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:28:43.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.710 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:28:43.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:28:43.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.710 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:28:43.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:28:43.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:28:43.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:28:43.711 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.711 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.711 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.711 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:43.711 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:43.711 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 106.160478191, 2023-12-06T18:28:43.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:28:43.712 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.712 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.712 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.712 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.712 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.712 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:28:43.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.712 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:43.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:28:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:28:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:28:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:28:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:28:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:28:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:28:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:28:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:28:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:28:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:28:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:28:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:28:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:28:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:28:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:28:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:28:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:28:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:28:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:28:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:28:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:28:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 106.15980455099999, 2023-12-06T18:28:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:28:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:28:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:28:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:28:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:28:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:28:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:28:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:28:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:28:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:28:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:28:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:28:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:28:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:28:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:28:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:28:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:28:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:28:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:28:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:28:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:28:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:28:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:28:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:28:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 106.159142352, 2023-12-06T18:28:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:28:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:28:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:28:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:28:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:28:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:28:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:28:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:28:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:28:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:28:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:28:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:28:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:28:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:28:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:28:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:28:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:28:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:28:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:28:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:28:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:28:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:28:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:28:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:28:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 106.086432917, 2023-12-06T18:28:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:28:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:28:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:28:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:28:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:28:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:28:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:28:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:28:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:28:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:28:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:28:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:28:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:28:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:28:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:28:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:28:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:28:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:28:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:28:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:28:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:28:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:28:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:28:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:28:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 106.085751208, 2023-12-06T18:28:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:28:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:28:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:28:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:28:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:28:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:28:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:28:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:28:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:28:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:28:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:28:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:28:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:28:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:28:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:28:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:28:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:28:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:28:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:28:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:28:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:28:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:28:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:28:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:28:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 105.00694712000001, 2023-12-06T18:28:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:28:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:28:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:28:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:28:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:28:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:28:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:28:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:28:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:28:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:28:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:28:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:28:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:28:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:28:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:28:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:28:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:28:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:28:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:28:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:28:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:28:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:28:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:28:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:28:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 105.005530546, 2023-12-06T18:28:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:28:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:28:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:28:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:28:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:28:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:28:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:28:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:28:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:28:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:28:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:28:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:28:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:28:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:28:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:28:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:28:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:28:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:28:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:28:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:28:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:28:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:28:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:28:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:28:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 105.004445421, 2023-12-06T18:28:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:28:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:28:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:28:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:28:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:28:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:28:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:28:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:28:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:28:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:28:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:28:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:28:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:28:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:28:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:28:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:28:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:28:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:28:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:28:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:28:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:28:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:28:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:28:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:28:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 104.96929631899999, 2023-12-06T18:28:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:28:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:28:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:28:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:28:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:28:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:28:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:28:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:28:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:28:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:28:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:28:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:28:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:28:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:28:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:28:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:28:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:28:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:28:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:28:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:28:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:28:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:28:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:28:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:28:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 98.419565793999993, 2023-12-06T18:28:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:28:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:28:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:28:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:28:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:28:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:28:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:28:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:28:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:28:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:28:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:28:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:28:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:28:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:28:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:28:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:28:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:28:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:28:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:28:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:28:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:28:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:28:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:28:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:28:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 98.418760186, 2023-12-06T18:28:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:28:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:28:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:28:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:28:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:28:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:28:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:28:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:28:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:28:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:28:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:28:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:28:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:28:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:28:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:28:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:28:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:28:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:28:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:28:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:28:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:28:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:28:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:28:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:28:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 98.418069376999995, 2023-12-06T18:28:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:28:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:28:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:28:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:28:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:28:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:28:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:28:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:28:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:28:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:28:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:28:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:28:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:28:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:28:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:28:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:28:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:28:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:28:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:28:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:28:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:28:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:28:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:28:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:28:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 98.413771869000001, 2023-12-06T18:28:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:28:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:28:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:28:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:28:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:28:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:28:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:28:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:28:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:28:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:28:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:28:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:28:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:28:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:28:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:28:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:28:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:28:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:28:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:28:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:28:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:28:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:28:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:28:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:28:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 98.413104193999999, 2023-12-06T18:28:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:28:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:28:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:28:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:28:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:28:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:28:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:28:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:28:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:28:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:28:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:28:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:28:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:28:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:28:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:28:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:28:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:28:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:28:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:28:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:28:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:28:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:28:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:28:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:28:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 98.412448765999997, 2023-12-06T18:28:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:28:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:28:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:28:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:28:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:28:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:28:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:28:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:28:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:28:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:28:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:28:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:28:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:28:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:28:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:28:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:28:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:28:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:28:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:28:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:28:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:28:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:28:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:28:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:28:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 98.371794218999995, 2023-12-06T18:28:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:28:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:28:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:28:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:28:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:28:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:28:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:28:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:28:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:28:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:28:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:28:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:28:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:28:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:28:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:28:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:28:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:28:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:28:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:28:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:28:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:28:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:28:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:28:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:28:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:43.798 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:43.799 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:28:43.799 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:43.796+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:43.900 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:43.896+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:43.978 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:43.978 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:43.978 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:43.979 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:43.987 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:28:44.000 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:43.997+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:44.101 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:44.097+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:44.167 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:44.168 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:44.168 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:44.168 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:44.168 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:44.177 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:28:44.222 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:44.218+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:44.303 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:44.299+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:44.317 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:44.317 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:44.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:44.317 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:44.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:44.317 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:28:44.317 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:44.317 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 82.501001310000007, 2023-12-06T18:28:44.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:28:44.317 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:44.317 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:28:44.318 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:44.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:44.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:44.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:44.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:44.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:44.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:44.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:44.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:44.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:44.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:44.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:44.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:44.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:28:44.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:28:44.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:44.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:44.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:44.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:28:44.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:28:44.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:44.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:44.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:44.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:28:44.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:28:44.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:44.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:44.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:44.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:28:44.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:28:44.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:44.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:44.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:44.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:28:44.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:28:44.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:44.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:44.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:44.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:28:44.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:28:44.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:44.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:44.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:44.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:28:44.321 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:28:44.321 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:44.321 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:44.321 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:44.321 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:44.321 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:44.321 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:44.325 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:28:44.411 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:44.407+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:44.505 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:44.501+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:44.508 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:44.508 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:44.508 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:44.508 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:44.517 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:28:44.605 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:44.602+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:44.706 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:44.702+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:44.709 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:44.709 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:44.710 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:44.710 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:44.710 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:44.722 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:28:44.807 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:44.804+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:44.909 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:44.905+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:44.917 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:44.917 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:44.917 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:44.917 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:44.917 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:44.926 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:28:45.010 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:45.006+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:45.065 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:45.066 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:45.066 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:45.066 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:45.075 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:28:45.111 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:45.107+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:45.203 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:45.203 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:45.203 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:45.203 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:45.203 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:45.212 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:28:45.212 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:45.208+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:45.296 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:28:45.297 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:28:45.325 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:45.321+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:45.341 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:45.341 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:45.341 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:45.342 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:28:45.342 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:45.342 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:28:45.342 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:45.342 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 23.848528676000001, 2023-12-06T18:28:45.342 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:28:45.342 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:45.342 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:28:45.342 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:28:45.342 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:28:45.342 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:28:45.342 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:28:45.342 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:45.342 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:45.343 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:45.343 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:45.343 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:45.343 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:45.343 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:45.343 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:45.343 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:45.343 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:45.343 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:45.343 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:45.343 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:45.343 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:45.343 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:28:45.343 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:28:45.343 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:45.343 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:45.344 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:45.344 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:28:45.344 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:28:45.344 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:45.344 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:45.344 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:45.344 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:28:45.344 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:28:45.344 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:45.345 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:45.345 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:45.345 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:28:45.345 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:28:45.345 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:45.345 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:45.345 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:45.345 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:28:45.345 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:28:45.346 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:45.346 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:45.346 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:45.346 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:28:45.346 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:28:45.346 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:45.346 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:45.346 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:45.346 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:45.346 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:45.346 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:45.350 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:28:45.389 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:28:45.389 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:28:45.414 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:45.410+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:45.480 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:45.480 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:45.480 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:45.480 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:45.489 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:28:45.510 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:28:45.510 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:28:45.533 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:45.530+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:45.601 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:28:45.602 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:28:45.615 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:45.612+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:45.617 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:45.617 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:45.617 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:45.617 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:45.618 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:45.626 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:28:45.671 INFO:tasks.ceph.mon.c.smithi061.stderr:2023-12-06T18:28:45.259+0000 ff5f640 -1 mon.c@2(leader).osd e43 definitely_dead 0 2023-12-06T18:28:45.693 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:28:45.693 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:28:45.717 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:45.713+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:45.781 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:28:45.782 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:28:45.817 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:45.814+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:45.822 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:45.822 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:45.834 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:28:45.851 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:28:45.868 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2023-12-06T18:28:45.868 INFO:tasks.ceph.mon.b:Restarting daemon 2023-12-06T18:28:45.868 DEBUG:teuthology.orchestra.run.smithi152:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2023-12-06T18:28:45.874 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:28:45.874 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:28:45.877 INFO:tasks.ceph.mon.b:Started 2023-12-06T18:28:45.877 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2023-12-06T18:28:45.877 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:28:45.878 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:28:45.882 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:28:45.882 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:28:45.924 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:45.920+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:46.009 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:46.009 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:46.018 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:28:46.019 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:46.015+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:46.120 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:46.116+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:46.149 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:46.149 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:46.160 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:28:46.221 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:46.217+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:46.291 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:46.291 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:46.301 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:28:46.346 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:46.342+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:46.422 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:46.419+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:46.440 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:46.440 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:46.449 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:28:46.523 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:46.520+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:46.585 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:03.629 95127== Warning: unimplemented fcntl command: 1036 2023-12-06T18:28:46.587 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:03.631 95127== Warning: unimplemented fcntl command: 1036 2023-12-06T18:28:46.616 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:32.549 98294== Warning: unimplemented fcntl command: 1036 2023-12-06T18:28:46.624 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:46.620+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:46.633 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:46.634 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:46.645 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:28:46.649 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:03.693 95127== Warning: unimplemented fcntl command: 1036 2023-12-06T18:28:46.694 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:32.626 98294== Warning: unimplemented fcntl command: 1036 2023-12-06T18:28:46.725 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:46.721+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:46.733 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is unset 2023-12-06T18:28:46.751 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:28:46.757 INFO:teuthology.orchestra.run.smithi061.stderr:marked in osd.5. 2023-12-06T18:28:46.774 INFO:tasks.thrashosds.thrasher:Added osd 5 2023-12-06T18:28:46.814 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:46.814 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:46.823 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:28:46.865 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:46.861+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:46.925 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:46.922+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:46.927 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:32.859 98294== Warning: unimplemented fcntl command: 1036 2023-12-06T18:28:47.003 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:47.003 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:47.012 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:28:47.054 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:47.050+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:47.127 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:47.124+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:47.200 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:47.200 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:47.209 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:28:47.228 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:47.224+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:47.329 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:47.325+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:47.348 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:47.348 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:47.358 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:28:47.443 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:47.439+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:47.531 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:47.527+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:47.535 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:47.535 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:47.545 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:28:47.632 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:47.629+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:47.733 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:47.730+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:47.776 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:47.777 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:47.786 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:28:47.834 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:47.830+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:47.884 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:28:47.933 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:47.933 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:47.942 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:28:48.005 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:47.995+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:48.044 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:48.040+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:48.105 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:48.105 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:48.114 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:28:48.164 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:28:48.164 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:28:48.173 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:48.169+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:48.248 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:48.244+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:48.263 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:48.263 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:48.273 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:28:48.306 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:28:48.306 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:28:48.350 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:48.347+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:48.408 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:28:48.408 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:28:48.439 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:48.435+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:48.462 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:48.462 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:48.475 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:28:48.531 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:28:48.531 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:28:48.539 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:48.536+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:48.639 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:28:48.639 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:28:48.640 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:48.636+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:48.640 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:48.640 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:48.640 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:48.640 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:48.650 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:28:48.731 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:28:48.732 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:28:48.753 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-12-06T18:28:48.777 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:48.773+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:48.822 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:48.822 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:48.822 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:48.822 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:48.823 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:48.832 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:28:48.864 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:28:48.864 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:28:48.875 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:48.870+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:48.881 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:28:48.885 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:28:48.885 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:28:48.943 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:48.939+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:49.044 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:49.040+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:49.061 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:49.061 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:49.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:49.061 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:28:49.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.062 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:28:49.062 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:49.062 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 27.567935719000001, 2023-12-06T18:28:49.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:28:49.062 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.062 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:28:49.062 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:28:49.062 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:28:49.062 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:28:49.062 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:28:49.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.062 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:49.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:49.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:28:49.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:28:49.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:28:49.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:28:49.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:28:49.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:28:49.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:28:49.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:28:49.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:28:49.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:28:49.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:28:49.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:28:49.065 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.065 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.065 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.065 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.065 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.065 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:49.071 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:28:49.145 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:49.141+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:49.202 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:49.202 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:49.202 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:49.202 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:49.211 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:28:49.253 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:49.249+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:49.349 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:49.344+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:49.384 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:49.384 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:49.384 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:49.384 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:49.384 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:49.395 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:28:49.480 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:49.476+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:49.550 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:49.546+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:49.605 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:49.605 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:49.605 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:49.605 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:28:49.605 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.605 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:28:49.605 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:49.605 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 112.989735751, 2023-12-06T18:28:49.605 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:28:49.605 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.606 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.606 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.606 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.606 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.606 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:49.606 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.606 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.606 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.606 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.606 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:49.606 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.606 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.606 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.606 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.606 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:28:49.607 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:28:49.607 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.607 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.607 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.607 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:28:49.607 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:28:49.607 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.607 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.607 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.607 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:28:49.607 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:28:49.607 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.607 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.607 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.607 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:28:49.608 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:28:49.608 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.608 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.608 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.608 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:28:49.608 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:28:49.608 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.608 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.608 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.608 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:28:49.608 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:28:49.608 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.608 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.608 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.608 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:28:49.608 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:28:49.609 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.609 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.609 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.609 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:28:49.609 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:28:49.609 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.609 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.609 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.609 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.609 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.609 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:28:49.609 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:49.609 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 111.966908742, 2023-12-06T18:28:49.609 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:28:49.609 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.610 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.610 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.610 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.610 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.610 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:49.610 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.610 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.610 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.610 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.610 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:49.610 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.610 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.610 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.610 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.611 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:28:49.611 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:28:49.611 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.611 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.611 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.611 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:28:49.611 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:28:49.611 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.611 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.611 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.611 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:28:49.611 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:28:49.611 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.611 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.611 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.612 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:28:49.612 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:28:49.612 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.612 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.612 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.612 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:28:49.612 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:28:49.612 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.612 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.612 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.612 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:28:49.612 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:28:49.612 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.612 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.612 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.613 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:28:49.613 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:28:49.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.613 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.613 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.613 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:28:49.613 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:28:49.613 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.613 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.613 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.613 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.613 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:28:49.613 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:49.613 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 111.954862104, 2023-12-06T18:28:49.614 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:28:49.614 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.614 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.614 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.614 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.614 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.614 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:49.614 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.614 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.614 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.614 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.614 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:49.614 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.614 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.614 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.615 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.615 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:28:49.615 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:28:49.615 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.615 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.615 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.615 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:28:49.615 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:28:49.615 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.615 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.615 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.615 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:28:49.615 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:28:49.615 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.615 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.615 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.616 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:28:49.616 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:28:49.616 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.616 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.616 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.616 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:28:49.616 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:28:49.616 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.616 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.616 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.616 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:28:49.616 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:28:49.616 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.616 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.616 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.616 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:28:49.617 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:28:49.617 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.617 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.617 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.617 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:28:49.617 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:28:49.617 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.617 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.617 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.617 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.617 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.617 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:28:49.617 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:49.617 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 111.933461497, 2023-12-06T18:28:49.617 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:28:49.617 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.618 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.618 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.618 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.618 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.618 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:49.618 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.618 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.618 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.618 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.618 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:49.618 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.618 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.618 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.618 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.618 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:28:49.619 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:28:49.619 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.619 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.619 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.619 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:28:49.619 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:28:49.619 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.619 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.619 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.619 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:28:49.619 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:28:49.619 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.619 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.619 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.620 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:28:49.620 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:28:49.620 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.620 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.620 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.620 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:28:49.620 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:28:49.620 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.620 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.620 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.620 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:28:49.620 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:28:49.620 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.620 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.620 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.620 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:28:49.621 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:28:49.621 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.621 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.621 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.621 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:28:49.621 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:28:49.621 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.621 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.621 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.621 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.621 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.621 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:28:49.621 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:49.621 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 111.93228217399999, 2023-12-06T18:28:49.621 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:28:49.622 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.622 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.622 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.622 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.622 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.622 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:49.622 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.622 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.622 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.622 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:49.622 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.622 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.622 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:28:49.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:28:49.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:28:49.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:28:49.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:28:49.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:28:49.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:28:49.624 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:28:49.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.624 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.624 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.624 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:28:49.624 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:28:49.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.624 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.624 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.624 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:28:49.624 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:28:49.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.624 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.624 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.624 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:28:49.625 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:28:49.625 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.625 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.625 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:28:49.625 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:28:49.625 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.625 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.625 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.625 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.625 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:28:49.625 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:49.625 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 111.931710193, 2023-12-06T18:28:49.625 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:28:49.626 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.626 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.626 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.626 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.626 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:49.626 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.626 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.626 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:49.626 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.626 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.627 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:28:49.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:28:49.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.627 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.627 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:28:49.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:28:49.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.627 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.627 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:28:49.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:28:49.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.627 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:28:49.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:28:49.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:28:49.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:28:49.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:28:49.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:28:49.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.629 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:28:49.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:28:49.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.629 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.629 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:28:49.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:28:49.629 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.629 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.629 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.629 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:28:49.629 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:49.629 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 111.48587472299999, 2023-12-06T18:28:49.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:28:49.630 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.630 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.630 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.630 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:49.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.630 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:49.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:28:49.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:28:49.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:28:49.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:28:49.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:28:49.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:28:49.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.632 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.632 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:28:49.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:28:49.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.632 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.632 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:28:49.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:28:49.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.632 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.632 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:28:49.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:28:49.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.633 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.633 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:28:49.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:28:49.633 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.633 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.633 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:28:49.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:28:49.633 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.633 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.633 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.633 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.633 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:28:49.633 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:49.634 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 111.484274867, 2023-12-06T18:28:49.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:28:49.634 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.634 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.634 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:49.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:49.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:28:49.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:28:49.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:28:49.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:28:49.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:28:49.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:28:49.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:28:49.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:28:49.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:28:49.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:28:49.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:28:49.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:28:49.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.637 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:28:49.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:28:49.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.637 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.637 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.637 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:28:49.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:28:49.637 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.637 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.637 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.637 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.637 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:28:49.637 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:49.637 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 111.105069251, 2023-12-06T18:28:49.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:28:49.638 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.638 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.638 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.638 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:49.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.638 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:49.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:28:49.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:28:49.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:28:49.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:28:49.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:28:49.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:28:49.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:28:49.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:28:49.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:28:49.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:28:49.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:28:49.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:28:49.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:28:49.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:28:49.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:28:49.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:28:49.641 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.641 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.641 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.641 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:28:49.641 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:49.641 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 110.867567069, 2023-12-06T18:28:49.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:28:49.642 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.642 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.642 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:49.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:49.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:28:49.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:28:49.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.643 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:28:49.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:28:49.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.643 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:28:49.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:28:49.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.643 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:28:49.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:28:49.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:28:49.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:28:49.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:28:49.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:28:49.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:28:49.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:28:49.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:28:49.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:28:49.645 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.645 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.645 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.645 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:28:49.645 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:49.645 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 110.86683907, 2023-12-06T18:28:49.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:28:49.646 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.646 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.646 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:49.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:49.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:28:49.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:28:49.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:28:49.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:28:49.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:28:49.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:28:49.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:28:49.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:28:49.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.648 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:28:49.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:28:49.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.648 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:28:49.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:28:49.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.648 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.649 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:28:49.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:28:49.649 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.649 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.649 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.649 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:28:49.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:28:49.649 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.649 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.649 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.649 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.649 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.649 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:28:49.649 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:49.649 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 110.86182666000001, 2023-12-06T18:28:49.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:28:49.650 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.650 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.650 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.650 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:49.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.650 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:49.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:28:49.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:28:49.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:28:49.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:28:49.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:28:49.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:28:49.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:28:49.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:28:49.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:28:49.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:28:49.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:28:49.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:28:49.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:28:49.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:28:49.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:28:49.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:28:49.653 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.653 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.653 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.653 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:28:49.653 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:49.653 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 110.856299581, 2023-12-06T18:28:49.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:28:49.654 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.654 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.654 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.654 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:49.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.654 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:49.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:28:49.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:28:49.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:28:49.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:28:49.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:28:49.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:28:49.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:28:49.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:28:49.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:28:49.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:28:49.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:28:49.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:28:49.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:28:49.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:28:49.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:28:49.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:28:49.657 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.657 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.657 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.657 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:28:49.657 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:49.657 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 110.85578874399999, 2023-12-06T18:28:49.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:28:49.658 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.658 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.658 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:49.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:49.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:28:49.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:28:49.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:28:49.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:28:49.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:28:49.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:28:49.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:28:49.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:28:49.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:28:49.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:28:49.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:28:49.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:28:49.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:28:49.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:28:49.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:28:49.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:28:49.661 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.661 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.661 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.661 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:28:49.661 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:49.662 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 110.85532948300001, 2023-12-06T18:28:49.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:28:49.662 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.662 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.662 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:49.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:49.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:28:49.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:28:49.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:28:49.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:28:49.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:28:49.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:28:49.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:28:49.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:28:49.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:28:49.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:28:49.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:28:49.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:28:49.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.665 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.665 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:28:49.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:28:49.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.665 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.665 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:28:49.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:28:49.665 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.665 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.665 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.666 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:28:49.666 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:49.666 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 104.275413676, 2023-12-06T18:28:49.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:28:49.666 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.666 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.666 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:49.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:49.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:28:49.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:28:49.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:28:49.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:28:49.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:28:49.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:28:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:28:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:28:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:28:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:28:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:28:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:28:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:28:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:28:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:28:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:28:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:28:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 104.269175871, 2023-12-06T18:28:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:28:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:28:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:28:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:28:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:28:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:28:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:28:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:28:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:28:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:28:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:28:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:28:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:28:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:28:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:28:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:28:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:28:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:28:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 104.263112792, 2023-12-06T18:28:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:28:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:28:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:28:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:28:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:28:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:28:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:28:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:28:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:28:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:28:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:28:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:28:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:28:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:28:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:28:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:28:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:28:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:28:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 104.125725707, 2023-12-06T18:28:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:28:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:28:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:28:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:28:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:28:49.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:28:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:28:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:28:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:28:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:28:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:28:49.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:28:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:28:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:28:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:28:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:28:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:28:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.681 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.682 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.682 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:28:49.682 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:49.682 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 104.11905498, 2023-12-06T18:28:49.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:28:49.682 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:49.682 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:49.682 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:49.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:49.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:49.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:49.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:49.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:49.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:49.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:28:49.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:28:49.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:49.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:28:49.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:28:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:28:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:28:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:28:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:28:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:28:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:28:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:28:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:28:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:28:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:28:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:28:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:28:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.685 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.686 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:49.686 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:49.686 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:49.687 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:28:49.688 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:49.684+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:49.769 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:28:49.814 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:49.810+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:49.816 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is set 2023-12-06T18:28:49.857 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:49.857 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:49.857 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:49.857 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:49.866 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:28:49.911 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:49.907+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:49.992 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:49.989+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:50.044 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:50.044 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:50.044 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:50.044 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:50.044 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:50.054 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:28:50.096 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:50.092+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:50.191 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:50.187+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:50.225 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:50.225 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:50.226 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:50.226 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:28:50.226 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:50.235 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:28:50.291 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:50.287+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:50.379 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:50.379 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:50.380 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:50.380 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:50.389 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:28:50.430 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:50.427+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:50.493 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:50.489+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:50.510 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":5,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:28:50.577 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:50.577 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:50.577 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:50.577 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:50.577 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:50.586 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:28:50.593 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:50.590+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:50.695 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:50.691+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:50.742 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:50.742 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:50.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:50.742 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:50.742 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:50.751 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:28:50.795 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:50.791+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:50.886 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:28:50.932 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:50.932 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:50.932 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:50.932 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:50.932 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:50.928+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:50.942 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:28:51.027 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:51.023+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:51.098 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:51.094+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:51.143 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:51.144 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:51.144 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:51.144 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:51.144 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:51.153 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:28:51.193 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:28:51.194 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:28:51.198 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:51.194+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:51.299 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:51.295+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:51.313 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:28:51.314 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:28:51.363 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:51.363 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:51.372 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:28:51.399 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:51.396+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:51.407 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:28:51.407 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:28:51.500 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:51.497+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:51.538 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:28:51.538 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:28:51.580 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:51.580 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:51.589 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:28:51.627 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:28:51.627 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:28:51.627 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:51.624+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:51.702 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:51.699+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:51.715 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:28:51.716 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:28:51.726 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:51.726 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:51.736 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:28:51.775 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [7] live_osds: [0, 1, 2, 3, 4, 5, 6] 2023-12-06T18:28:51.775 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:28:51.776 INFO:tasks.thrashosds.thrasher:primary_affinity 2023-12-06T18:28:51.776 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.634578 2023-12-06T18:28:51.777 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0.6345782980233435 2023-12-06T18:28:51.804 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:28:51.804 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:28:51.805 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:51.801+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:51.807 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:28:51.811 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:28:51.812 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:28:51.834 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-12-06T18:28:51.903 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:51.900+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:51.922 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:51.922 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:51.932 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:28:52.005 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:52.001+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:52.071 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:52.072 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:52.082 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:28:52.126 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:52.122+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:52.206 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:52.203+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:52.242 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:52.243 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:52.255 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:28:52.307 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:52.304+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:52.408 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:52.405+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:52.435 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:52.436 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:52.445 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:28:52.510 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:52.506+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:52.575 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:52.575 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:52.586 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:28:52.611 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:52.607+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:52.712 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:52.708+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:52.718 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:52.718 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:52.730 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:28:52.822 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:52.818+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:52.903 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:52.904 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:52.913 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:28:52.915 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:52.911+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:53.016 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:53.012+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:53.043 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:53.043 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:53.053 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:28:53.117 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:53.113+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:53.182 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:53.183 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:53.193 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:28:53.218 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:53.214+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:53.284 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is set 2023-12-06T18:28:53.300 INFO:teuthology.orchestra.run.smithi061.stderr:set osd.2 primary-affinity to 0.634578 (8415872) 2023-12-06T18:28:53.356 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:53.352+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:53.366 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:53.366 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:53.375 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:28:53.420 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:53.416+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:53.505 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:53.505 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:53.514 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:28:53.528 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:28:53.559 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:53.555+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:53.622 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:53.618+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:53.644 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:53.644 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:53.654 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:28:53.722 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:53.719+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:53.813 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:28:53.836 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:53.836 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:53.847 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:28:53.859 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:53.854+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:53.923 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:53.920+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:54.024 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:54.021+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:54.030 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:54.030 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:54.043 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:28:54.101 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:28:54.101 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:28:54.104 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:08.166 98186== Warning: unimplemented fcntl command: 1036 2023-12-06T18:28:54.125 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:54.122+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:54.197 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:28:54.197 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:28:54.214 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":9,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:28:54.223 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:54.223 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:54.223 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:54.223 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:54.223 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:54.226 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:54.223+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:54.233 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:28:54.293 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:28:54.293 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:28:54.328 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:54.324+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:54.343 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:08.405 98186== Warning: unimplemented fcntl command: 1036 2023-12-06T18:28:54.409 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:28:54.409 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:28:54.429 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:54.425+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:54.442 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:54.442 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:54.442 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:54.442 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:54.442 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:54.452 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:28:54.497 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:28:54.497 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:28:54.530 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:54.526+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:54.584 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:28:54.584 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:28:54.635 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:54.632+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:54.643 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:54.644 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:54.644 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:54.644 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:54.653 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:28:54.685 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:28:54.685 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:28:54.689 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:28:54.693 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:28:54.693 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:28:54.737 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:54.733+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:54.822 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:54.822 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:54.822 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:54.822 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:54.823 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:54.832 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:28:54.838 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:54.834+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:54.938 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:54.935+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:55.070 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:55.070 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:55.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:55.071 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:55.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.071 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:28:55.071 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:55.071 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 119.65985332, 2023-12-06T18:28:55.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:28:55.071 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.071 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.071 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.071 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.071 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.071 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:28:55.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.071 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.072 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.072 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:55.072 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.072 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.072 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.072 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:28:55.072 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.072 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.072 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.072 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:28:55.072 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:28:55.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.072 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:28:55.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:28:55.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:28:55.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:28:55.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:28:55.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:28:55.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.074 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:28:55.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:28:55.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:28:55.074 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:28:55.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:28:55.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.074 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:28:55.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:28:55.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:28:55.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:28:55.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.075 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:28:55.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:28:55.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.075 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:28:55.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:28:55.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.075 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:28:55.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:28:55.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.076 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:28:55.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:28:55.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.076 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:28:55.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:28:55.076 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.076 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.076 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.077 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.077 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.077 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:55.077 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:55.077 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 117.551280081, 2023-12-06T18:28:55.077 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:28:55.077 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.077 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.077 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.077 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.077 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.077 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:28:55.077 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.077 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.078 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.078 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:55.078 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.078 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.078 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:28:55.078 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.078 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.078 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:28:55.078 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:28:55.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:28:55.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:28:55.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:28:55.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:28:55.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:28:55.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:28:55.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:28:55.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:28:55.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:28:55.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:28:55.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:28:55.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:28:55.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:28:55.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:28:55.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.081 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:28:55.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:28:55.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.081 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:28:55.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:28:55.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.082 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.082 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:28:55.082 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:28:55.082 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.082 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.082 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.082 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.082 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.082 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:55.082 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:55.082 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 117.550534849, 2023-12-06T18:28:55.082 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:28:55.082 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.083 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.083 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.083 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.083 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.083 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:28:55.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.083 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.083 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.083 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.083 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:55.083 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.083 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.083 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.083 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:28:55.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:28:55.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:28:55.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:28:55.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:28:55.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:28:55.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:28:55.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:28:55.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:28:55.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:28:55.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:28:55.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:28:55.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:28:55.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:28:55.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:28:55.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:28:55.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:28:55.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:28:55.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:28:55.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.087 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:28:55.087 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:28:55.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.087 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:28:55.087 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:28:55.087 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.087 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.087 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.087 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:55.087 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:55.088 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 117.549851094, 2023-12-06T18:28:55.088 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:28:55.088 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.088 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.088 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.088 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.088 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.088 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:28:55.088 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.088 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.088 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.088 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.088 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:55.088 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.088 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.088 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:28:55.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:28:55.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:28:55.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:28:55.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:28:55.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:28:55.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:28:55.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:28:55.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:28:55.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:28:55.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:28:55.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:28:55.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:28:55.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:28:55.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:28:55.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:28:55.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:28:55.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:28:55.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:28:55.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:28:55.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:28:55.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:28:55.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:28:55.092 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.092 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.092 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.093 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.093 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:55.093 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:55.093 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 117.544277839, 2023-12-06T18:28:55.093 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:28:55.093 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.093 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.093 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.093 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.093 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.093 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:28:55.093 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.093 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.093 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.093 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.093 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:55.094 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.094 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.094 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.094 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.094 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:28:55.094 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.094 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.094 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.094 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.094 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:28:55.094 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:28:55.094 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.094 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.094 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.094 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:28:55.094 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:28:55.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.095 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.095 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.095 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:28:55.095 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:28:55.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.095 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.095 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.095 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:28:55.095 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:28:55.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.095 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.095 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.095 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:28:55.095 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:28:55.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.096 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.096 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:28:55.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:28:55.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.096 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.096 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:28:55.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:28:55.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.096 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.096 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:28:55.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:28:55.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.097 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.097 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:28:55.097 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:28:55.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.097 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.097 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:28:55.097 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:28:55.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.097 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.097 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:28:55.097 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:28:55.097 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.097 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.098 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.098 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.098 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.098 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:55.098 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:55.098 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 117.543604199, 2023-12-06T18:28:55.098 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:28:55.098 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.098 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.098 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.098 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.098 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.098 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:28:55.098 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.098 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.099 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.099 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.099 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:55.099 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.099 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.099 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.099 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:28:55.099 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.099 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.099 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.099 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:28:55.099 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:28:55.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.099 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.100 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:28:55.100 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:28:55.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.100 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:28:55.100 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:28:55.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.100 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:28:55.100 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:28:55.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:28:55.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:28:55.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:28:55.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:28:55.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:28:55.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:28:55.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:28:55.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:28:55.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:28:55.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:28:55.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:28:55.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:28:55.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:28:55.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:28:55.103 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.103 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.103 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.103 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.103 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:55.103 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:55.103 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 117.542942, 2023-12-06T18:28:55.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:28:55.103 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.103 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.103 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.103 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.103 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.103 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:28:55.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.104 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.104 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.104 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:55.104 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.104 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.104 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:28:55.104 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.104 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.104 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:28:55.104 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:28:55.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.105 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.105 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:28:55.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:28:55.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.105 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.105 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:28:55.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:28:55.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.105 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.105 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:28:55.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:28:55.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.106 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.106 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:28:55.106 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:28:55.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.106 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.106 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:28:55.106 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:28:55.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.107 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:28:55.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:28:55.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.107 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:28:55.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:28:55.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.107 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:28:55.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:28:55.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.108 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.108 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:28:55.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:28:55.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.108 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.108 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:28:55.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:28:55.108 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.108 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.108 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.109 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:55.109 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:55.109 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 117.470232565, 2023-12-06T18:28:55.109 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:28:55.109 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.109 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.109 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.109 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.109 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.109 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:28:55.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.109 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.109 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.109 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.110 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:55.110 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.110 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.110 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.110 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.110 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:28:55.110 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.110 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.110 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.110 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.110 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:28:55.110 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:28:55.110 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.110 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:28:55.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:28:55.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:28:55.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:28:55.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:28:55.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:28:55.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:28:55.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:28:55.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:28:55.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:28:55.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:28:55.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:28:55.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:28:55.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:28:55.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:28:55.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:28:55.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:28:55.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:28:55.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:28:55.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:28:55.114 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.114 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.114 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.114 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:28:55.114 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:55.114 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 117.469550856, 2023-12-06T18:28:55.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:28:55.114 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.114 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.114 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.114 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.114 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.115 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:28:55.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.115 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:55.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:28:55.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:28:55.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:28:55.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:28:55.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:28:55.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:28:55.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:28:55.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:28:55.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:28:55.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:28:55.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:28:55.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:28:55.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:28:55.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:28:55.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:28:55.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:28:55.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:28:55.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:28:55.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:28:55.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:28:55.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:28:55.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:28:55.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:28:55.119 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.119 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.119 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.119 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:55.119 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:55.119 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 116.390746768, 2023-12-06T18:28:55.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:28:55.119 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.119 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.119 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.120 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.120 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.120 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:28:55.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.120 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:55.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:28:55.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:28:55.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:28:55.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:28:55.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:28:55.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:28:55.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:28:55.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:28:55.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:28:55.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:28:55.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:28:55.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:28:55.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:28:55.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:28:55.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:28:55.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:28:55.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:28:55.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:28:55.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:28:55.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.124 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.124 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:28:55.124 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:28:55.124 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.124 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.124 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.124 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:28:55.124 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:28:55.124 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.124 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.124 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.124 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.124 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.124 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:55.124 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:55.124 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 116.389330194, 2023-12-06T18:28:55.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:28:55.125 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.125 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.125 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.125 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.125 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.125 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:28:55.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.125 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.125 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.125 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:55.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.125 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.126 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:28:55.126 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.126 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.126 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.126 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.126 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:28:55.126 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:28:55.126 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.126 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.126 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.126 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:28:55.126 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:28:55.126 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.126 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.126 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.127 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:28:55.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:28:55.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.127 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.127 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:28:55.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:28:55.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.127 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.127 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:28:55.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:28:55.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.127 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.127 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:28:55.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:28:55.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.128 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:28:55.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:28:55.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.128 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:28:55.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:28:55.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.128 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:28:55.129 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:28:55.129 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.129 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.129 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.129 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:28:55.129 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:28:55.129 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.129 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.129 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.129 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:28:55.129 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:28:55.129 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.129 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.129 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.130 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.130 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:55.130 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:55.130 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 116.38824506900001, 2023-12-06T18:28:55.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:28:55.130 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.130 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.130 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.130 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.130 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.130 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:28:55.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.130 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.130 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:55.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:28:55.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:28:55.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:28:55.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:28:55.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:28:55.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:28:55.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:28:55.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:28:55.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:28:55.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:28:55.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:28:55.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:28:55.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:28:55.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:28:55.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:28:55.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:28:55.134 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:28:55.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:28:55.134 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:28:55.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:28:55.134 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:28:55.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:28:55.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:28:55.135 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.135 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.135 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.135 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:28:55.135 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:55.135 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 116.353095967, 2023-12-06T18:28:55.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:28:55.135 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.135 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.135 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.135 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.135 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.135 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:28:55.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.136 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:55.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:28:55.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:28:55.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:28:55.137 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.137 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:28:55.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:28:55.137 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.137 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:28:55.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:28:55.137 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.137 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:28:55.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:28:55.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.138 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:28:55.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:28:55.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.138 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:28:55.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:28:55.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.138 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:28:55.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:28:55.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:28:55.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:28:55.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:28:55.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:28:55.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:28:55.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:28:55.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:28:55.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:28:55.140 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.140 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.140 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.140 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:55.140 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:55.140 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 109.803365442, 2023-12-06T18:28:55.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:28:55.140 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.140 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.140 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.140 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.141 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.141 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:28:55.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.141 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:55.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:28:55.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:28:55.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:28:55.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:28:55.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:28:55.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:28:55.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:28:55.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.143 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:28:55.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:28:55.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.143 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:28:55.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:28:55.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.143 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:28:55.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:28:55.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:28:55.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:28:55.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:28:55.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:28:55.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:28:55.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:28:55.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:28:55.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:28:55.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:28:55.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:28:55.145 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.145 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.145 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.145 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:55.145 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:55.145 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 109.80255983399999, 2023-12-06T18:28:55.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:28:55.146 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.146 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.146 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.146 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.146 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.146 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:28:55.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.146 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:55.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:28:55.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:28:55.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:28:55.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:28:55.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:28:55.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:28:55.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:28:55.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:28:55.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:28:55.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:28:55.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:28:55.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:28:55.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:28:55.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:28:55.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:28:55.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:28:55.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:28:55.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:28:55.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:28:55.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:28:55.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:28:55.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:28:55.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:28:55.150 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.150 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.150 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.151 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:55.151 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:55.151 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 109.801869025, 2023-12-06T18:28:55.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:28:55.151 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.151 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.151 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.151 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.151 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.151 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:28:55.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.151 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:55.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:28:55.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:28:55.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:28:55.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.153 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.153 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:28:55.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:28:55.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.153 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.153 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:28:55.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:28:55.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.153 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.153 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:28:55.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:28:55.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:28:55.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:28:55.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:28:55.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:28:55.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:28:55.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:28:55.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:28:55.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:28:55.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:28:55.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:28:55.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:28:55.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:28:55.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:28:55.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:28:55.156 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.156 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.156 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.156 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:55.156 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:55.156 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 109.79757151699999, 2023-12-06T18:28:55.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:28:55.156 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.156 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.156 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.156 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.156 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.157 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:28:55.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.157 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:55.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:28:55.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:28:55.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:28:55.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.158 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:28:55.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:28:55.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.158 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:28:55.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:28:55.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.158 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:28:55.159 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:28:55.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.159 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.159 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:28:55.159 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:28:55.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.159 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.159 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:28:55.159 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:28:55.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.159 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.159 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:28:55.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:28:55.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:28:55.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:28:55.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:28:55.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:28:55.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:28:55.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:28:55.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:28:55.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:28:55.161 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.161 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.161 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.161 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:55.161 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:55.161 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 109.79690384200001, 2023-12-06T18:28:55.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:28:55.161 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.161 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.162 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.162 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.162 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.162 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:28:55.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.162 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.162 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.162 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:55.162 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.162 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.162 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:28:55.162 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.163 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.163 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.163 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.163 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:28:55.163 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:28:55.163 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.163 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.163 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.163 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:28:55.163 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:28:55.163 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.163 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.163 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.163 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:28:55.163 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:28:55.163 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.163 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.164 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.164 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:28:55.164 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:28:55.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.164 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.164 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.164 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:28:55.164 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:28:55.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.164 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.164 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.164 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:28:55.164 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:28:55.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.164 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:28:55.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:28:55.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:28:55.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:28:55.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:28:55.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:28:55.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.166 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:28:55.166 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:28:55.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.166 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.166 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.166 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:28:55.166 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:28:55.166 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.166 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.166 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.166 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.166 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:55.166 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:55.166 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 109.796248414, 2023-12-06T18:28:55.166 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:28:55.167 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.167 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.167 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.167 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.167 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.167 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:28:55.167 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.167 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.167 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.167 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.167 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:55.167 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.167 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.167 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.167 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.167 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:28:55.168 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.168 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.168 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.168 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.168 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:28:55.168 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:28:55.168 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.168 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.168 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.168 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:28:55.168 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:28:55.168 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.168 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.168 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.168 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:28:55.168 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:28:55.169 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.169 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.169 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.169 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:28:55.169 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:28:55.169 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.169 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.169 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.169 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:28:55.169 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:28:55.169 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.169 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.169 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.169 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:28:55.169 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:28:55.169 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.170 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.170 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.170 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:28:55.170 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:28:55.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.170 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.170 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.170 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:28:55.170 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:28:55.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.170 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.170 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.170 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:28:55.170 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:28:55.171 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.171 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.171 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.171 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:28:55.171 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:28:55.171 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.171 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.171 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.171 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:28:55.171 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:28:55.171 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.171 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.171 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.171 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.171 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.171 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:28:55.172 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:55.172 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 109.755593867, 2023-12-06T18:28:55.172 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:28:55.172 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.172 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:55.172 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:28:55.172 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:28:55.172 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:28:55.172 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:28:55.172 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.172 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.172 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.172 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.172 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:55.172 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.173 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.173 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.173 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.173 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:28:55.173 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.173 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.173 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.173 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.173 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:28:55.173 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:28:55.173 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.173 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.173 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.174 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:28:55.174 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:28:55.174 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.174 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.174 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.174 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:28:55.174 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:28:55.174 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.178 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.178 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.178 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:28:55.178 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:28:55.178 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.178 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.178 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.178 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:28:55.178 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:28:55.178 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.178 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.179 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.179 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:28:55.179 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:28:55.179 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.179 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.179 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:28:55.179 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:28:55.179 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:28:55.179 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.179 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.179 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:28:55.179 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:28:55.179 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:28:55.179 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.179 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.180 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:28:55.180 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:28:55.180 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:28:55.180 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.180 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.180 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:28:55.180 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:28:55.180 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:28:55.180 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.180 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.180 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.180 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:28:55.180 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:28:55.180 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.180 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.181 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.181 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.181 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.181 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:55.182 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:55.068+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:55.183 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:28:55.183 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:55.179+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:55.283 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:55.280+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:55.300 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-12-06T18:28:55.361 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:55.361 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:55.361 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:55.361 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:55.370 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:28:55.384 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:55.380+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:55.485 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:55.481+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:55.530 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:55.531 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:55.531 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:55.531 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:55.531 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:55.540 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:28:55.585 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:55.582+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:55.687 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:55.683+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:55.721 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:55.721 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:55.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:55.721 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:28:55.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.721 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:28:55.721 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:55.721 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 93.904860421999999, 2023-12-06T18:28:55.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:28:55.722 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:28:55.722 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:28:55.722 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:28:55.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:28:55.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:55.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:28:55.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:28:55.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:28:55.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:28:55.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:28:55.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:28:55.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:28:55.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:28:55.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:28:55.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:28:55.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:28:55.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:28:55.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:28:55.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:28:55.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:28:55.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:28:55.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:28:55.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:28:55.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:28:55.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:28:55.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:28:55.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:28:55.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:28:55.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:28:55.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:28:55.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:28:55.725 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.725 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.725 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.725 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:28:55.726 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:28:55.726 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:55.731 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:28:55.788 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:55.784+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:55.877 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:55.877 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:55.878 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:28:55.878 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:55.886 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:28:55.888 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:55.885+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:55.989 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:55.985+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:56.064 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:56.065 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:28:56.065 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:28:56.065 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:28:56.065 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:56.073 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:28:56.118 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:56.114+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:56.190 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:56.187+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:56.211 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:28:56.212 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:28:56.212 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:28:56.212 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:28:56.212 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:28:56.220 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:28:56.292 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:56.288+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:56.370 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:56.371 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:56.371 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:56.371 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:56.380 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:28:56.424 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:56.420+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:56.493 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:56.490+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:56.510 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:56.510 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:56.510 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:56.510 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:56.511 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:56.520 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:28:56.594 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:56.591+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:56.693 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:56.693 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:56.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:56.693 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:28:56.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:56.693 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:28:56.693 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:56.693 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 35.199421745000002, 2023-12-06T18:28:56.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:28:56.693 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:56.693 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:28:56.693 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:28:56.694 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:28:56.694 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:28:56.694 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:28:56.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:56.694 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:56.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:56.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:56.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:56.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:56.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:56.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:56.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:56.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:28:56.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:56.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:56.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:56.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:56.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:28:56.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:28:56.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:56.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:56.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:56.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:28:56.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:28:56.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:56.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:56.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:56.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:28:56.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:28:56.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:56.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:56.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:56.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:28:56.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:28:56.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:56.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:56.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:56.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:28:56.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:28:56.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:56.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:56.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:56.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:28:56.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:28:56.697 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:56.697 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:56.697 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:56.698 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:56.698 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:56.698 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:56.698 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:28:56.699 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:56.696+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:56.702 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:28:56.800 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:56.797+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:56.874 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:56.874 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:56.875 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:56.875 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:56.884 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:28:56.901 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:56.897+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:57.008 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is unset 2023-12-06T18:28:57.010 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:57.006+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:57.025 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:57.025 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:57.025 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:28:57.025 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:28:57.025 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:57.031 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:28:57.036 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:28:57.036 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:28:57.116 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:57.113+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:57.129 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:28:57.129 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:28:57.229 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:28:57.233 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:28:57.234 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:28:57.240 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:57.240 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:28:57.240 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:28:57.240 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:28:57.240 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.240 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:28:57.240 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:57.240 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 120.624737467, 2023-12-06T18:28:57.240 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:28:57.240 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.240 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.240 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.241 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.241 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.241 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:57.241 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.241 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.241 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.241 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:28:57.241 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.241 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.241 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.242 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:28:57.242 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:28:57.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.242 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.242 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.242 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:28:57.242 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:28:57.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.242 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.242 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.242 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:28:57.242 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:28:57.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.242 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.242 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.243 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:28:57.243 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:28:57.243 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.243 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.243 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.243 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:28:57.243 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:28:57.243 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.243 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.243 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.243 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:28:57.243 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:28:57.243 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.243 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.243 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.243 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:28:57.243 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:28:57.244 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.244 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.244 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.244 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:28:57.244 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:28:57.244 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.244 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.244 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.244 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.244 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.244 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:28:57.244 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:57.244 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 119.60191045800001, 2023-12-06T18:28:57.244 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:28:57.245 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.245 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.245 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.245 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.245 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.245 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:57.245 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.245 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.245 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.245 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:28:57.245 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.245 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.245 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.246 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:28:57.246 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:28:57.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.246 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.246 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.246 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:28:57.246 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:28:57.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.246 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.246 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.246 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:28:57.246 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:28:57.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.246 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.246 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.247 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:28:57.247 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:28:57.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.247 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.247 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.247 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:28:57.247 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:28:57.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.247 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.247 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.247 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:28:57.247 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:28:57.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.247 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.247 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.248 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:28:57.248 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:28:57.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.248 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.248 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.248 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:28:57.248 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:28:57.248 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.248 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.248 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.248 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.248 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:28:57.248 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:57.248 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 119.58986382000001, 2023-12-06T18:28:57.249 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:28:57.249 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.249 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.249 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.249 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.249 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.249 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:57.249 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.249 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.249 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.249 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.249 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:28:57.249 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.249 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.249 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.250 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.250 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:28:57.250 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:28:57.250 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.250 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.250 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.250 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:28:57.250 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:28:57.250 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.250 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.250 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.250 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:28:57.250 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:28:57.250 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.251 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.251 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.251 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:28:57.251 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:28:57.251 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.251 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.251 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.251 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:28:57.251 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:28:57.251 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.251 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.251 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.251 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:28:57.251 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:28:57.251 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.252 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.252 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.252 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:28:57.252 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:28:57.252 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.252 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.252 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.252 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:28:57.252 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:28:57.252 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.252 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.252 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.252 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.252 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.252 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:28:57.252 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:57.252 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 119.568463213, 2023-12-06T18:28:57.253 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:28:57.253 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.253 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.253 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.253 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.253 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.253 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:57.253 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.253 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.253 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.253 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.253 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:28:57.253 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.253 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.253 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.253 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.253 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:28:57.254 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:28:57.254 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.254 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.254 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.254 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:28:57.254 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:28:57.254 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.254 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.254 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.254 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:28:57.254 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:28:57.254 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.254 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.254 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.255 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:28:57.255 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:28:57.255 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.255 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.255 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.255 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:28:57.255 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:28:57.255 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.255 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.255 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.255 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:28:57.255 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:28:57.255 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.255 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.255 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.256 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:28:57.256 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:28:57.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.256 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.256 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.256 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:28:57.256 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:28:57.256 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.256 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.256 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.256 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.256 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:28:57.257 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:57.257 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 119.56728389, 2023-12-06T18:28:57.257 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:28:57.257 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.257 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.257 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.257 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.257 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.257 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:57.257 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.257 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.257 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.257 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.257 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:28:57.257 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.257 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.257 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.258 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.258 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:28:57.258 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:28:57.258 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.258 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.258 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.258 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:28:57.258 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:28:57.258 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.258 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.258 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.258 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:28:57.258 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:28:57.258 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.259 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.259 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.259 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:28:57.259 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:28:57.259 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.259 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.259 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.259 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:28:57.259 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:28:57.259 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.259 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.259 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.259 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:28:57.259 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:28:57.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.260 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.260 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.260 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:28:57.260 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:28:57.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.260 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.260 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.260 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:28:57.260 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:28:57.260 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.260 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.260 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.261 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.261 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:28:57.261 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:57.261 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 119.56671190900001, 2023-12-06T18:28:57.261 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:28:57.261 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.261 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.261 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.261 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.261 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.261 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:57.261 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.261 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.261 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.262 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.262 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:28:57.262 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.262 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.262 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.262 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.262 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:28:57.262 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:28:57.262 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.262 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.262 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.262 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:28:57.262 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:28:57.263 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.263 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.263 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.263 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:28:57.263 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:28:57.263 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.263 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.263 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.263 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:28:57.263 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:28:57.263 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.263 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.263 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.263 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:28:57.263 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:28:57.263 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.264 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.264 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.264 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:28:57.264 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:28:57.264 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.264 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.264 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.264 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:28:57.264 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:28:57.264 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.264 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.264 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.264 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:28:57.264 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:28:57.264 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.264 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.265 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.265 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.265 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.265 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:28:57.265 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:57.265 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 119.120876439, 2023-12-06T18:28:57.265 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:28:57.265 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.265 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.265 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.265 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.265 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.265 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:57.265 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.265 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.266 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.266 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.266 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:28:57.266 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.266 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.266 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.266 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.266 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:28:57.266 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:28:57.266 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.266 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.267 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.267 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:28:57.267 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:28:57.267 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.267 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.267 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.267 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:28:57.267 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:28:57.267 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.267 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.267 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.267 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:28:57.267 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:28:57.267 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.268 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.268 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.268 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:28:57.268 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:28:57.268 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.268 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.268 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.268 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:28:57.268 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:28:57.268 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.268 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.268 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.268 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:28:57.268 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:28:57.268 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.268 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.269 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.269 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:28:57.269 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:28:57.269 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.269 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.269 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.269 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.269 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.269 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:28:57.269 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:57.269 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 119.119276583, 2023-12-06T18:28:57.269 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:28:57.270 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.270 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.270 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.270 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.270 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.270 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:57.270 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.270 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.270 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.270 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.270 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:28:57.270 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.270 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.270 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.270 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.271 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:28:57.271 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:28:57.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.271 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.271 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.271 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:28:57.271 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:28:57.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.271 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.271 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.271 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:28:57.271 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:28:57.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.271 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.271 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.272 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:28:57.272 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:28:57.272 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.272 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.272 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.272 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:28:57.272 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:28:57.272 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.272 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.272 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.272 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:28:57.272 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:28:57.272 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.272 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.272 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.272 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:28:57.272 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:28:57.273 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.273 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.273 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.273 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:28:57.273 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:28:57.273 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.273 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.273 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.273 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.273 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.273 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:28:57.273 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:57.273 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 118.74007096699999, 2023-12-06T18:28:57.273 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:28:57.273 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.273 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.273 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.274 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.274 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.274 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:57.274 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.274 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.274 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.274 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.274 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:28:57.274 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.274 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.274 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.274 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.274 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:28:57.274 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:28:57.274 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.274 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.275 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.275 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:28:57.275 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:28:57.275 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.275 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.275 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.275 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:28:57.275 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:28:57.275 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.275 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.275 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.275 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:28:57.275 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:28:57.275 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.275 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.275 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.275 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:28:57.276 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:28:57.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.276 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.276 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.276 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:28:57.276 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:28:57.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.276 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.276 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.276 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:28:57.276 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:28:57.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.276 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.276 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.277 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:28:57.277 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:28:57.277 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.277 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.277 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.277 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.277 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.277 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:28:57.277 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:57.277 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 118.50256878499999, 2023-12-06T18:28:57.277 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:28:57.277 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.277 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.277 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.277 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.277 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.278 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:57.278 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.278 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.278 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.278 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.278 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:28:57.278 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.278 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.278 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.278 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.278 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:28:57.278 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:28:57.278 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.279 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.279 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:28:57.279 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:28:57.279 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.279 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.279 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:28:57.279 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:28:57.279 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.279 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.279 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:28:57.279 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:28:57.279 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.280 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.280 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:28:57.280 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:28:57.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.280 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.280 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:28:57.280 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:28:57.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.280 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.280 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:28:57.280 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:28:57.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.281 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:28:57.281 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:28:57.281 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.281 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.281 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.281 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:28:57.281 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:57.281 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 118.501840786, 2023-12-06T18:28:57.281 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:28:57.281 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.281 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.281 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:57.282 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.282 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.282 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:28:57.282 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.282 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.282 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:28:57.282 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:28:57.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.283 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.283 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.283 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:28:57.283 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:28:57.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.283 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.283 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.283 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:28:57.283 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:28:57.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.283 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.283 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.283 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:28:57.283 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:28:57.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.284 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.284 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.284 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:28:57.284 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:28:57.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.284 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.284 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.284 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:28:57.284 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:28:57.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.284 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.284 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.284 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:28:57.284 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:28:57.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.285 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.285 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:28:57.285 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:28:57.285 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.285 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.285 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.285 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:28:57.285 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:57.285 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 118.496828376, 2023-12-06T18:28:57.285 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:28:57.285 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.285 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.286 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:57.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:28:57.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:28:57.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:28:57.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:28:57.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:28:57.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:28:57.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:28:57.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:28:57.288 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:28:57.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.288 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.288 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.288 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:28:57.288 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:28:57.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.288 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.288 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.288 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:28:57.288 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:28:57.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.289 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.289 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.289 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:28:57.289 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:28:57.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.289 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.289 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.289 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:28:57.289 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:28:57.289 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.289 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.289 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.289 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.289 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:28:57.290 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:57.290 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 118.49130129700001, 2023-12-06T18:28:57.290 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:28:57.290 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.290 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.290 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.290 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.290 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:57.290 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.290 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:28:57.291 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:28:57.291 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:28:57.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:28:57.291 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:28:57.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.292 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.292 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.292 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:28:57.292 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:28:57.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.292 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.292 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.292 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:28:57.292 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:28:57.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.292 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.292 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.292 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:28:57.293 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:28:57.293 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.293 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.293 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.293 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:28:57.293 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:28:57.293 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.293 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.293 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.293 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:28:57.293 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:28:57.293 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.293 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.294 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.294 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:28:57.294 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:28:57.294 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.294 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.294 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.294 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.294 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.294 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:28:57.294 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:57.294 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 118.49079046, 2023-12-06T18:28:57.294 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:28:57.294 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.294 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.294 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.294 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.295 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.295 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:57.295 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.295 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.295 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.295 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:28:57.295 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.295 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.295 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.295 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:28:57.295 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:28:57.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.295 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.296 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.296 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:28:57.296 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:28:57.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.296 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.296 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.296 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:28:57.296 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:28:57.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.296 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.296 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.296 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:28:57.296 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:28:57.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.296 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.297 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.297 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:28:57.297 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:28:57.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.297 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.297 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.297 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:28:57.297 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:28:57.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.297 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.297 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.297 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:28:57.297 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:28:57.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.297 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.298 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.298 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:28:57.298 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:28:57.298 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.298 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.298 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.298 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.298 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:28:57.298 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:57.298 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 118.490331199, 2023-12-06T18:28:57.298 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:28:57.298 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.298 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.298 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.299 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.299 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:57.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.299 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.299 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:28:57.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.299 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.299 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:28:57.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:28:57.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:28:57.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:28:57.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:28:57.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:28:57.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:28:57.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:28:57.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.301 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.301 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:28:57.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:28:57.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.301 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.301 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:28:57.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:28:57.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.301 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.301 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:28:57.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:28:57.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.301 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.302 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:28:57.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:28:57.302 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.302 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.302 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.302 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:28:57.302 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:57.302 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 111.910415392, 2023-12-06T18:28:57.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:28:57.302 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.302 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.302 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:57.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:28:57.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:28:57.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:28:57.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:28:57.304 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:28:57.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.304 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:28:57.304 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:28:57.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.304 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:28:57.304 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:28:57.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.304 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.305 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:28:57.305 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:28:57.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.305 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.305 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.305 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:28:57.305 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:28:57.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.305 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.305 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.305 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:28:57.305 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:28:57.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.305 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.306 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.306 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:28:57.306 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:28:57.306 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.306 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.306 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.306 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.306 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:28:57.306 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:57.306 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 111.90417758700001, 2023-12-06T18:28:57.306 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:28:57.307 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.307 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.307 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.307 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.307 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:57.307 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.307 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.307 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:28:57.307 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.308 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.308 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:28:57.308 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:28:57.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.308 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.308 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:28:57.308 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:28:57.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.308 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.308 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:28:57.308 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:28:57.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.309 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.309 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:28:57.309 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:28:57.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.309 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.309 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.309 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:28:57.309 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:28:57.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.309 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.309 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.309 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:28:57.309 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:28:57.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.309 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.309 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.310 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:28:57.310 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:28:57.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.310 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.310 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.310 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:28:57.310 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:28:57.310 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.310 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.310 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.310 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.310 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:28:57.311 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:57.311 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 111.89811450800001, 2023-12-06T18:28:57.311 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:28:57.311 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.311 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.311 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.311 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.311 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:57.311 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.311 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.311 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:28:57.311 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.312 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.312 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:28:57.312 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:28:57.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.312 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.312 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.312 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:28:57.312 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:28:57.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.312 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.312 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.312 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:28:57.312 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:28:57.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.312 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.312 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.312 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:28:57.313 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:28:57.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.313 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.313 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:28:57.313 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:28:57.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.313 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.313 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:28:57.313 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:28:57.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.313 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.314 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:28:57.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:28:57.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.314 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.314 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:28:57.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:28:57.314 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.314 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.314 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.314 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:28:57.314 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:57.314 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 111.76072742300001, 2023-12-06T18:28:57.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:28:57.315 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.315 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.315 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:57.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:28:57.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:28:57.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:28:57.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.316 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:28:57.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:28:57.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.316 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:28:57.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:28:57.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.316 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:28:57.317 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:28:57.317 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.317 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.317 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.317 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:28:57.317 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:28:57.317 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.317 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.317 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.317 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:28:57.317 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:28:57.317 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.317 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.317 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.317 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:28:57.317 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:28:57.318 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.318 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.318 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.318 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:28:57.318 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:28:57.318 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.318 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.318 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.318 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.318 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.318 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:28:57.318 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:57.318 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 111.75405669600001, 2023-12-06T18:28:57.318 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:28:57.319 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:28:57.319 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:28:57.319 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:28:57.319 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.319 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:28:57.319 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:57.319 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.319 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.319 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.319 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:28:57.319 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:28:57.319 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:28:57.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.320 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.320 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:28:57.320 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:28:57.320 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:28:57.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.320 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.320 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:28:57.320 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:28:57.320 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:28:57.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.320 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.320 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:28:57.321 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:28:57.321 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:28:57.321 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.321 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.321 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:28:57.321 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:28:57.321 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:28:57.321 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.321 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.321 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:28:57.321 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:28:57.321 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:28:57.321 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.321 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.322 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:28:57.322 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:28:57.322 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:28:57.322 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.322 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.322 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:28:57.322 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:28:57.322 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:28:57.322 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:28:57.322 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:28:57.322 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:28:57.322 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:28:57.322 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:28:57.323 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.323 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.323 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.323 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:28:57.323 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:28:57.323 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:57.323 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:57.236+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:57.324 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:28:57.326 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:57.322+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:57.327 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:28:57.328 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:28:57.427 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:57.423+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:57.427 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:28:57.428 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:28:57.518 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:28:57.519 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:28:57.519 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:28:57.519 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:28:57.527 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:28:57.528 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:28:57.528 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:28:57.529 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:57.524+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:57.621 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:28:57.622 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:28:57.629 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:57.625+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:57.709 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:28:57.715 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:28:57.719 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:28:57.719 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:28:57.761 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:57.757+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:57.830 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:57.827+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:57.837 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:57.837 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:57.847 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:28:57.896 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:28:57.931 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:57.927+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:57.989 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:57.989 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:57.998 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:28:58.031 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:58.028+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:58.133 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:58.129+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:58.174 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:58.174 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:58.184 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:28:58.234 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:58.230+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:58.315 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [7] live_osds: [0, 1, 2, 3, 4, 5, 6] 2023-12-06T18:28:58.315 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:28:58.322 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:58.322 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:58.331 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:28:58.334 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:58.331+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:58.435 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:58.431+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:58.467 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:58.467 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:58.476 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:28:58.536 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:28:58.532+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:28:58.615 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:58.615 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:58.626 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:28:58.636 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:58.633+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:58.738 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:58.734+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:58.809 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:58.810 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:58.819 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:28:58.862 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:28:58.857+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:28:58.940 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:58.936+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:58.996 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:58.996 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:59.005 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:28:59.027 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-12-06T18:28:59.047 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:59.043+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:59.141 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:59.138+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:59.188 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:59.188 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:59.199 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:28:59.243 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:59.239+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:59.344 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:28:59.340+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:28:59.352 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:59.352 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:59.361 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:28:59.445 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:59.441+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:28:59.546 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:28:59.542+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:28:59.557 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:59.557 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:59.567 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:28:59.647 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:28:59.643+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:28:59.709 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:59.709 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:59.719 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:28:59.720 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:28:59.748 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:59.744+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:59.849 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:28:59.846+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:28:59.871 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is unset 2023-12-06T18:28:59.937 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:28:59.938 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:28:59.947 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:28:59.950 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:28:59.947+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:00.051 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:00.047+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:00.087 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:00.087 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:00.097 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:29:00.123 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:29:00.124 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:29:00.151 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:00.148+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:00.216 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:29:00.217 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:29:00.226 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:00.227 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:00.237 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:29:00.282 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:00.278+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:00.308 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:29:00.308 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:29:00.353 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:00.350+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:00.365 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:00.365 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:00.375 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:29:00.402 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:29:00.402 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:29:00.454 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:00.450+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:00.495 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:29:00.495 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:29:00.554 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:00.551+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:00.602 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:29:00.602 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:29:00.602 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:00.603 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:00.603 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:00.603 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:00.613 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:29:00.661 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:00.655+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:00.696 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:29:00.696 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:29:00.712 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:29:00.715 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:29:00.716 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:29:00.743 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:00.743 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:00.743 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:00.744 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:00.744 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:00.752 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:29:00.758 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:00.754+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:00.857 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:00.853+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:00.911 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:29:00.923 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:00.923 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:00.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:00.923 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:00.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.923 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:29:00.923 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:00.923 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 124.30353235299999, 2023-12-06T18:29:00.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:29:00.923 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:00.923 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:00.924 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:00.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:00.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:00.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:00.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:00.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:00.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:29:00.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:29:00.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:00.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:29:00.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:29:00.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:00.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:29:00.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:29:00.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:00.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:29:00.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:29:00.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:00.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:29:00.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:29:00.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:00.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:29:00.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:29:00.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:00.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:29:00.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:29:00.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:00.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:29:00.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:29:00.927 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.927 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:00.927 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.927 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:29:00.927 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:00.927 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 123.280705344, 2023-12-06T18:29:00.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:29:00.928 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:00.928 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:00.928 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:00.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:00.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:00.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:00.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:00.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:00.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:29:00.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:29:00.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:00.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:29:00.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:29:00.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:00.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:29:00.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:29:00.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:00.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:29:00.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:29:00.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:00.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:29:00.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:29:00.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:00.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:29:00.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:29:00.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:00.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:29:00.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:29:00.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:00.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:29:00.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:29:00.931 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.931 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:00.931 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.931 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:29:00.932 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:00.932 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 123.268658706, 2023-12-06T18:29:00.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:29:00.932 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:00.932 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:00.932 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:00.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:00.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:00.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:00.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:00.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:00.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:29:00.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:29:00.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.935 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:00.935 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:29:00.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:29:00.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.935 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:00.935 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:29:00.936 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:29:00.936 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.936 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.936 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:00.936 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:29:00.936 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:29:00.936 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:00.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:29:00.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:29:00.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:00.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:29:00.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:29:00.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:00.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:29:00.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:29:00.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.939 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:00.939 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:29:00.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:29:00.939 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.939 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:00.939 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.940 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:29:00.940 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:00.940 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 123.24725809900001, 2023-12-06T18:29:00.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:29:00.940 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:00.941 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:00.941 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:00.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:00.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:00.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:00.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:00.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:00.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:29:00.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:29:00.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:00.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:29:00.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:29:00.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:00.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:29:00.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:29:00.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.943 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:00.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:29:00.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:29:00.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.943 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:00.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:29:00.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:29:00.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.943 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:00.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:29:00.944 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:29:00.944 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.944 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.944 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:00.944 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:29:00.944 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:29:00.944 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.944 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.944 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:00.944 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:29:00.944 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:29:00.944 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.944 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:00.944 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.944 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.944 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.945 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:29:00.945 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:00.945 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 123.246078776, 2023-12-06T18:29:00.945 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:29:00.945 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:00.945 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:00.945 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:00.945 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.945 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:00.945 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:00.945 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.945 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.945 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.945 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:00.945 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:00.946 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.946 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.946 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.946 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:00.946 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:29:00.946 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:29:00.946 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.946 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.946 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:00.946 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:29:00.946 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:29:00.946 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.946 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.946 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:00.947 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:29:00.947 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:29:00.947 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.947 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.947 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:00.947 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:29:00.947 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:29:00.947 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.947 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.947 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:00.947 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:29:00.947 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:29:00.947 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.947 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.947 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:00.948 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:29:00.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:29:00.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.948 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:00.948 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:29:00.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:29:00.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.948 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:00.948 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:29:00.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:29:00.948 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.948 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:00.949 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.949 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.949 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.949 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:29:00.949 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:00.949 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 123.245506795, 2023-12-06T18:29:00.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:29:00.949 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:00.949 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:00.949 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:00.949 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.949 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:00.949 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:00.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.949 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.949 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.950 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:00.950 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:00.950 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.950 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.950 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:00.950 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:29:00.950 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:29:00.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.950 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.950 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:00.950 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:29:00.950 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:29:00.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.951 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:00.951 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:29:00.951 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:29:00.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.951 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:00.951 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:29:00.951 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:29:00.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.951 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:00.951 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:29:00.951 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:29:00.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.952 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:00.952 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:29:00.952 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:29:00.952 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.952 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.952 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:00.952 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:29:00.952 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:29:00.952 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.952 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.952 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:00.952 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:29:00.952 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:29:00.952 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.952 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:00.953 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.953 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.953 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.953 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:29:00.953 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:00.953 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 122.79967132500001, 2023-12-06T18:29:00.953 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:29:00.953 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:00.953 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:00.953 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:00.953 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.953 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:00.953 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:00.953 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.953 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.953 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.954 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:00.954 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:00.954 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.954 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.954 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.954 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:00.954 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:29:00.954 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:29:00.954 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.954 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.954 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:00.954 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:29:00.954 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:29:00.955 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.955 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.955 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:00.955 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:29:00.955 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:29:00.955 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.955 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.955 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:00.955 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:29:00.955 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:29:00.955 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.955 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.955 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:00.955 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:29:00.955 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:29:00.955 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.956 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.956 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:00.956 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:29:00.956 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:29:00.956 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.956 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.956 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:00.956 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:29:00.956 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:29:00.956 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.956 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.956 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:00.956 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:29:00.956 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:29:00.956 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.957 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:00.957 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.957 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.957 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.957 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:29:00.957 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:00.957 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 122.79807146900001, 2023-12-06T18:29:00.957 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:29:00.957 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:00.957 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:00.957 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:00.957 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.957 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:00.957 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:00.957 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.958 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.958 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.958 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:00.958 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:00.958 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.958 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.958 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.958 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:00.958 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:29:00.958 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:29:00.958 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.958 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.958 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:00.958 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:29:00.959 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:29:00.959 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.959 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.959 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:00.959 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:29:00.959 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:29:00.959 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.959 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.959 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:00.959 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:29:00.959 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:29:00.959 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.959 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.959 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:00.959 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:29:00.959 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:29:00.959 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.960 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.960 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:00.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:29:00.960 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:29:00.960 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.960 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.960 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:00.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:29:00.960 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:29:00.960 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.960 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.960 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:00.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:29:00.960 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:29:00.960 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.961 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:00.961 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.961 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.961 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.961 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:29:00.961 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:00.961 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 122.418865853, 2023-12-06T18:29:00.961 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:29:00.961 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:00.961 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:00.962 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:00.962 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.962 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:00.962 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:00.962 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.962 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.962 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.962 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:00.962 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:00.962 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.962 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.962 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.962 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:00.963 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:29:00.963 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:29:00.963 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.963 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.963 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:00.963 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:29:00.963 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:29:00.963 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.963 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.963 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:00.963 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:29:00.963 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:29:00.963 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.963 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.963 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:00.964 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:29:00.964 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:29:00.964 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.964 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.964 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:00.964 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:29:00.964 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:29:00.964 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.964 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.964 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:00.964 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:29:00.965 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:29:00.965 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.965 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.965 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:00.965 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:29:00.965 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:29:00.965 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.965 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.966 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:00.966 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:29:00.966 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:29:00.966 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.966 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:00.966 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.966 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.966 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.966 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:29:00.967 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:00.967 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 122.181363671, 2023-12-06T18:29:00.967 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:29:00.967 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:00.967 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:00.967 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:00.967 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.967 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:00.967 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:00.967 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.968 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.968 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.968 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:00.968 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:00.968 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.968 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.968 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.968 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:00.968 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:29:00.969 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:29:00.969 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.969 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.969 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:00.969 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:29:00.969 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:29:00.969 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.969 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.970 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:00.970 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:29:00.970 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:29:00.970 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.970 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.970 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:00.970 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:29:00.970 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:29:00.970 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.971 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:00.971 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:29:00.971 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:29:00.971 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.971 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:00.971 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:29:00.971 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:29:00.971 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.971 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:00.971 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:29:00.971 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:29:00.972 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.972 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.972 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:00.972 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:29:00.972 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:29:00.972 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.972 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:00.972 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.972 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.973 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.973 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:29:00.973 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:00.973 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 122.18063567199999, 2023-12-06T18:29:00.973 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:29:00.973 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:00.973 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:00.973 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:00.973 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.974 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:00.974 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:00.974 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.974 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.974 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.974 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:00.974 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:00.974 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.974 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.975 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:00.975 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:29:00.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:29:00.975 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.975 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:00.975 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:29:00.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:29:00.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.976 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:00.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:29:00.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:29:00.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.976 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:00.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:29:00.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:29:00.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.977 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:00.977 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:29:00.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:29:00.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.977 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:00.978 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:29:00.978 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:29:00.978 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.978 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:00.978 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:29:00.978 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:29:00.978 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.979 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:00.979 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:29:00.979 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:29:00.979 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.979 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:00.979 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.979 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.979 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.979 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:29:00.980 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:00.980 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 122.175623262, 2023-12-06T18:29:00.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:29:00.980 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:00.980 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:00.980 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:00.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.980 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:00.980 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:00.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:00.981 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:00.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:00.981 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:29:00.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:29:00.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:00.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:29:00.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:29:00.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:00.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:29:00.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:29:00.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:00.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:29:00.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:29:00.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:00.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:29:00.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:29:00.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:00.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:29:00.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:29:00.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:00.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:29:00.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:29:00.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:00.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:29:00.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:29:00.984 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.984 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:00.984 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.985 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:29:00.985 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:00.985 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 122.170096183, 2023-12-06T18:29:00.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:29:00.985 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:00.985 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:00.985 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:00.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:00.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:00.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:00.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:00.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:00.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:29:00.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:29:00.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:00.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:29:00.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:29:00.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:00.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:29:00.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:29:00.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:00.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:29:00.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:29:00.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:00.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:29:00.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:29:00.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:00.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:29:00.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:29:00.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:00.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:29:00.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:29:00.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:00.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:29:00.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:29:00.988 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.989 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:00.989 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.989 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:29:00.989 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:00.989 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 122.16958534600001, 2023-12-06T18:29:00.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:29:00.989 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:00.989 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:00.989 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:00.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:00.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:00.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:00.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:00.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:00.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:29:00.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:29:00.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:00.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:29:00.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:29:00.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:00.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:29:00.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:29:00.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:00.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:29:00.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:29:00.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:00.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:29:00.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:29:00.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:00.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:29:00.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:29:00.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:00.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:29:00.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:29:00.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:00.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:29:00.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:29:00.993 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.993 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:00.993 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.993 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:29:00.993 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:00.993 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 122.169126085, 2023-12-06T18:29:00.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:29:00.993 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:00.993 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:00.993 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:00.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:00.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:00.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:00.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:00.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:00.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:29:00.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:29:00.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:00.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:29:00.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:29:00.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:00.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:29:00.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:29:00.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:00.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:29:00.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:29:00.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:00.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:29:00.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:29:00.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:00.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:29:00.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:29:00.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:00.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:29:00.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:29:00.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:00.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:29:00.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:29:00.997 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.997 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:00.997 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:00.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.998 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:29:00.998 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:00.998 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 115.589210278, 2023-12-06T18:29:00.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:29:00.998 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:00.998 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:00.998 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:00.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:00.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:00.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:00.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:00.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:00.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:00.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:29:00.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:29:00.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:00.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:00.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:00.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:29:00.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:29:00.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:01.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:29:01.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:29:01.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:01.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:29:01.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:29:01.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:01.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:29:01.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:29:01.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:01.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:29:01.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:29:01.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:01.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:29:01.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:29:01.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:01.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:29:01.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:29:01.001 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:01.001 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:01.002 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:01.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.002 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:29:01.002 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:01.002 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 115.582972473, 2023-12-06T18:29:01.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:29:01.002 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:01.002 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:01.002 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:01.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:01.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:01.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:01.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:01.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:01.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:01.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:01.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:29:01.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:29:01.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:01.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:29:01.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:29:01.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:01.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:29:01.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:29:01.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:01.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:29:01.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:29:01.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:01.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:29:01.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:29:01.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:01.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:29:01.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:29:01.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:01.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:29:01.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:29:01.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:01.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:29:01.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:29:01.006 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:01.006 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:01.006 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:01.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.006 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:29:01.007 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:01.007 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 115.576909394, 2023-12-06T18:29:01.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:29:01.007 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:01.007 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:01.007 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:01.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:01.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:01.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:01.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:01.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:01.008 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:01.008 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.008 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.008 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:01.008 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:29:01.008 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:29:01.008 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.008 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.008 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:01.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:29:01.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:29:01.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:01.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:29:01.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:29:01.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:01.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:29:01.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:29:01.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:01.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:29:01.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:29:01.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:01.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:29:01.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:29:01.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:01.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:29:01.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:29:01.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:01.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:29:01.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:29:01.011 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:01.011 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:01.011 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:01.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.011 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:29:01.011 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:01.011 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 115.439522309, 2023-12-06T18:29:01.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:29:01.011 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:01.011 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:01.012 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:01.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:01.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:01.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:01.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:01.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:01.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:01.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.013 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:01.013 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:29:01.013 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:29:01.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.013 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:01.013 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:29:01.013 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:29:01.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.013 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:01.014 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:29:01.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:29:01.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.014 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:01.014 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:29:01.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:29:01.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.014 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:01.014 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:29:01.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:29:01.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:01.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:29:01.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:29:01.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:01.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:29:01.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:29:01.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:01.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:29:01.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:29:01.015 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:01.015 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:01.016 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:01.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.016 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:29:01.016 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:01.016 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 115.432851582, 2023-12-06T18:29:01.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:29:01.016 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:01.016 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:01.016 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:01.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:01.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:01.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:01.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:01.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:01.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:01.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:01.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:29:01.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:29:01.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:01.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:29:01.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:29:01.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:01.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:29:01.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:29:01.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:01.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:29:01.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:29:01.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:01.019 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:29:01.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:29:01.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:01.019 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:29:01.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:29:01.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:01.019 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:29:01.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:29:01.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:01.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:01.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:01.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:29:01.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:29:01.020 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:01.020 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:01.020 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:01.020 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:01.020 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:01.020 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:01.022 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:29:01.022 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:01.018+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:01.123 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:01.119+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:01.193 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:01.193 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:01.193 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:01.194 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:01.202 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:29:01.224 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:01.220+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:01.325 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:01.322+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:01.331 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:01.331 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:01.331 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:01.331 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:01.331 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:01.340 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:29:01.426 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:01.423+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:01.470 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:01.470 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:01.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:01.471 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:29:01.471 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:01.480 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:29:01.527 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:01.524+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:01.629 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:01.625+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:01.640 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:01.640 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:01.640 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:01.640 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:01.650 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:29:01.705 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":17,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:29:01.729 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:01.726+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:01.827 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:01.827 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:01.828 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:01.828 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:01.828 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:01.830 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:01.827+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:01.837 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:29:01.886 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-12-06T18:29:01.931 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:01.928+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:02.015 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:02.015 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:02.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:02.015 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:29:02.015 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:02.027 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:29:02.071 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:02.067+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:02.135 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:02.131+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:02.175 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:02.175 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:02.176 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:02.176 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:02.185 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:29:02.269 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:02.265+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:02.336 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:02.332+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:02.361 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:02.361 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:02.361 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:02.362 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:02.362 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:02.371 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:29:02.436 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:02.433+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:02.538 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:02.534+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:02.611 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:02.611 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:02.612 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:02.612 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:02.612 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.612 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:29:02.612 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:02.612 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 127.197889785, 2023-12-06T18:29:02.612 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:29:02.612 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.612 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.612 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.612 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.612 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.612 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:29:02.612 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.612 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.613 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.613 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.613 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:02.613 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.613 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.613 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.613 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:02.613 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.613 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.613 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:29:02.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:29:02.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.614 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:29:02.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:29:02.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.614 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:29:02.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:29:02.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.615 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:29:02.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:29:02.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:29:02.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:29:02.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:29:02.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:29:02.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:29:02.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:29:02.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:29:02.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:29:02.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:29:02.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:29:02.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:29:02.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.619 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.619 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:29:02.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:29:02.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.620 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.620 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.620 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:29:02.620 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:29:02.620 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.620 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.620 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.621 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:29:02.621 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:29:02.621 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.621 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.621 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.621 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:29:02.622 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:29:02.622 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.622 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.622 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.622 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.622 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.622 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:02.622 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:02.622 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 125.08931654600001, 2023-12-06T18:29:02.622 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:29:02.623 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.623 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.623 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.623 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.623 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.623 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:29:02.623 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.623 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.623 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.623 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.623 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:02.623 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.623 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.623 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.624 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.624 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:02.624 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.624 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.624 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.624 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.624 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:29:02.624 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:29:02.624 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.624 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.624 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.624 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:29:02.624 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:29:02.624 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.624 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:29:02.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:29:02.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:29:02.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:29:02.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:29:02.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:29:02.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.626 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:29:02.626 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:29:02.626 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.626 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.626 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.626 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:29:02.626 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:29:02.626 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.626 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.626 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.626 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:29:02.626 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:29:02.626 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.627 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:29:02.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:29:02.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.627 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:29:02.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:29:02.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.627 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:29:02.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:29:02.627 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.628 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.628 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.628 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.628 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:02.628 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:02.628 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 125.08857131400001, 2023-12-06T18:29:02.628 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:29:02.628 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.628 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.628 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.629 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.629 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.629 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:29:02.629 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.629 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.629 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.629 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.629 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:02.629 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.629 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.629 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.629 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.629 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:02.629 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.629 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.630 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.630 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.630 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:29:02.630 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:29:02.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.630 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.630 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.630 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:29:02.630 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:29:02.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.630 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.630 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.630 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:29:02.630 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:29:02.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.631 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.631 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.631 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:29:02.631 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:29:02.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.631 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.631 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.631 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:29:02.631 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:29:02.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.631 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.631 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.631 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:29:02.631 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:29:02.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.632 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.632 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:29:02.632 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:29:02.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.632 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.632 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:29:02.632 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:29:02.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.632 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.633 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:29:02.633 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:29:02.633 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.633 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.633 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.633 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:29:02.633 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:29:02.633 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.633 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.633 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.633 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:29:02.633 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:29:02.633 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.633 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.634 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.634 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.634 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.634 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:02.634 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:02.634 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 125.087887559, 2023-12-06T18:29:02.634 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:29:02.634 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.634 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.634 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.634 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.634 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.634 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:29:02.635 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.635 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.635 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.635 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.635 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:02.635 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.635 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.635 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.635 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.635 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:02.635 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.635 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.635 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.635 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.635 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:29:02.635 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:29:02.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:29:02.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:29:02.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:29:02.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:29:02.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:29:02.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:29:02.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:29:02.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:29:02.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:29:02.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:29:02.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:29:02.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:29:02.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:29:02.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:29:02.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:29:02.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:29:02.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:29:02.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:29:02.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:29:02.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:29:02.639 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.639 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.639 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.639 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:02.639 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:02.639 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 125.08231430399999, 2023-12-06T18:29:02.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:29:02.640 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.640 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.640 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.640 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.640 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.640 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:29:02.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.640 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:02.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:02.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:29:02.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:29:02.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:29:02.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:29:02.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.642 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:29:02.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:29:02.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.642 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.642 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.642 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:29:02.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:29:02.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.642 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.642 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.642 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:29:02.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:29:02.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.643 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.643 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.643 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:29:02.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:29:02.643 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.643 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.643 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.643 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:29:02.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:29:02.643 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.643 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.643 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.643 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:29:02.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:29:02.643 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.643 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:29:02.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:29:02.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:29:02.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:29:02.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:29:02.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:29:02.644 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.645 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.645 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.645 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:02.645 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:02.645 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 125.08164066400001, 2023-12-06T18:29:02.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:29:02.645 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.645 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.645 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.645 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.645 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.645 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:29:02.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.646 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:02.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:02.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:29:02.647 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:29:02.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.647 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:29:02.647 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:29:02.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.647 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:29:02.647 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:29:02.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.647 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:29:02.648 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:29:02.648 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.648 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.648 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.648 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:29:02.648 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:29:02.648 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.648 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.648 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.648 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:29:02.648 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:29:02.648 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.648 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.648 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.648 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:29:02.649 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:29:02.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.649 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.649 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.649 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:29:02.649 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:29:02.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.649 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.649 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.650 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:29:02.650 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:29:02.650 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.650 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.650 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:29:02.650 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:29:02.650 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.650 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.650 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:29:02.650 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:29:02.650 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.651 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.651 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.651 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.651 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.651 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:02.651 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:02.651 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 125.080978465, 2023-12-06T18:29:02.651 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:29:02.651 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.651 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.651 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.651 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.651 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.651 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:29:02.651 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.652 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.652 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:02.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.652 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:02.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.653 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.653 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:29:02.653 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:29:02.653 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.653 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.653 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.654 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:29:02.654 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:29:02.654 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.654 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.654 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.654 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:29:02.654 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:29:02.654 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.654 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.654 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.654 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:29:02.654 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:29:02.654 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.655 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.655 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.655 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:29:02.655 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:29:02.655 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.655 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.655 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.655 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:29:02.655 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:29:02.655 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.655 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.655 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.655 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:29:02.655 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:29:02.655 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.656 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.656 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.656 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:29:02.656 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:29:02.656 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.656 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.656 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.656 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:29:02.656 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:29:02.656 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.656 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.656 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.656 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:29:02.656 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:29:02.656 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.656 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.657 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.657 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:29:02.657 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:29:02.657 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.657 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.657 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.657 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.657 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.657 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:02.657 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:02.657 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 125.00826902999999, 2023-12-06T18:29:02.657 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:29:02.657 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.657 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.657 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.658 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.658 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.658 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:29:02.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.658 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.658 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:02.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.658 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:02.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.659 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:29:02.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:29:02.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.659 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:29:02.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:29:02.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.660 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:29:02.660 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:29:02.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.660 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.660 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:29:02.660 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:29:02.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.660 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.660 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:29:02.660 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:29:02.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.660 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:29:02.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:29:02.661 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.661 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.661 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:29:02.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:29:02.661 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.661 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.661 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:29:02.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:29:02.661 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.661 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.661 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:29:02.662 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:29:02.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.662 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.662 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.662 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:29:02.662 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:29:02.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.662 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.662 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.662 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:29:02.662 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:29:02.662 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.662 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.662 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.663 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.663 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:02.663 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:02.663 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 125.007587321, 2023-12-06T18:29:02.663 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:29:02.663 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.663 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.663 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.663 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.663 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.663 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:29:02.663 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.663 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.663 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.663 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.663 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:02.664 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.664 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.664 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.664 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.664 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:02.664 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.664 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.664 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.664 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.664 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:29:02.664 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:29:02.664 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.664 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.664 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.664 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:29:02.665 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:29:02.665 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.665 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.665 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.665 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:29:02.665 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:29:02.665 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.665 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.665 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.665 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:29:02.665 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:29:02.665 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.665 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.665 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.665 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:29:02.665 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:29:02.666 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.666 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.666 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.666 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:29:02.666 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:29:02.666 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.666 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.666 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.666 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:29:02.666 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:29:02.666 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.666 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.667 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.667 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:29:02.667 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:29:02.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.667 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.667 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.667 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:29:02.667 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:29:02.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.667 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.667 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.667 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:29:02.667 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:29:02.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.667 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.667 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.668 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:29:02.668 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:29:02.668 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.668 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.668 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.668 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.668 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:02.668 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:02.668 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 123.928783233, 2023-12-06T18:29:02.668 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:29:02.668 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.668 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.668 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.668 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.669 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.669 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:29:02.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.669 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.669 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.669 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:02.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.669 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.669 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:02.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.669 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.670 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:29:02.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:29:02.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.670 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.670 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:29:02.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:29:02.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.670 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.670 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:29:02.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:29:02.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:29:02.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:29:02.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:29:02.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:29:02.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:29:02.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:29:02.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:29:02.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:29:02.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:29:02.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:29:02.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:29:02.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:29:02.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:29:02.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:29:02.673 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.673 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.673 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.673 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:29:02.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:29:02.673 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.673 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.673 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.673 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.673 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.673 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:02.673 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:02.673 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 123.927366659, 2023-12-06T18:29:02.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:29:02.673 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.673 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.673 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.674 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.674 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.674 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:29:02.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.674 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:02.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:02.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:29:02.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:29:02.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:29:02.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:29:02.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:29:02.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:29:02.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.676 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.676 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:29:02.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:29:02.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.676 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.676 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.676 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:29:02.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:29:02.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.676 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.676 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.676 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:29:02.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:29:02.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:29:02.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:29:02.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:29:02.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:29:02.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:29:02.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:29:02.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:29:02.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:29:02.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:29:02.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:29:02.679 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.679 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.679 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.679 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:02.679 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:02.679 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 123.926281534, 2023-12-06T18:29:02.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:29:02.679 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.679 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.679 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.679 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.679 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.679 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:29:02.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.680 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:02.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:02.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.681 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:29:02.681 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:29:02.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:29:02.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:29:02.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.682 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.682 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:29:02.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:29:02.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.683 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.683 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.683 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:29:02.683 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:29:02.683 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.683 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.684 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.684 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:29:02.684 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:29:02.684 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.684 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.684 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.684 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:29:02.685 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:29:02.685 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:29:02.685 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:29:02.685 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.686 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.686 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.686 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:29:02.686 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:29:02.686 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.686 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.686 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.686 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:29:02.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:29:02.687 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.687 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.687 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.687 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:29:02.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:29:02.687 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.688 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.688 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:29:02.688 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:29:02.688 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.688 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.689 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.689 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:02.689 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:02.689 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 123.89113243200001, 2023-12-06T18:29:02.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:29:02.690 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.690 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.690 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.690 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.690 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.690 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:29:02.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.690 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:02.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:02.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:29:02.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:29:02.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:29:02.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:29:02.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:29:02.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:29:02.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:29:02.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:29:02.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:29:02.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:29:02.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:29:02.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:29:02.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:29:02.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:29:02.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:29:02.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:29:02.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:29:02.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:29:02.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:29:02.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:29:02.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:29:02.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:29:02.695 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.695 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.695 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.696 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:02.696 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:02.696 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 117.34140190700001, 2023-12-06T18:29:02.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:29:02.696 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.696 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.696 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.696 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.696 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.696 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:29:02.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.696 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:02.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:02.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:29:02.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:29:02.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:29:02.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:29:02.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:29:02.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:29:02.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:29:02.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:29:02.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:29:02.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:29:02.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:29:02.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:29:02.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:29:02.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:29:02.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:29:02.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:29:02.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:29:02.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:29:02.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:29:02.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:29:02.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:29:02.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:29:02.702 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.702 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.702 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.702 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:02.702 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:02.702 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 117.340596299, 2023-12-06T18:29:02.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:29:02.702 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.703 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.703 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.703 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.703 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.703 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:29:02.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.703 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:02.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:02.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:29:02.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:29:02.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:29:02.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:29:02.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:29:02.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:29:02.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.705 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.705 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:29:02.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:29:02.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.705 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.705 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:29:02.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:29:02.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.705 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.705 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:29:02.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:29:02.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:29:02.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:29:02.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:29:02.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:29:02.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:29:02.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:29:02.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:29:02.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:29:02.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:29:02.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:29:02.707 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.707 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.707 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.708 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:02.708 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:02.708 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 117.33990549000001, 2023-12-06T18:29:02.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:29:02.708 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.708 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.709 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.709 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.709 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:29:02.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.709 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:02.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:02.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.710 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:29:02.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:29:02.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.710 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:29:02.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:29:02.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.710 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:29:02.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:29:02.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:29:02.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:29:02.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:29:02.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:29:02.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:29:02.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:29:02.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:29:02.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:29:02.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:29:02.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:29:02.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:29:02.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:29:02.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:29:02.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:29:02.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:29:02.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:29:02.714 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.714 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.714 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.714 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:02.714 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:02.714 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 117.335607982, 2023-12-06T18:29:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:29:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:29:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:02.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:29:02.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:29:02.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:29:02.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:29:02.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:29:02.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:29:02.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:29:02.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:29:02.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:29:02.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:29:02.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:29:02.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:29:02.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:29:02.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:29:02.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:29:02.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:29:02.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:29:02.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:29:02.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:29:02.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:29:02.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:29:02.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:29:02.720 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.720 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.720 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.720 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:02.720 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:02.720 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 117.334940307, 2023-12-06T18:29:02.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:29:02.720 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.720 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.720 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.721 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.721 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.721 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:29:02.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.721 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:02.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:02.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:29:02.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:29:02.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:29:02.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:29:02.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:29:02.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:29:02.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:29:02.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:29:02.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:29:02.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:29:02.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:29:02.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:29:02.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:29:02.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:29:02.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:29:02.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:29:02.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:29:02.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:29:02.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:29:02.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:29:02.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:29:02.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:29:02.726 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.726 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.726 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.726 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:02.726 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:02.726 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 117.33428487899999, 2023-12-06T18:29:02.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:29:02.726 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.726 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.726 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.726 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.726 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.727 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:29:02.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.727 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:02.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:02.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:29:02.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:29:02.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:29:02.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:29:02.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:29:02.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:29:02.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:29:02.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:29:02.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:29:02.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:29:02.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:29:02.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:29:02.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:29:02.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:29:02.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:29:02.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:29:02.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:29:02.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:29:02.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:29:02.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:29:02.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:29:02.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:29:02.731 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.731 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.732 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.732 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:02.732 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:02.732 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 117.29363033200001, 2023-12-06T18:29:02.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:29:02.732 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:02.732 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:02.732 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:02.732 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:02.732 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:02.732 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:29:02.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.733 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:02.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:02.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:02.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:02.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:02.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:02.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:02.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:29:02.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:29:02.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:02.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:29:02.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:29:02.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:02.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:29:02.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:29:02.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:02.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:29:02.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:29:02.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:02.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:29:02.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:29:02.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:02.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:29:02.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:29:02.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:02.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:29:02.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:29:02.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:02.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:29:02.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:29:02.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:02.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:29:02.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:29:02.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:02.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:29:02.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:29:02.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:02.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:02.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:02.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:29:02.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:29:02.737 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.737 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.737 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.738 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:02.738 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:02.738 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:02.738 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:29:02.740 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:29:02.741 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:02.736+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:02.841 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:02.837+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:02.921 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:02.921 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:02.921 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:02.921 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:02.931 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:29:02.942 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:02.938+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:03.042 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:29:03.043 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:29:03.043 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:03.039+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:03.118 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:03.119 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:03.119 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:03.119 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:03.119 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:03.128 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:29:03.134 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is set 2023-12-06T18:29:03.175 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:29:03.175 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:29:03.177 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:03.172+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:03.245 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:03.241+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:03.287 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:29:03.287 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:29:03.316 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [7] live_osds: [0, 1, 2, 3, 4, 5, 6] 2023-12-06T18:29:03.316 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:29:03.316 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2023-12-06T18:29:03.317 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2023-12-06T18:29:03.327 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:03.327 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:03.337 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:29:03.379 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:03.375+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:03.380 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:29:03.380 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:29:03.447 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:03.443+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:03.471 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:29:03.471 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:29:03.518 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:03.518 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:03.527 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:29:03.548 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:03.544+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:03.557 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:29:03.557 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:29:03.649 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:03.645+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:03.651 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:29:03.651 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:29:03.672 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:03.672 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:03.681 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:29:03.745 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:29:03.750 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:03.746+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:03.750 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:29:03.751 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:29:03.851 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:03.847+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:03.862 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:03.862 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:03.871 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:29:03.951 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:03.948+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:03.999 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:03.999 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:04.008 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:29:04.052 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:04.049+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:04.067 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:29:04.067 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:04.067 INFO:teuthology.orchestra.run.smithi061.stdout: "epoch": 51, 2023-12-06T18:29:04.068 INFO:teuthology.orchestra.run.smithi061.stdout: "fsid": "497e3283-cc16-40ff-a72c-a7487474fb4b", 2023-12-06T18:29:04.068 INFO:teuthology.orchestra.run.smithi061.stdout: "created": "2023-12-06T18:26:07.122862+0000", 2023-12-06T18:29:04.068 INFO:teuthology.orchestra.run.smithi061.stdout: "modified": "2023-12-06T18:29:03.014891+0000", 2023-12-06T18:29:04.068 INFO:teuthology.orchestra.run.smithi061.stdout: "last_up_change": "2023-12-06T18:28:15.021910+0000", 2023-12-06T18:29:04.068 INFO:teuthology.orchestra.run.smithi061.stdout: "last_in_change": "2023-12-06T18:28:45.621304+0000", 2023-12-06T18:29:04.068 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-06T18:29:04.068 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_num": 5801984, 2023-12-06T18:29:04.068 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_set": [ 2023-12-06T18:29:04.068 INFO:teuthology.orchestra.run.smithi061.stdout: "noscrub", 2023-12-06T18:29:04.068 INFO:teuthology.orchestra.run.smithi061.stdout: "pglog_hardlimit", 2023-12-06T18:29:04.068 INFO:teuthology.orchestra.run.smithi061.stdout: "purged_snapdirs", 2023-12-06T18:29:04.068 INFO:teuthology.orchestra.run.smithi061.stdout: "recovery_deletes", 2023-12-06T18:29:04.068 INFO:teuthology.orchestra.run.smithi061.stdout: "sortbitwise" 2023-12-06T18:29:04.068 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:29:04.068 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_version": 7, 2023-12-06T18:29:04.069 INFO:teuthology.orchestra.run.smithi061.stdout: "full_ratio": 0.94999998807907104, 2023-12-06T18:29:04.069 INFO:teuthology.orchestra.run.smithi061.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-12-06T18:29:04.069 INFO:teuthology.orchestra.run.smithi061.stdout: "nearfull_ratio": 0.85000002384185791, 2023-12-06T18:29:04.069 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_snapshot": "", 2023-12-06T18:29:04.069 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_max": 2, 2023-12-06T18:29:04.069 INFO:teuthology.orchestra.run.smithi061.stdout: "max_osd": 8, 2023-12-06T18:29:04.069 INFO:teuthology.orchestra.run.smithi061.stdout: "require_min_compat_client": "luminous", 2023-12-06T18:29:04.069 INFO:teuthology.orchestra.run.smithi061.stdout: "min_compat_client": "jewel", 2023-12-06T18:29:04.069 INFO:teuthology.orchestra.run.smithi061.stdout: "require_osd_release": "reef", 2023-12-06T18:29:04.069 INFO:teuthology.orchestra.run.smithi061.stdout: "allow_crimson": false, 2023-12-06T18:29:04.069 INFO:teuthology.orchestra.run.smithi061.stdout: "pools": [ 2023-12-06T18:29:04.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.069 INFO:teuthology.orchestra.run.smithi061.stdout: "pool": 1, 2023-12-06T18:29:04.069 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_name": ".mgr", 2023-12-06T18:29:04.069 INFO:teuthology.orchestra.run.smithi061.stdout: "create_time": "2023-12-06T18:26:52.648377+0000", 2023-12-06T18:29:04.070 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-12-06T18:29:04.070 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_names": "hashpspool", 2023-12-06T18:29:04.070 INFO:teuthology.orchestra.run.smithi061.stdout: "type": 1, 2023-12-06T18:29:04.070 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 2, 2023-12-06T18:29:04.070 INFO:teuthology.orchestra.run.smithi061.stdout: "min_size": 1, 2023-12-06T18:29:04.070 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_rule": 0, 2023-12-06T18:29:04.070 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_count": 0, 2023-12-06T18:29:04.070 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_target": 0, 2023-12-06T18:29:04.070 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_barrier": 0, 2023-12-06T18:29:04.070 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-06T18:29:04.070 INFO:teuthology.orchestra.run.smithi061.stdout: "object_hash": 2, 2023-12-06T18:29:04.070 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscale_mode": "off", 2023-12-06T18:29:04.070 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num": 1, 2023-12-06T18:29:04.070 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num": 1, 2023-12-06T18:29:04.070 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num_target": 1, 2023-12-06T18:29:04.071 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_target": 1, 2023-12-06T18:29:04.071 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_pending": 1, 2023-12-06T18:29:04.071 INFO:teuthology.orchestra.run.smithi061.stdout: "last_pg_merge_meta": { 2023-12-06T18:29:04.071 INFO:teuthology.orchestra.run.smithi061.stdout: "source_pgid": "0.0", 2023-12-06T18:29:04.071 INFO:teuthology.orchestra.run.smithi061.stdout: "ready_epoch": 0, 2023-12-06T18:29:04.071 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_started": 0, 2023-12-06T18:29:04.071 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_clean": 0, 2023-12-06T18:29:04.071 INFO:teuthology.orchestra.run.smithi061.stdout: "source_version": "0'0", 2023-12-06T18:29:04.071 INFO:teuthology.orchestra.run.smithi061.stdout: "target_version": "0'0" 2023-12-06T18:29:04.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.071 INFO:teuthology.orchestra.run.smithi061.stdout: "last_change": "19", 2023-12-06T18:29:04.071 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend": "0", 2023-12-06T18:29:04.071 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-06T18:29:04.071 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_preluminous": "0", 2023-12-06T18:29:04.071 INFO:teuthology.orchestra.run.smithi061.stdout: "auid": 0, 2023-12-06T18:29:04.071 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_mode": "selfmanaged", 2023-12-06T18:29:04.072 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_seq": 0, 2023-12-06T18:29:04.072 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_epoch": 0, 2023-12-06T18:29:04.072 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_snaps": [], 2023-12-06T18:29:04.072 INFO:teuthology.orchestra.run.smithi061.stdout: "removed_snaps": "[]", 2023-12-06T18:29:04.072 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_bytes": 0, 2023-12-06T18:29:04.072 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_objects": 0, 2023-12-06T18:29:04.072 INFO:teuthology.orchestra.run.smithi061.stdout: "tiers": [], 2023-12-06T18:29:04.072 INFO:teuthology.orchestra.run.smithi061.stdout: "tier_of": -1, 2023-12-06T18:29:04.072 INFO:teuthology.orchestra.run.smithi061.stdout: "read_tier": -1, 2023-12-06T18:29:04.072 INFO:teuthology.orchestra.run.smithi061.stdout: "write_tier": -1, 2023-12-06T18:29:04.072 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_mode": "none", 2023-12-06T18:29:04.072 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_bytes": 0, 2023-12-06T18:29:04.072 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_objects": 0, 2023-12-06T18:29:04.072 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-06T18:29:04.072 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-06T18:29:04.073 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-06T18:29:04.073 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_flush_age": 0, 2023-12-06T18:29:04.073 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_evict_age": 0, 2023-12-06T18:29:04.073 INFO:teuthology.orchestra.run.smithi061.stdout: "erasure_code_profile": "", 2023-12-06T18:29:04.073 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_params": { 2023-12-06T18:29:04.073 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "none" 2023-12-06T18:29:04.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.073 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_period": 0, 2023-12-06T18:29:04.073 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_count": 0, 2023-12-06T18:29:04.073 INFO:teuthology.orchestra.run.smithi061.stdout: "use_gmt_hitset": true, 2023-12-06T18:29:04.073 INFO:teuthology.orchestra.run.smithi061.stdout: "min_read_recency_for_promote": 0, 2023-12-06T18:29:04.073 INFO:teuthology.orchestra.run.smithi061.stdout: "min_write_recency_for_promote": 0, 2023-12-06T18:29:04.073 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_grade_decay_rate": 0, 2023-12-06T18:29:04.073 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_search_last_n": 0, 2023-12-06T18:29:04.073 INFO:teuthology.orchestra.run.smithi061.stdout: "grade_table": [], 2023-12-06T18:29:04.074 INFO:teuthology.orchestra.run.smithi061.stdout: "stripe_width": 0, 2023-12-06T18:29:04.074 INFO:teuthology.orchestra.run.smithi061.stdout: "expected_num_objects": 0, 2023-12-06T18:29:04.074 INFO:teuthology.orchestra.run.smithi061.stdout: "fast_read": false, 2023-12-06T18:29:04.074 INFO:teuthology.orchestra.run.smithi061.stdout: "options": { 2023-12-06T18:29:04.074 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_max": 32, 2023-12-06T18:29:04.074 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_min": 1 2023-12-06T18:29:04.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.074 INFO:teuthology.orchestra.run.smithi061.stdout: "application_metadata": { 2023-12-06T18:29:04.074 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr": {} 2023-12-06T18:29:04.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.074 INFO:teuthology.orchestra.run.smithi061.stdout: "read_balance": { 2023-12-06T18:29:04.074 INFO:teuthology.orchestra.run.smithi061.stdout: "score_acting": 6.9000000953674316, 2023-12-06T18:29:04.074 INFO:teuthology.orchestra.run.smithi061.stdout: "score_stable": 6.9000000953674316, 2023-12-06T18:29:04.074 INFO:teuthology.orchestra.run.smithi061.stdout: "optimal_score": 0.28999999165534973, 2023-12-06T18:29:04.075 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_acting": 2, 2023-12-06T18:29:04.075 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_stable": 2, 2023-12-06T18:29:04.075 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity_weighted": 0.82932090759277344, 2023-12-06T18:29:04.075 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity": 0.94999998807907104, 2023-12-06T18:29:04.075 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2023-12-06T18:29:04.075 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.075 INFO:teuthology.orchestra.run.smithi061.stdout: "pool": 2, 2023-12-06T18:29:04.075 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_name": "rbd", 2023-12-06T18:29:04.075 INFO:teuthology.orchestra.run.smithi061.stdout: "create_time": "2023-12-06T18:27:01.712581+0000", 2023-12-06T18:29:04.075 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-12-06T18:29:04.075 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_names": "hashpspool", 2023-12-06T18:29:04.075 INFO:teuthology.orchestra.run.smithi061.stdout: "type": 1, 2023-12-06T18:29:04.075 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 2, 2023-12-06T18:29:04.075 INFO:teuthology.orchestra.run.smithi061.stdout: "min_size": 1, 2023-12-06T18:29:04.076 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_rule": 0, 2023-12-06T18:29:04.076 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_count": 0, 2023-12-06T18:29:04.076 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_target": 0, 2023-12-06T18:29:04.076 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_barrier": 0, 2023-12-06T18:29:04.076 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-06T18:29:04.076 INFO:teuthology.orchestra.run.smithi061.stdout: "object_hash": 2, 2023-12-06T18:29:04.076 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscale_mode": "off", 2023-12-06T18:29:04.076 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num": 8, 2023-12-06T18:29:04.076 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num": 8, 2023-12-06T18:29:04.076 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num_target": 8, 2023-12-06T18:29:04.076 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_target": 8, 2023-12-06T18:29:04.076 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_pending": 8, 2023-12-06T18:29:04.076 INFO:teuthology.orchestra.run.smithi061.stdout: "last_pg_merge_meta": { 2023-12-06T18:29:04.076 INFO:teuthology.orchestra.run.smithi061.stdout: "source_pgid": "0.0", 2023-12-06T18:29:04.076 INFO:teuthology.orchestra.run.smithi061.stdout: "ready_epoch": 0, 2023-12-06T18:29:04.077 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_started": 0, 2023-12-06T18:29:04.077 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_clean": 0, 2023-12-06T18:29:04.077 INFO:teuthology.orchestra.run.smithi061.stdout: "source_version": "0'0", 2023-12-06T18:29:04.077 INFO:teuthology.orchestra.run.smithi061.stdout: "target_version": "0'0" 2023-12-06T18:29:04.077 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.077 INFO:teuthology.orchestra.run.smithi061.stdout: "last_change": "24", 2023-12-06T18:29:04.077 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend": "0", 2023-12-06T18:29:04.077 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-06T18:29:04.077 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_preluminous": "0", 2023-12-06T18:29:04.077 INFO:teuthology.orchestra.run.smithi061.stdout: "auid": 0, 2023-12-06T18:29:04.077 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_mode": "selfmanaged", 2023-12-06T18:29:04.077 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_seq": 0, 2023-12-06T18:29:04.077 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_epoch": 0, 2023-12-06T18:29:04.077 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_snaps": [], 2023-12-06T18:29:04.077 INFO:teuthology.orchestra.run.smithi061.stdout: "removed_snaps": "[]", 2023-12-06T18:29:04.078 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_bytes": 0, 2023-12-06T18:29:04.078 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_objects": 0, 2023-12-06T18:29:04.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tiers": [], 2023-12-06T18:29:04.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tier_of": -1, 2023-12-06T18:29:04.078 INFO:teuthology.orchestra.run.smithi061.stdout: "read_tier": -1, 2023-12-06T18:29:04.078 INFO:teuthology.orchestra.run.smithi061.stdout: "write_tier": -1, 2023-12-06T18:29:04.078 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_mode": "none", 2023-12-06T18:29:04.078 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_bytes": 0, 2023-12-06T18:29:04.078 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_objects": 0, 2023-12-06T18:29:04.078 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-06T18:29:04.078 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-06T18:29:04.078 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-06T18:29:04.078 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_flush_age": 0, 2023-12-06T18:29:04.078 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_evict_age": 0, 2023-12-06T18:29:04.078 INFO:teuthology.orchestra.run.smithi061.stdout: "erasure_code_profile": "", 2023-12-06T18:29:04.078 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_params": { 2023-12-06T18:29:04.079 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "none" 2023-12-06T18:29:04.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.079 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_period": 0, 2023-12-06T18:29:04.079 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_count": 0, 2023-12-06T18:29:04.079 INFO:teuthology.orchestra.run.smithi061.stdout: "use_gmt_hitset": true, 2023-12-06T18:29:04.079 INFO:teuthology.orchestra.run.smithi061.stdout: "min_read_recency_for_promote": 0, 2023-12-06T18:29:04.079 INFO:teuthology.orchestra.run.smithi061.stdout: "min_write_recency_for_promote": 0, 2023-12-06T18:29:04.079 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_grade_decay_rate": 0, 2023-12-06T18:29:04.079 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_search_last_n": 0, 2023-12-06T18:29:04.079 INFO:teuthology.orchestra.run.smithi061.stdout: "grade_table": [], 2023-12-06T18:29:04.079 INFO:teuthology.orchestra.run.smithi061.stdout: "stripe_width": 0, 2023-12-06T18:29:04.079 INFO:teuthology.orchestra.run.smithi061.stdout: "expected_num_objects": 0, 2023-12-06T18:29:04.079 INFO:teuthology.orchestra.run.smithi061.stdout: "fast_read": false, 2023-12-06T18:29:04.079 INFO:teuthology.orchestra.run.smithi061.stdout: "options": {}, 2023-12-06T18:29:04.079 INFO:teuthology.orchestra.run.smithi061.stdout: "application_metadata": { 2023-12-06T18:29:04.079 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd": {} 2023-12-06T18:29:04.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.080 INFO:teuthology.orchestra.run.smithi061.stdout: "read_balance": { 2023-12-06T18:29:04.080 INFO:teuthology.orchestra.run.smithi061.stdout: "score_acting": 4.3600001335144043, 2023-12-06T18:29:04.080 INFO:teuthology.orchestra.run.smithi061.stdout: "score_stable": 4.3600001335144043, 2023-12-06T18:29:04.080 INFO:teuthology.orchestra.run.smithi061.stdout: "optimal_score": 0.86000001430511475, 2023-12-06T18:29:04.080 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_acting": 3.75, 2023-12-06T18:29:04.080 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_stable": 3.75, 2023-12-06T18:29:04.080 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity_weighted": 0.82932090759277344, 2023-12-06T18:29:04.080 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity": 0.94999998807907104, 2023-12-06T18:29:04.080 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2023-12-06T18:29:04.080 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.080 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.080 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:29:04.080 INFO:teuthology.orchestra.run.smithi061.stdout: "osds": [ 2023-12-06T18:29:04.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.080 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 0, 2023-12-06T18:29:04.080 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "5e6e17b9-e72e-4e34-a531-0830d659bfe0", 2023-12-06T18:29:04.080 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:29:04.080 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:29:04.081 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:29:04.081 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:29:04.081 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:29:04.081 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:29:04.081 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 16, 2023-12-06T18:29:04.081 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 0, 2023-12-06T18:29:04.081 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:29:04.081 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:29:04.081 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:29:04.081 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.081 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.081 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6805", 2023-12-06T18:29:04.081 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:29:04.081 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.081 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.082 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.082 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:29:04.082 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.082 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.082 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.082 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6806", 2023-12-06T18:29:04.082 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:29:04.082 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.082 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.082 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.082 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:29:04.082 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.082 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.082 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.083 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6808", 2023-12-06T18:29:04.083 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:29:04.083 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.083 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.083 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:29:04.083 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.083 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.083 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.083 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6807", 2023-12-06T18:29:04.083 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:29:04.083 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.083 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.083 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6805/281956997", 2023-12-06T18:29:04.083 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6806/281956997", 2023-12-06T18:29:04.084 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6808/281956997", 2023-12-06T18:29:04.084 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6807/281956997", 2023-12-06T18:29:04.084 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:29:04.084 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:29:04.084 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:29:04.084 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.084 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 1, 2023-12-06T18:29:04.084 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "b78c49b0-360d-4351-b44e-65ccaaeb768f", 2023-12-06T18:29:04.084 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:29:04.084 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:29:04.084 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:29:04.084 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:29:04.084 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:29:04.084 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:29:04.084 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 16, 2023-12-06T18:29:04.085 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 44, 2023-12-06T18:29:04.085 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:29:04.085 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:29:04.085 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:29:04.085 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.085 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.085 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6809", 2023-12-06T18:29:04.085 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:29:04.085 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.085 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.085 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:29:04.085 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.086 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.086 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6810", 2023-12-06T18:29:04.086 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:29:04.086 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.086 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.086 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:29:04.086 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.086 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.086 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6812", 2023-12-06T18:29:04.086 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:29:04.086 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.086 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.087 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:29:04.087 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.087 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.087 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6811", 2023-12-06T18:29:04.087 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:29:04.087 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.087 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.087 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6809/3306625622", 2023-12-06T18:29:04.087 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6810/3306625622", 2023-12-06T18:29:04.087 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6812/3306625622", 2023-12-06T18:29:04.087 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6811/3306625622", 2023-12-06T18:29:04.087 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:29:04.087 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:29:04.088 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:29:04.088 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.088 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.088 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.088 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 2, 2023-12-06T18:29:04.088 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "a2af3151-a4da-4c34-8fb6-78abf035785b", 2023-12-06T18:29:04.088 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:29:04.088 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:29:04.088 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:29:04.088 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 0.6345672607421875, 2023-12-06T18:29:04.088 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:29:04.088 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:29:04.088 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 16, 2023-12-06T18:29:04.088 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 35, 2023-12-06T18:29:04.088 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:29:04.088 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:29:04.088 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:29:04.089 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.089 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.089 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6801", 2023-12-06T18:29:04.089 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:29:04.089 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.089 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.089 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:29:04.089 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.089 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.089 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6802", 2023-12-06T18:29:04.089 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:29:04.089 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.089 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.089 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:29:04.090 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.090 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.090 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6804", 2023-12-06T18:29:04.090 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:29:04.090 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.090 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.090 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:29:04.090 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.090 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.090 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6803", 2023-12-06T18:29:04.090 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:29:04.090 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.090 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.091 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6801/3589270120", 2023-12-06T18:29:04.091 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6802/3589270120", 2023-12-06T18:29:04.091 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6804/3589270120", 2023-12-06T18:29:04.091 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6803/3589270120", 2023-12-06T18:29:04.091 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:29:04.091 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:29:04.091 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:29:04.091 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.091 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 3, 2023-12-06T18:29:04.091 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "1c754ed6-c535-454f-b35b-00f62cd6ce5a", 2023-12-06T18:29:04.091 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:29:04.091 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:29:04.091 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:29:04.091 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:29:04.092 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:29:04.092 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:29:04.092 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 18, 2023-12-06T18:29:04.092 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 44, 2023-12-06T18:29:04.092 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:29:04.092 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:29:04.092 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:29:04.092 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.092 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.092 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6813", 2023-12-06T18:29:04.092 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:29:04.092 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.092 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.092 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:29:04.092 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.093 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.093 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.093 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6814", 2023-12-06T18:29:04.093 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:29:04.093 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.093 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.093 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.093 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:29:04.093 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.093 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.093 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.093 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6816", 2023-12-06T18:29:04.093 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:29:04.093 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.093 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.093 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.094 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:29:04.094 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.094 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.094 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.094 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6815", 2023-12-06T18:29:04.094 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:29:04.094 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.094 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.094 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.094 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6813/2849920842", 2023-12-06T18:29:04.094 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6814/2849920842", 2023-12-06T18:29:04.094 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6816/2849920842", 2023-12-06T18:29:04.094 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6815/2849920842", 2023-12-06T18:29:04.094 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:29:04.094 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:29:04.095 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:29:04.095 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.095 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.095 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 4, 2023-12-06T18:29:04.095 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "766572af-b9ca-4f38-927d-36f26101ba51", 2023-12-06T18:29:04.095 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:29:04.095 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:29:04.095 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:29:04.095 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:29:04.095 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:29:04.095 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:29:04.095 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 15, 2023-12-06T18:29:04.095 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 38, 2023-12-06T18:29:04.095 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:29:04.095 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:29:04.095 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:29:04.096 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.096 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.096 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6800", 2023-12-06T18:29:04.096 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:29:04.096 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.096 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.096 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:29:04.096 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.096 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.096 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6801", 2023-12-06T18:29:04.096 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:29:04.096 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.096 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.097 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:29:04.097 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.097 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.097 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6803", 2023-12-06T18:29:04.097 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:29:04.097 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.097 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.097 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:29:04.097 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.097 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.097 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6802", 2023-12-06T18:29:04.097 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:29:04.098 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.098 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.098 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.098 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6800/2778770069", 2023-12-06T18:29:04.098 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6801/2778770069", 2023-12-06T18:29:04.098 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6803/2778770069", 2023-12-06T18:29:04.098 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6802/2778770069", 2023-12-06T18:29:04.098 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:29:04.098 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:29:04.098 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:29:04.098 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.098 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.098 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.098 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 5, 2023-12-06T18:29:04.098 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "7a566b60-1543-41d7-894c-e2fbb63e48dc", 2023-12-06T18:29:04.099 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:29:04.099 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:29:04.099 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:29:04.099 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:29:04.099 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:29:04.099 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:29:04.099 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 15, 2023-12-06T18:29:04.099 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 46, 2023-12-06T18:29:04.099 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:29:04.099 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:29:04.099 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:29:04.099 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.099 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.099 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.099 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6809", 2023-12-06T18:29:04.100 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:29:04.100 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.100 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.100 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:29:04.100 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.100 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.100 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6810", 2023-12-06T18:29:04.100 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:29:04.100 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.100 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.100 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:29:04.100 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.101 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.101 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6814", 2023-12-06T18:29:04.101 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:29:04.101 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.101 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.101 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:29:04.101 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.101 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.101 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6812", 2023-12-06T18:29:04.101 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:29:04.101 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.101 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.101 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6809/263671335", 2023-12-06T18:29:04.102 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6810/263671335", 2023-12-06T18:29:04.102 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6814/263671335", 2023-12-06T18:29:04.102 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6812/263671335", 2023-12-06T18:29:04.102 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:29:04.102 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:29:04.102 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:29:04.102 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.102 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 6, 2023-12-06T18:29:04.102 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "28cc2358-4ac4-40b2-91a4-20edfe625a73", 2023-12-06T18:29:04.102 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:29:04.102 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:29:04.102 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:29:04.102 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 0, 2023-12-06T18:29:04.103 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:29:04.103 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:29:04.103 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 14, 2023-12-06T18:29:04.103 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 25, 2023-12-06T18:29:04.103 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:29:04.103 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:29:04.103 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:29:04.103 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.103 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.103 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.103 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6808", 2023-12-06T18:29:04.103 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:29:04.103 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.103 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.103 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:29:04.104 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.104 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.104 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6811", 2023-12-06T18:29:04.104 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:29:04.104 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.104 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.104 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:29:04.104 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.104 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.104 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6815", 2023-12-06T18:29:04.104 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:29:04.104 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.105 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.105 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:29:04.105 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.105 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.105 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6813", 2023-12-06T18:29:04.105 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:29:04.105 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.105 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.105 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6808/4245639079", 2023-12-06T18:29:04.105 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6811/4245639079", 2023-12-06T18:29:04.105 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6815/4245639079", 2023-12-06T18:29:04.105 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6813/4245639079", 2023-12-06T18:29:04.106 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:29:04.106 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:29:04.106 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:29:04.106 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.106 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 7, 2023-12-06T18:29:04.106 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "8f79279b-d2d0-438e-b3c2-88df1241ff30", 2023-12-06T18:29:04.106 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 0, 2023-12-06T18:29:04.106 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:29:04.106 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:29:04.106 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:29:04.106 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:29:04.106 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:29:04.106 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 14, 2023-12-06T18:29:04.107 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 25, 2023-12-06T18:29:04.107 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 38, 2023-12-06T18:29:04.107 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:29:04.107 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:29:04.107 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.107 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.107 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6804", 2023-12-06T18:29:04.107 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3940055127 2023-12-06T18:29:04.107 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.107 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.107 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:29:04.107 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.107 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.108 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6805", 2023-12-06T18:29:04.108 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3940055127 2023-12-06T18:29:04.108 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.108 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.108 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:29:04.108 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.108 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.108 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6807", 2023-12-06T18:29:04.108 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3940055127 2023-12-06T18:29:04.108 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.108 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.108 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:29:04.108 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:29:04.109 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.109 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:29:04.109 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6806", 2023-12-06T18:29:04.109 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3940055127 2023-12-06T18:29:04.109 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.109 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.109 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6804/3940055127", 2023-12-06T18:29:04.109 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6805/3940055127", 2023-12-06T18:29:04.109 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6807/3940055127", 2023-12-06T18:29:04.109 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6806/3940055127", 2023-12-06T18:29:04.109 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:29:04.109 INFO:teuthology.orchestra.run.smithi061.stdout: "exists" 2023-12-06T18:29:04.109 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.109 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.109 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:29:04.109 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_xinfo": [ 2023-12-06T18:29:04.110 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.110 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 0, 2023-12-06T18:29:04.110 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:29:04.110 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:29:04.110 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:29:04.110 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:29:04.110 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:29:04.110 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:52.171904+0000", 2023-12-06T18:29:04.110 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:29:04.110 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.110 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.110 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 1, 2023-12-06T18:29:04.110 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:29:04.110 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:29:04.111 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:29:04.111 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:29:04.111 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:29:04.111 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:52.428840+0000", 2023-12-06T18:29:04.111 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:29:04.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.111 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 2, 2023-12-06T18:29:04.111 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:29:04.111 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:29:04.111 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:29:04.111 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:29:04.111 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:29:04.111 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:51.681954+0000", 2023-12-06T18:29:04.111 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:29:04.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.112 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 3, 2023-12-06T18:29:04.112 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:29:04.112 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:29:04.112 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:29:04.112 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:29:04.112 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:29:04.112 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:53.687417+0000", 2023-12-06T18:29:04.112 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:29:04.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.112 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 4, 2023-12-06T18:29:04.112 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:29:04.112 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:29:04.112 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:29:04.112 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:29:04.113 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:29:04.113 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:50.158682+0000", 2023-12-06T18:29:04.113 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:29:04.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.113 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 5, 2023-12-06T18:29:04.113 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:29:04.113 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:29:04.113 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:29:04.113 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:29:04.113 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:29:04.113 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:50.443807+0000", 2023-12-06T18:29:04.113 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:29:04.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.113 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 6, 2023-12-06T18:29:04.114 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:29:04.114 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:29:04.114 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:29:04.114 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:29:04.114 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:29:04.114 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:49.750276+0000", 2023-12-06T18:29:04.114 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:29:04.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.114 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 7, 2023-12-06T18:29:04.114 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:29:04.114 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:29:04.114 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:29:04.114 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:29:04.114 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:29:04.114 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:49.770987+0000", 2023-12-06T18:29:04.115 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 38 2023-12-06T18:29:04.115 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.115 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:29:04.115 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_upmap": [], 2023-12-06T18:29:04.115 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_upmap_items": [], 2023-12-06T18:29:04.115 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_upmap_primaries": [], 2023-12-06T18:29:04.115 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_temp": [ 2023-12-06T18:29:04.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:04.115 INFO:teuthology.orchestra.run.smithi061.stdout: "pgid": "2.1", 2023-12-06T18:29:04.115 INFO:teuthology.orchestra.run.smithi061.stdout: "osds": [ 2023-12-06T18:29:04.115 INFO:teuthology.orchestra.run.smithi061.stdout: 5, 2023-12-06T18:29:04.115 INFO:teuthology.orchestra.run.smithi061.stdout: 4 2023-12-06T18:29:04.115 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:04.115 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.115 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:29:04.115 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_temp": [], 2023-12-06T18:29:04.116 INFO:teuthology.orchestra.run.smithi061.stdout: "blocklist": {}, 2023-12-06T18:29:04.116 INFO:teuthology.orchestra.run.smithi061.stdout: "range_blocklist": {}, 2023-12-06T18:29:04.116 INFO:teuthology.orchestra.run.smithi061.stdout: "erasure_code_profiles": { 2023-12-06T18:29:04.116 INFO:teuthology.orchestra.run.smithi061.stdout: "default": { 2023-12-06T18:29:04.116 INFO:teuthology.orchestra.run.smithi061.stdout: "crush-failure-domain": "osd", 2023-12-06T18:29:04.116 INFO:teuthology.orchestra.run.smithi061.stdout: "k": "2", 2023-12-06T18:29:04.116 INFO:teuthology.orchestra.run.smithi061.stdout: "m": "1", 2023-12-06T18:29:04.116 INFO:teuthology.orchestra.run.smithi061.stdout: "plugin": "jerasure", 2023-12-06T18:29:04.116 INFO:teuthology.orchestra.run.smithi061.stdout: "technique": "reed_sol_van" 2023-12-06T18:29:04.116 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:04.116 INFO:teuthology.orchestra.run.smithi061.stdout: "removed_snaps_queue": [], 2023-12-06T18:29:04.116 INFO:teuthology.orchestra.run.smithi061.stdout: "new_removed_snaps": [], 2023-12-06T18:29:04.116 INFO:teuthology.orchestra.run.smithi061.stdout: "new_purged_snaps": [], 2023-12-06T18:29:04.116 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_node_flags": {}, 2023-12-06T18:29:04.116 INFO:teuthology.orchestra.run.smithi061.stdout: "device_class_flags": {}, 2023-12-06T18:29:04.117 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode": { 2023-12-06T18:29:04.117 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode_enabled": false, 2023-12-06T18:29:04.117 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_bucket_count": 0, 2023-12-06T18:29:04.117 INFO:teuthology.orchestra.run.smithi061.stdout: "degraded_stretch_mode": 0, 2023-12-06T18:29:04.117 INFO:teuthology.orchestra.run.smithi061.stdout: "recovering_stretch_mode": 0, 2023-12-06T18:29:04.117 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode_bucket": 0 2023-12-06T18:29:04.117 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:04.117 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:04.118 INFO:tasks.thrashosds.thrasher:j is {'epoch': 51, 'fsid': '497e3283-cc16-40ff-a72c-a7487474fb4b', 'created': '2023-12-06T18:26:07.122862+0000', 'modified': '2023-12-06T18:29:03.014891+0000', 'last_up_change': '2023-12-06T18:28:15.021910+0000', 'last_in_change': '2023-12-06T18:28:45.621304+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-06T18:26:52.648377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8293209075927734, 'average_primary_affinity': 0.949999988079071, 'average_primary_affinity_weighted': 0.8299999833106995}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-12-06T18:27:01.712581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.360000133514404, 'score_stable': 4.360000133514404, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 3.75, 'raw_score_stable': 3.75, 'primary_affinity_weighted': 0.8293209075927734, 'average_primary_affinity': 0.949999988079071, 'average_primary_affinity_weighted': 0.8299999833106995}}], 'osds': [{'osd': 0, 'uuid': '5e6e17b9-e72e-4e34-a531-0830d659bfe0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6805', 'nonce': 281956997}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6806', 'nonce': 281956997}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6808', 'nonce': 281956997}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6807', 'nonce': 281956997}]}, 'public_addr': '172.21.15.61:6805/281956997', 'cluster_addr': '172.21.15.61:6806/281956997', 'heartbeat_back_addr': '172.21.15.61:6808/281956997', 'heartbeat_front_addr': '172.21.15.61:6807/281956997', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b78c49b0-360d-4351-b44e-65ccaaeb768f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6809', 'nonce': 3306625622}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6810', 'nonce': 3306625622}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6812', 'nonce': 3306625622}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6811', 'nonce': 3306625622}]}, 'public_addr': '172.21.15.61:6809/3306625622', 'cluster_addr': '172.21.15.61:6810/3306625622', 'heartbeat_back_addr': '172.21.15.61:6812/3306625622', 'heartbeat_front_addr': '172.21.15.61:6811/3306625622', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'a2af3151-a4da-4c34-8fb6-78abf035785b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.6345672607421875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 35, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6801', 'nonce': 3589270120}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6802', 'nonce': 3589270120}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6804', 'nonce': 3589270120}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6803', 'nonce': 3589270120}]}, 'public_addr': '172.21.15.61:6801/3589270120', 'cluster_addr': '172.21.15.61:6802/3589270120', 'heartbeat_back_addr': '172.21.15.61:6804/3589270120', 'heartbeat_front_addr': '172.21.15.61:6803/3589270120', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '1c754ed6-c535-454f-b35b-00f62cd6ce5a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6813', 'nonce': 2849920842}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6814', 'nonce': 2849920842}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6816', 'nonce': 2849920842}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6815', 'nonce': 2849920842}]}, 'public_addr': '172.21.15.61:6813/2849920842', 'cluster_addr': '172.21.15.61:6814/2849920842', 'heartbeat_back_addr': '172.21.15.61:6816/2849920842', 'heartbeat_front_addr': '172.21.15.61:6815/2849920842', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '766572af-b9ca-4f38-927d-36f26101ba51', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 38, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6800', 'nonce': 2778770069}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6801', 'nonce': 2778770069}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6803', 'nonce': 2778770069}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6802', 'nonce': 2778770069}]}, 'public_addr': '172.21.15.152:6800/2778770069', 'cluster_addr': '172.21.15.152:6801/2778770069', 'heartbeat_back_addr': '172.21.15.152:6803/2778770069', 'heartbeat_front_addr': '172.21.15.152:6802/2778770069', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7a566b60-1543-41d7-894c-e2fbb63e48dc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6809', 'nonce': 263671335}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6810', 'nonce': 263671335}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6814', 'nonce': 263671335}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6812', 'nonce': 263671335}]}, 'public_addr': '172.21.15.152:6809/263671335', 'cluster_addr': '172.21.15.152:6810/263671335', 'heartbeat_back_addr': '172.21.15.152:6814/263671335', 'heartbeat_front_addr': '172.21.15.152:6812/263671335', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '28cc2358-4ac4-40b2-91a4-20edfe625a73', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6808', 'nonce': 4245639079}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6811', 'nonce': 4245639079}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6815', 'nonce': 4245639079}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6813', 'nonce': 4245639079}]}, 'public_addr': '172.21.15.152:6808/4245639079', 'cluster_addr': '172.21.15.152:6811/4245639079', 'heartbeat_back_addr': '172.21.15.152:6815/4245639079', 'heartbeat_front_addr': '172.21.15.152:6813/4245639079', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '8f79279b-d2d0-438e-b3c2-88df1241ff30', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 25, 'down_at': 38, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6804', 'nonce': 3940055127}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6805', 'nonce': 3940055127}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6807', 'nonce': 3940055127}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6806', 'nonce': 3940055127}]}, 'public_addr': '172.21.15.152:6804/3940055127', 'cluster_addr': '172.21.15.152:6805/3940055127', 'heartbeat_back_addr': '172.21.15.152:6807/3940055127', 'heartbeat_front_addr': '172.21.15.152:6806/3940055127', 'state': ['exists']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:52.171904+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:52.428840+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:51.681954+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:53.687417+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:50.158682+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:50.443807+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:49.750276+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:49.770987+0000', 'dead_epoch': 38}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '2.1', 'osds': [5, 4]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2023-12-06T18:29:04.119 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:29:04.134 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:04.134 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:04.143 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:29:04.161 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:04.156+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:04.254 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:04.251+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:04.312 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:04.313 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:04.321 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:29:04.355 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:04.352+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:04.456 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:04.453+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:04.496 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:04.497 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:04.506 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:29:04.557 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:04.554+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:04.634 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:04.634 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:04.643 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:29:04.686 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:04.681+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:04.719 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:29:04.757 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:29:04.757 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:29:04.759 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:04.756+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:04.769 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2023-12-06T18:29:04.237071+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":17,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220192,"kb_used_data":4024,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754528,"statfs":{"total":773094113280,"available":772868636672,"internally_reserved":0,"allocated":4120576,"data_stored":2195712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.222828"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-06T18:28:58.653285+0000","last_change":"2023-12-06T18:27:04.849231+0000","last_active":"2023-12-06T18:28:58.653285+0000","last_peered":"2023-12-06T18:28:58.653285+0000","last_clean":"2023-12-06T18:28:58.653285+0000","last_became_active":"2023-12-06T18:27:04.837862+0000","last_became_peered":"2023-12-06T18:27:04.837862+0000","last_unstale":"2023-12-06T18:28:58.653285+0000","last_undegraded":"2023-12-06T18:28:58.653285+0000","last_fullsized":"2023-12-06T18:28:58.653285+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:03:55.366586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-12-06T18:28:55.085319+0000","last_change":"2023-12-06T18:27:04.883840+0000","last_active":"2023-12-06T18:28:55.085319+0000","last_peered":"2023-12-06T18:28:55.085319+0000","last_clean":"2023-12-06T18:28:55.085319+0000","last_became_active":"2023-12-06T18:27:04.858001+0000","last_became_peered":"2023-12-06T18:27:04.858001+0000","last_unstale":"2023-12-06T18:28:55.085319+0000","last_undegraded":"2023-12-06T18:28:55.085319+0000","last_fullsized":"2023-12-06T18:28:55.085319+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:44:02.916244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-06T18:28:58.649670+0000","last_change":"2023-12-06T18:27:04.336778+0000","last_active":"2023-12-06T18:28:58.649670+0000","last_peered":"2023-12-06T18:28:58.649670+0000","last_clean":"2023-12-06T18:28:58.649670+0000","last_became_active":"2023-12-06T18:27:04.321269+0000","last_became_peered":"2023-12-06T18:27:04.321269+0000","last_unstale":"2023-12-06T18:28:58.649670+0000","last_undegraded":"2023-12-06T18:28:58.649670+0000","last_fullsized":"2023-12-06T18:28:58.649670+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T22:25:15.703579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-12-06T18:28:55.419072+0000","last_change":"2023-12-06T18:28:47.898016+0000","last_active":"2023-12-06T18:28:55.419072+0000","last_peered":"2023-12-06T18:28:55.419072+0000","last_clean":"2023-12-06T18:28:55.419072+0000","last_became_active":"2023-12-06T18:28:47.891814+0000","last_became_peered":"2023-12-06T18:28:47.891814+0000","last_unstale":"2023-12-06T18:28:55.419072+0000","last_undegraded":"2023-12-06T18:28:55.419072+0000","last_fullsized":"2023-12-06T18:28:55.419072+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T00:49:06.937273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-06T18:28:58.643372+0000","last_change":"2023-12-06T18:27:04.408081+0000","last_active":"2023-12-06T18:28:58.643372+0000","last_peered":"2023-12-06T18:28:58.643372+0000","last_clean":"2023-12-06T18:28:58.643372+0000","last_became_active":"2023-12-06T18:27:04.399765+0000","last_became_peered":"2023-12-06T18:27:04.399765+0000","last_unstale":"2023-12-06T18:28:58.643372+0000","last_undegraded":"2023-12-06T18:28:58.643372+0000","last_fullsized":"2023-12-06T18:28:58.643372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:40:33.570114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":48,"state":"active+clean+remapped","last_fresh":"2023-12-06T18:28:55.421430+0000","last_change":"2023-12-06T18:28:50.416372+0000","last_active":"2023-12-06T18:28:55.421430+0000","last_peered":"2023-12-06T18:28:55.421430+0000","last_clean":"2023-12-06T18:28:55.421430+0000","last_became_active":"2023-12-06T18:28:50.406375+0000","last_became_peered":"2023-12-06T18:28:50.406375+0000","last_unstale":"2023-12-06T18:28:55.421430+0000","last_undegraded":"2023-12-06T18:28:55.421430+0000","last_fullsized":"2023-12-06T18:28:55.421430+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T21:53:06.907341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-06T18:28:58.640667+0000","last_change":"2023-12-06T18:28:47.876556+0000","last_active":"2023-12-06T18:28:58.640667+0000","last_peered":"2023-12-06T18:28:58.640667+0000","last_clean":"2023-12-06T18:28:58.640667+0000","last_became_active":"2023-12-06T18:28:47.865169+0000","last_became_peered":"2023-12-06T18:28:47.865169+0000","last_unstale":"2023-12-06T18:28:58.640667+0000","last_undegraded":"2023-12-06T18:28:58.640667+0000","last_fullsized":"2023-12-06T18:28:58.640667+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:16:40.095542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-06T18:28:58.646580+0000","last_change":"2023-12-06T18:27:04.609381+0000","last_active":"2023-12-06T18:28:58.646580+0000","last_peered":"2023-12-06T18:28:58.646580+0000","last_clean":"2023-12-06T18:28:58.646580+0000","last_became_active":"2023-12-06T18:27:04.598701+0000","last_became_peered":"2023-12-06T18:27:04.598701+0000","last_unstale":"2023-12-06T18:28:58.646580+0000","last_undegraded":"2023-12-06T18:28:58.646580+0000","last_fullsized":"2023-12-06T18:28:58.646580+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:29:58.962692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":171,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-12-06T18:28:55.593959+0000","last_change":"2023-12-06T18:28:48.207134+0000","last_active":"2023-12-06T18:28:55.593959+0000","last_peered":"2023-12-06T18:28:55.593959+0000","last_clean":"2023-12-06T18:28:55.593959+0000","last_became_active":"2023-12-06T18:28:47.875258+0000","last_became_peered":"2023-12-06T18:28:47.875258+0000","last_unstale":"2023-12-06T18:28:55.593959+0000","last_undegraded":"2023-12-06T18:28:55.593959+0000","last_fullsized":"2023-12-06T18:28:55.593959+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:14:32.667039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":15,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1204224,"data_stored":1193520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411356,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":184803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:29:01 2023","interfaces":[{"interface":"back","average":{"1min":8.5950000000000006,"5min":10.083,"15min":10.331},"min":{"1min":5.4009999999999998,"5min":5.4009999999999998,"15min":5.4009999999999998},"max":{"1min":13.397,"5min":17.366,"15min":17.366},"last":11.135999999999999},{"interface":"front","average":{"1min":8.4710000000000001,"5min":11.901999999999999,"15min":12.474},"min":{"1min":3.7200000000000002,"5min":3.7200000000000002,"15min":3.7200000000000002},"max":{"1min":15.451000000000001,"5min":43.768999999999998,"15min":43.768999999999998},"last":10.503}]},{"osd":1,"last update":"Wed Dec 6 18:29:01 2023","interfaces":[{"interface":"back","average":{"1min":10.885,"5min":12.621,"15min":12.911},"min":{"1min":6.6959999999999997,"5min":6.2999999999999998,"15min":6.2999999999999998},"max":{"1min":20.099,"5min":25.812999999999999,"15min":25.812999999999999},"last":20.099},{"interface":"front","average":{"1min":12.037000000000001,"5min":15.355,"15min":15.907999999999999},"min":{"1min":7.0910000000000002,"5min":7.0910000000000002,"15min":7.0910000000000002},"max":{"1min":18.747,"5min":32.920000000000002,"15min":32.920000000000002},"last":8.1969999999999992}]},{"osd":2,"last update":"Wed Dec 6 18:29:01 2023","interfaces":[{"interface":"back","average":{"1min":14.443,"5min":14.679,"15min":14.718},"min":{"1min":7.7690000000000001,"5min":7.5149999999999997,"15min":7.5149999999999997},"max":{"1min":24.486999999999998,"5min":34.072000000000003,"15min":34.072000000000003},"last":9.1820000000000004},{"interface":"front","average":{"1min":15.952999999999999,"5min":17.811,"15min":18.120999999999999},"min":{"1min":10.708,"5min":8.8290000000000006,"15min":8.8290000000000006},"max":{"1min":30.007000000000001,"5min":30.007000000000001,"15min":30.007000000000001},"last":17.811}]},{"osd":4,"last update":"Wed Dec 6 18:29:01 2023","interfaces":[{"interface":"back","average":{"1min":10.893000000000001,"5min":11.661,"15min":11.789},"min":{"1min":5.9400000000000004,"5min":5.9400000000000004,"15min":5.9400000000000004},"max":{"1min":19.504999999999999,"5min":19.504999999999999,"15min":19.504999999999999},"last":6.5650000000000004},{"interface":"front","average":{"1min":14.688000000000001,"5min":15.814,"15min":16.001999999999999},"min":{"1min":7.8049999999999997,"5min":7.8049999999999997,"15min":7.8049999999999997},"max":{"1min":25.126999999999999,"5min":29.542999999999999,"15min":29.542999999999999},"last":13.534000000000001}]},{"osd":5,"last update":"Wed Dec 6 18:29:01 2023","interfaces":[{"interface":"back","average":{"1min":19.265000000000001,"5min":18.018000000000001,"15min":17.809999999999999},"min":{"1min":9.798,"5min":8.3179999999999996,"15min":8.3179999999999996},"max":{"1min":31.305,"5min":32.003,"15min":32.003},"last":17.073},{"interface":"front","average":{"1min":18.405000000000001,"5min":18.344999999999999,"15min":18.335000000000001},"min":{"1min":8.4079999999999995,"5min":8.4079999999999995,"15min":8.4079999999999995},"max":{"1min":32.57,"5min":32.57,"15min":32.57},"last":15.298}]},{"osd":6,"last update":"Wed Dec 6 18:29:01 2023","interfaces":[{"interface":"back","average":{"1min":14.874000000000001,"5min":16.138000000000002,"15min":16.349},"min":{"1min":7.0919999999999996,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":28.469999999999999,"5min":29.245000000000001,"15min":29.245000000000001},"last":14.571},{"interface":"front","average":{"1min":18.356000000000002,"5min":20.949999999999999,"15min":21.382000000000001},"min":{"1min":10.973000000000001,"5min":10.973000000000001,"15min":10.973000000000001},"max":{"1min":27.805,"5min":30.931999999999999,"15min":30.931999999999999},"last":19.445}]},{"osd":7,"last update":"Wed Dec 6 18:27:59 2023","interfaces":[{"interface":"back","average":{"1min":20.715,"5min":20.715,"15min":20.715},"min":{"1min":12.949999999999999,"5min":12.949999999999999,"15min":12.949999999999999},"max":{"1min":34.771000000000001,"5min":34.771000000000001,"15min":34.771000000000001},"last":18.085000000000001},{"interface":"front","average":{"1min":19.077999999999999,"5min":19.077999999999999,"15min":19.077999999999999},"min":{"1min":12.787000000000001,"5min":12.787000000000001,"15min":12.787000000000001},"max":{"1min":32.286000000000001,"5min":32.286000000000001,"15min":32.286000000000001},"last":15.298999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476763,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":180452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:28:58 2023","interfaces":[{"interface":"back","average":{"1min":8.4039999999999999,"5min":10.895,"15min":11.31},"min":{"1min":4.609,"5min":4.609,"15min":4.609},"max":{"1min":16.423999999999999,"5min":38.740000000000002,"15min":38.740000000000002},"last":15.704000000000001},{"interface":"front","average":{"1min":8.5120000000000005,"5min":10.804,"15min":11.186},"min":{"1min":5.4820000000000002,"5min":5.4820000000000002,"15min":5.4820000000000002},"max":{"1min":14.06,"5min":26.106999999999999,"15min":26.106999999999999},"last":8.7460000000000004}]},{"osd":1,"last update":"Wed Dec 6 18:27:59 2023","interfaces":[{"interface":"back","average":{"1min":11.631,"5min":11.631,"15min":11.631},"min":{"1min":6.7240000000000002,"5min":6.7240000000000002,"15min":6.7240000000000002},"max":{"1min":20.989000000000001,"5min":20.989000000000001,"15min":20.989000000000001},"last":12.997999999999999},{"interface":"front","average":{"1min":14.010999999999999,"5min":14.010999999999999,"15min":14.010999999999999},"min":{"1min":8.3290000000000006,"5min":8.3290000000000006,"15min":8.3290000000000006},"max":{"1min":31.561,"5min":31.561,"15min":31.561},"last":11.314}]},{"osd":3,"last update":"Wed Dec 6 18:27:59 2023","interfaces":[{"interface":"back","average":{"1min":19.887,"5min":19.887,"15min":19.887},"min":{"1min":7.0380000000000003,"5min":7.0380000000000003,"15min":7.0380000000000003},"max":{"1min":112.129,"5min":112.129,"15min":112.129},"last":14.01},{"interface":"front","average":{"1min":25.227,"5min":25.227,"15min":25.227},"min":{"1min":7.6559999999999997,"5min":7.6559999999999997,"15min":7.6559999999999997},"max":{"1min":181.601,"5min":181.601,"15min":181.601},"last":19.259}]},{"osd":4,"last update":"Wed Dec 6 18:28:58 2023","interfaces":[{"interface":"back","average":{"1min":12.164999999999999,"5min":17.186,"15min":18.023},"min":{"1min":6.609,"5min":6.609,"15min":6.609},"max":{"1min":16.062000000000001,"5min":97.088999999999999,"15min":97.088999999999999},"last":10.074},{"interface":"front","average":{"1min":13.157999999999999,"5min":20.222000000000001,"15min":21.399999999999999},"min":{"1min":8.9689999999999994,"5min":8.7089999999999996,"15min":8.7089999999999996},"max":{"1min":21.690000000000001,"5min":98.069000000000003,"15min":98.069000000000003},"last":7.7320000000000002}]},{"osd":5,"last update":"Wed Dec 6 18:28:58 2023","interfaces":[{"interface":"back","average":{"1min":16.234000000000002,"5min":17.882000000000001,"15min":18.157},"min":{"1min":7.8170000000000002,"5min":7.8170000000000002,"15min":7.8170000000000002},"max":{"1min":32.639000000000003,"5min":41.192999999999998,"15min":41.192999999999998},"last":18.390999999999998},{"interface":"front","average":{"1min":15.137,"5min":15.699,"15min":15.792999999999999},"min":{"1min":8.843,"5min":8.843,"15min":8.843},"max":{"1min":30.515000000000001,"5min":30.515000000000001,"15min":30.515000000000001},"last":10.74}]},{"osd":6,"last update":"Wed Dec 6 18:28:58 2023","interfaces":[{"interface":"back","average":{"1min":16.670999999999999,"5min":41.350999999999999,"15min":45.463999999999999},"min":{"1min":8.7289999999999992,"5min":8.7289999999999992,"15min":8.7289999999999992},"max":{"1min":29.888999999999999,"5min":498.38499999999999,"15min":498.38499999999999},"last":17.231999999999999},{"interface":"front","average":{"1min":18.648,"5min":43.427,"15min":47.557000000000002},"min":{"1min":12.116,"5min":11.191000000000001,"15min":11.191000000000001},"max":{"1min":31.478000000000002,"5min":496.17399999999998,"15min":496.17399999999998},"last":16.516999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:27:57 2023","interfaces":[{"interface":"back","average":{"1min":27.364999999999998,"5min":27.364999999999998,"15min":27.364999999999998},"min":{"1min":12.321,"5min":12.321,"15min":12.321},"max":{"1min":182.93600000000001,"5min":182.93600000000001,"15min":182.93600000000001},"last":23.16},{"interface":"front","average":{"1min":45.905000000000001,"5min":45.905000000000001,"15min":45.905000000000001},"min":{"1min":13.724,"5min":13.724,"15min":13.724},"max":{"1min":497.67700000000002,"5min":497.67700000000002,"15min":497.67700000000002},"last":20.088000000000001}]}]},{"osd":1,"up_from":16,"seq":68719476764,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":827392,"data_stored":578292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:27:58 2023","interfaces":[{"interface":"back","average":{"1min":24.363,"5min":24.363,"15min":24.363},"min":{"1min":6.5490000000000004,"5min":6.5490000000000004,"15min":6.5490000000000004},"max":{"1min":238.87100000000001,"5min":238.87100000000001,"15min":238.87100000000001},"last":7.492},{"interface":"front","average":{"1min":50.237000000000002,"5min":50.237000000000002,"15min":50.237000000000002},"min":{"1min":6.7949999999999999,"5min":6.7949999999999999,"15min":6.7949999999999999},"max":{"1min":661.33600000000001,"5min":661.33600000000001,"15min":661.33600000000001},"last":8.5239999999999991}]},{"osd":2,"last update":"Wed Dec 6 18:27:58 2023","interfaces":[{"interface":"back","average":{"1min":37.024000000000001,"5min":37.024000000000001,"15min":37.024000000000001},"min":{"1min":9.1790000000000003,"5min":9.1790000000000003,"15min":9.1790000000000003},"max":{"1min":384.315,"5min":384.315,"15min":384.315},"last":14.936},{"interface":"front","average":{"1min":53.218000000000004,"5min":53.218000000000004,"15min":53.218000000000004},"min":{"1min":5.7009999999999996,"5min":5.7009999999999996,"15min":5.7009999999999996},"max":{"1min":656.98699999999997,"5min":656.98699999999997,"15min":656.98699999999997},"last":15.667999999999999}]},{"osd":3,"last update":"Wed Dec 6 18:27:58 2023","interfaces":[{"interface":"back","average":{"1min":13.6,"5min":13.6,"15min":13.6},"min":{"1min":7.4829999999999997,"5min":7.4829999999999997,"15min":7.4829999999999997},"max":{"1min":34.057000000000002,"5min":34.057000000000002,"15min":34.057000000000002},"last":6.6029999999999998},{"interface":"front","average":{"1min":16.722000000000001,"5min":16.722000000000001,"15min":16.722000000000001},"min":{"1min":8.4499999999999993,"5min":8.4499999999999993,"15min":8.4499999999999993},"max":{"1min":35.832999999999998,"5min":35.832999999999998,"15min":35.832999999999998},"last":12.039999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:27:58 2023","interfaces":[{"interface":"back","average":{"1min":46.415999999999997,"5min":46.415999999999997,"15min":46.415999999999997},"min":{"1min":10.843999999999999,"5min":10.843999999999999,"15min":10.843999999999999},"max":{"1min":447.67200000000003,"5min":447.67200000000003,"15min":447.67200000000003},"last":13.413},{"interface":"front","average":{"1min":54.677,"5min":54.677,"15min":54.677},"min":{"1min":7.0919999999999996,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":630.572,"5min":630.572,"15min":630.572},"last":9.9629999999999992}]},{"osd":5,"last update":"Wed Dec 6 18:27:58 2023","interfaces":[{"interface":"back","average":{"1min":48.198,"5min":48.198,"15min":48.198},"min":{"1min":9.6699999999999999,"5min":9.6699999999999999,"15min":9.6699999999999999},"max":{"1min":389.55000000000001,"5min":389.55000000000001,"15min":389.55000000000001},"last":9.2829999999999995},{"interface":"front","average":{"1min":64.882000000000005,"5min":64.882000000000005,"15min":64.882000000000005},"min":{"1min":11.458,"5min":11.458,"15min":11.458},"max":{"1min":658.48599999999999,"5min":658.48599999999999,"15min":658.48599999999999},"last":12.680999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:27:58 2023","interfaces":[{"interface":"back","average":{"1min":40.109000000000002,"5min":40.109000000000002,"15min":40.109000000000002},"min":{"1min":9.2309999999999999,"5min":9.2309999999999999,"15min":9.2309999999999999},"max":{"1min":439.56400000000002,"5min":439.56400000000002,"15min":439.56400000000002},"last":14.052},{"interface":"front","average":{"1min":40.896000000000001,"5min":40.896000000000001,"15min":40.896000000000001},"min":{"1min":11.135999999999999,"5min":11.135999999999999,"15min":11.135999999999999},"max":{"1min":386.71499999999997,"5min":386.71499999999997,"15min":386.71499999999997},"last":11.279}]},{"osd":7,"last update":"Wed Dec 6 18:27:58 2023","interfaces":[{"interface":"back","average":{"1min":43.825000000000003,"5min":43.825000000000003,"15min":43.825000000000003},"min":{"1min":11.693,"5min":11.693,"15min":11.693},"max":{"1min":431.46100000000001,"5min":431.46100000000001,"15min":431.46100000000001},"last":17.163},{"interface":"front","average":{"1min":42.889000000000003,"5min":42.889000000000003,"15min":42.889000000000003},"min":{"1min":11.731999999999999,"5min":11.731999999999999,"15min":11.731999999999999},"max":{"1min":388.52999999999997,"5min":388.52999999999997,"15min":388.52999999999997},"last":18.279}]}]},{"osd":0,"up_from":16,"seq":68719476765,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":438272,"data_stored":184803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 6 18:28:59 2023","interfaces":[{"interface":"back","average":{"1min":9.9359999999999999,"5min":16.824000000000002,"15min":17.972000000000001},"min":{"1min":6.1589999999999998,"5min":6.1589999999999998,"15min":6.1589999999999998},"max":{"1min":40.871000000000002,"5min":116.593,"15min":116.593},"last":10.869999999999999},{"interface":"front","average":{"1min":13.831,"5min":17.888999999999999,"15min":18.565000000000001},"min":{"1min":6.8630000000000004,"5min":5.024,"15min":5.024},"max":{"1min":42.280000000000001,"5min":117.896,"15min":117.896},"last":19.486000000000001}]},{"osd":2,"last update":"Wed Dec 6 18:28:59 2023","interfaces":[{"interface":"back","average":{"1min":17.062000000000001,"5min":14.122999999999999,"15min":13.632999999999999},"min":{"1min":6.0860000000000003,"5min":6.0860000000000003,"15min":6.0860000000000003},"max":{"1min":43.752000000000002,"5min":43.752000000000002,"15min":43.752000000000002},"last":12.603},{"interface":"front","average":{"1min":15.958,"5min":11.970000000000001,"15min":11.305},"min":{"1min":7.6390000000000002,"5min":7.2489999999999997,"15min":7.2489999999999997},"max":{"1min":46.387999999999998,"5min":46.387999999999998,"15min":46.387999999999998},"last":12.893000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:28:00 2023","interfaces":[{"interface":"back","average":{"1min":16.844999999999999,"5min":16.844999999999999,"15min":16.844999999999999},"min":{"1min":8.7449999999999992,"5min":8.7449999999999992,"15min":8.7449999999999992},"max":{"1min":46.689999999999998,"5min":46.689999999999998,"15min":46.689999999999998},"last":18.882999999999999},{"interface":"front","average":{"1min":16.280000000000001,"5min":16.280000000000001,"15min":16.280000000000001},"min":{"1min":8.0329999999999995,"5min":8.0329999999999995,"15min":8.0329999999999995},"max":{"1min":26.748000000000001,"5min":26.748000000000001,"15min":26.748000000000001},"last":9.6440000000000001}]},{"osd":4,"last update":"Wed Dec 6 18:28:59 2023","interfaces":[{"interface":"back","average":{"1min":15.266,"5min":20.158000000000001,"15min":20.972999999999999},"min":{"1min":7.2069999999999999,"5min":7.2069999999999999,"15min":7.2069999999999999},"max":{"1min":43.920999999999999,"5min":82.432000000000002,"15min":82.432000000000002},"last":7.3920000000000003},{"interface":"front","average":{"1min":18.812000000000001,"5min":20.405999999999999,"15min":20.672000000000001},"min":{"1min":9.5239999999999991,"5min":6.2629999999999999,"15min":6.2629999999999999},"max":{"1min":61.402999999999999,"5min":81.239999999999995,"15min":81.239999999999995},"last":15.817}]},{"osd":5,"last update":"Wed Dec 6 18:28:59 2023","interfaces":[{"interface":"back","average":{"1min":19.922999999999998,"5min":18.09,"15min":17.783999999999999},"min":{"1min":9.343,"5min":8.6479999999999997,"15min":8.6479999999999997},"max":{"1min":71.367000000000004,"5min":71.367000000000004,"15min":71.367000000000004},"last":16.832999999999998},{"interface":"front","average":{"1min":20.561,"5min":18.675000000000001,"15min":18.361000000000001},"min":{"1min":9.859,"5min":9.4440000000000008,"15min":9.4440000000000008},"max":{"1min":72.307000000000002,"5min":72.307000000000002,"15min":72.307000000000002},"last":10.286}]},{"osd":6,"last update":"Wed Dec 6 18:28:59 2023","interfaces":[{"interface":"back","average":{"1min":20.942,"5min":19.393999999999998,"15min":19.135999999999999},"min":{"1min":8.9179999999999993,"5min":7.7560000000000002,"15min":7.7560000000000002},"max":{"1min":62.932000000000002,"5min":62.932000000000002,"15min":62.932000000000002},"last":18.023},{"interface":"front","average":{"1min":18.109000000000002,"5min":19.257999999999999,"15min":19.449999999999999},"min":{"1min":12.183,"5min":10.116,"15min":10.116},"max":{"1min":43.088000000000001,"5min":52.627000000000002,"15min":52.627000000000002},"last":20.582000000000001}]},{"osd":7,"last update":"Wed Dec 6 18:27:55 2023","interfaces":[{"interface":"back","average":{"1min":21.66,"5min":21.66,"15min":21.66},"min":{"1min":12.804,"5min":12.804,"15min":12.804},"max":{"1min":48.781999999999996,"5min":48.781999999999996,"15min":48.781999999999996},"last":17.870999999999999},{"interface":"front","average":{"1min":19.440000000000001,"5min":19.440000000000001,"15min":19.440000000000001},"min":{"1min":10.537000000000001,"5min":10.537000000000001,"15min":10.537000000000001},"max":{"1min":38.530000000000001,"5min":38.530000000000001,"15min":38.530000000000001},"last":11.573}]}]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":578292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:27:55 2023","interfaces":[{"interface":"back","average":{"1min":16.562000000000001,"5min":16.562000000000001,"15min":16.562000000000001},"min":{"1min":4.4219999999999997,"5min":4.4219999999999997,"15min":4.4219999999999997},"max":{"1min":208.291,"5min":208.291,"15min":208.291},"last":22.923999999999999},{"interface":"front","average":{"1min":20.984999999999999,"5min":20.984999999999999,"15min":20.984999999999999},"min":{"1min":6.9509999999999996,"5min":6.9509999999999996,"15min":6.9509999999999996},"max":{"1min":199.94900000000001,"5min":199.94900000000001,"15min":199.94900000000001},"last":13.337999999999999}]},{"osd":1,"last update":"Wed Dec 6 18:27:55 2023","interfaces":[{"interface":"back","average":{"1min":66.596999999999994,"5min":66.596999999999994,"15min":66.596999999999994},"min":{"1min":8.1180000000000003,"5min":8.1180000000000003,"15min":8.1180000000000003},"max":{"1min":1211.6010000000001,"5min":1211.6010000000001,"15min":1211.6010000000001},"last":14.102},{"interface":"front","average":{"1min":61.418999999999997,"5min":61.418999999999997,"15min":61.418999999999997},"min":{"1min":5.1340000000000003,"5min":5.1340000000000003,"15min":5.1340000000000003},"max":{"1min":1189.3869999999999,"5min":1189.3869999999999,"15min":1189.3869999999999},"last":6.6299999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:27:55 2023","interfaces":[{"interface":"back","average":{"1min":26.553000000000001,"5min":26.553000000000001,"15min":26.553000000000001},"min":{"1min":7.7729999999999997,"5min":7.7729999999999997,"15min":7.7729999999999997},"max":{"1min":283.46800000000002,"5min":283.46800000000002,"15min":283.46800000000002},"last":11.481999999999999},{"interface":"front","average":{"1min":30.622,"5min":30.622,"15min":30.622},"min":{"1min":8.9350000000000005,"5min":8.9350000000000005,"15min":8.9350000000000005},"max":{"1min":296.47800000000001,"5min":296.47800000000001,"15min":296.47800000000001},"last":14.710000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:27:59 2023","interfaces":[{"interface":"back","average":{"1min":19.622,"5min":19.622,"15min":19.622},"min":{"1min":13.295,"5min":13.295,"15min":13.295},"max":{"1min":47.926000000000002,"5min":47.926000000000002,"15min":47.926000000000002},"last":12.121},{"interface":"front","average":{"1min":15.702999999999999,"5min":15.702999999999999,"15min":15.702999999999999},"min":{"1min":8.1739999999999995,"5min":8.1739999999999995,"15min":8.1739999999999995},"max":{"1min":66.176000000000002,"5min":66.176000000000002,"15min":66.176000000000002},"last":7.5759999999999996}]},{"osd":4,"last update":"Wed Dec 6 18:27:55 2023","interfaces":[{"interface":"back","average":{"1min":20.242999999999999,"5min":20.242999999999999,"15min":20.242999999999999},"min":{"1min":11.977,"5min":11.977,"15min":11.977},"max":{"1min":67.248000000000005,"5min":67.248000000000005,"15min":67.248000000000005},"last":12.733000000000001},{"interface":"front","average":{"1min":13.545999999999999,"5min":13.545999999999999,"15min":13.545999999999999},"min":{"1min":6.2000000000000002,"5min":6.2000000000000002,"15min":6.2000000000000002},"max":{"1min":77.266999999999996,"5min":77.266999999999996,"15min":77.266999999999996},"last":9.548}]},{"osd":6,"last update":"Wed Dec 6 18:27:55 2023","interfaces":[{"interface":"back","average":{"1min":16.41,"5min":16.41,"15min":16.41},"min":{"1min":7.069,"5min":7.069,"15min":7.069},"max":{"1min":76.751999999999995,"5min":76.751999999999995,"15min":76.751999999999995},"last":5.915},{"interface":"front","average":{"1min":21.501999999999999,"5min":21.501999999999999,"15min":21.501999999999999},"min":{"1min":12.782,"5min":12.782,"15min":12.782},"max":{"1min":57.101999999999997,"5min":57.101999999999997,"15min":57.101999999999997},"last":8.8390000000000004}]},{"osd":7,"last update":"Wed Dec 6 18:27:55 2023","interfaces":[{"interface":"back","average":{"1min":16.204000000000001,"5min":16.204000000000001,"15min":16.204000000000001},"min":{"1min":7.367,"5min":7.367,"15min":7.367},"max":{"1min":70.768000000000001,"5min":70.768000000000001,"15min":70.768000000000001},"last":10.436},{"interface":"front","average":{"1min":18.081,"5min":18.081,"15min":18.081},"min":{"1min":6.3150000000000004,"5min":6.3150000000000004,"15min":6.3150000000000004},"max":{"1min":63.991999999999997,"5min":63.991999999999997,"15min":63.991999999999997},"last":12.419}]}]},{"osd":4,"up_from":15,"seq":64424509470,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:28:59 2023","interfaces":[{"interface":"back","average":{"1min":10.362,"5min":13.427,"15min":13.938000000000001},"min":{"1min":4.1479999999999997,"5min":4.1479999999999997,"15min":4.1479999999999997},"max":{"1min":25.327999999999999,"5min":53.298999999999999,"15min":53.298999999999999},"last":8.093},{"interface":"front","average":{"1min":9.5570000000000004,"5min":10.763999999999999,"15min":10.965},"min":{"1min":5.1150000000000002,"5min":3.9670000000000001,"15min":3.9670000000000001},"max":{"1min":23.462,"5min":52.567,"15min":52.567},"last":5.298}]},{"osd":1,"last update":"Wed Dec 6 18:28:59 2023","interfaces":[{"interface":"back","average":{"1min":12.065,"5min":13.991,"15min":14.311999999999999},"min":{"1min":5.8499999999999996,"5min":5.835,"15min":5.835},"max":{"1min":27.251999999999999,"5min":49.183,"15min":49.183},"last":11.066000000000001},{"interface":"front","average":{"1min":13.124000000000001,"5min":15.617000000000001,"15min":16.033000000000001},"min":{"1min":5.96,"5min":5.96,"15min":5.96},"max":{"1min":26.495999999999999,"5min":45.412999999999997,"15min":45.412999999999997},"last":10.35}]},{"osd":2,"last update":"Wed Dec 6 18:28:59 2023","interfaces":[{"interface":"back","average":{"1min":11.475,"5min":21.600000000000001,"15min":23.288},"min":{"1min":5.6390000000000002,"5min":5.6390000000000002,"15min":5.6390000000000002},"max":{"1min":20.661999999999999,"5min":85.394000000000005,"15min":85.394000000000005},"last":19.077000000000002},{"interface":"front","average":{"1min":13.08,"5min":19.946000000000002,"15min":21.09},"min":{"1min":6.3639999999999999,"5min":6.3639999999999999,"15min":6.3639999999999999},"max":{"1min":21.678000000000001,"5min":69.530000000000001,"15min":69.530000000000001},"last":20.628}]},{"osd":3,"last update":"Wed Dec 6 18:27:59 2023","interfaces":[{"interface":"back","average":{"1min":14.401,"5min":14.401,"15min":14.401},"min":{"1min":7.008,"5min":7.008,"15min":7.008},"max":{"1min":25.146000000000001,"5min":25.146000000000001,"15min":25.146000000000001},"last":15.66},{"interface":"front","average":{"1min":15.581,"5min":15.581,"15min":15.581},"min":{"1min":7.7409999999999997,"5min":7.7409999999999997,"15min":7.7409999999999997},"max":{"1min":28.571000000000002,"5min":28.571000000000002,"15min":28.571000000000002},"last":14.815}]},{"osd":5,"last update":"Wed Dec 6 18:28:59 2023","interfaces":[{"interface":"back","average":{"1min":14.589,"5min":17.058,"15min":17.469999999999999},"min":{"1min":6.125,"5min":6.125,"15min":6.125},"max":{"1min":27.356999999999999,"5min":78.798000000000002,"15min":78.798000000000002},"last":18.263000000000002},{"interface":"front","average":{"1min":16.584,"5min":18.638999999999999,"15min":18.981000000000002},"min":{"1min":10.956,"5min":6.4450000000000003,"15min":6.4450000000000003},"max":{"1min":28.533999999999999,"5min":55.814999999999998,"15min":55.814999999999998},"last":11.795}]},{"osd":6,"last update":"Wed Dec 6 18:28:59 2023","interfaces":[{"interface":"back","average":{"1min":16.974,"5min":17.899999999999999,"15min":18.053999999999998},"min":{"1min":8.2300000000000004,"5min":7.7850000000000001,"15min":7.7850000000000001},"max":{"1min":51.149000000000001,"5min":51.149000000000001,"15min":51.149000000000001},"last":16.349},{"interface":"front","average":{"1min":14.340999999999999,"5min":16.411999999999999,"15min":16.757000000000001},"min":{"1min":8.5139999999999993,"5min":7.8700000000000001,"15min":7.8700000000000001},"max":{"1min":40.426000000000002,"5min":40.426000000000002,"15min":40.426000000000002},"last":12.564}]},{"osd":7,"last update":"Wed Dec 6 18:27:57 2023","interfaces":[{"interface":"back","average":{"1min":18.751999999999999,"5min":18.751999999999999,"15min":18.751999999999999},"min":{"1min":11.249000000000001,"5min":11.249000000000001,"15min":11.249000000000001},"max":{"1min":33.067999999999998,"5min":33.067999999999998,"15min":33.067999999999998},"last":13.907},{"interface":"front","average":{"1min":17.401,"5min":17.401,"15min":17.401},"min":{"1min":7.6050000000000004,"5min":7.6050000000000004,"15min":7.6050000000000004},"max":{"1min":32.356999999999999,"5min":32.356999999999999,"15min":32.356999999999999},"last":10.754}]}]},{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":128166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:27:57 2023","interfaces":[{"interface":"back","average":{"1min":26.312999999999999,"5min":26.312999999999999,"15min":26.312999999999999},"min":{"1min":3.6179999999999999,"5min":3.6179999999999999,"15min":3.6179999999999999},"max":{"1min":198.46700000000001,"5min":198.46700000000001,"15min":198.46700000000001},"last":7.0339999999999998},{"interface":"front","average":{"1min":26.082000000000001,"5min":26.082000000000001,"15min":26.082000000000001},"min":{"1min":4.5910000000000002,"5min":4.5910000000000002,"15min":4.5910000000000002},"max":{"1min":196.93000000000001,"5min":196.93000000000001,"15min":196.93000000000001},"last":8.1379999999999999}]},{"osd":1,"last update":"Wed Dec 6 18:27:57 2023","interfaces":[{"interface":"back","average":{"1min":29.395,"5min":29.395,"15min":29.395},"min":{"1min":6.6520000000000001,"5min":6.6520000000000001,"15min":6.6520000000000001},"max":{"1min":307.00200000000001,"5min":307.00200000000001,"15min":307.00200000000001},"last":12.029999999999999},{"interface":"front","average":{"1min":30.045999999999999,"5min":30.045999999999999,"15min":30.045999999999999},"min":{"1min":6.5919999999999996,"5min":6.5919999999999996,"15min":6.5919999999999996},"max":{"1min":274.411,"5min":274.411,"15min":274.411},"last":9.548}]},{"osd":2,"last update":"Wed Dec 6 18:27:57 2023","interfaces":[{"interface":"back","average":{"1min":20.195,"5min":20.195,"15min":20.195},"min":{"1min":6.3840000000000003,"5min":6.3840000000000003,"15min":6.3840000000000003},"max":{"1min":69.350999999999999,"5min":69.350999999999999,"15min":69.350999999999999},"last":11.413},{"interface":"front","average":{"1min":18.774999999999999,"5min":18.774999999999999,"15min":18.774999999999999},"min":{"1min":5.7110000000000003,"5min":5.7110000000000003,"15min":5.7110000000000003},"max":{"1min":61.414999999999999,"5min":61.414999999999999,"15min":61.414999999999999},"last":15.096}]},{"osd":3,"last update":"Wed Dec 6 18:27:57 2023","interfaces":[{"interface":"back","average":{"1min":19.164999999999999,"5min":19.164999999999999,"15min":19.164999999999999},"min":{"1min":7.9969999999999999,"5min":7.9969999999999999,"15min":7.9969999999999999},"max":{"1min":73.280000000000001,"5min":73.280000000000001,"15min":73.280000000000001},"last":13.268000000000001},{"interface":"front","average":{"1min":16.798999999999999,"5min":16.798999999999999,"15min":16.798999999999999},"min":{"1min":9.7789999999999999,"5min":9.7789999999999999,"15min":9.7789999999999999},"max":{"1min":51.746000000000002,"5min":51.746000000000002,"15min":51.746000000000002},"last":8.9740000000000002}]},{"osd":4,"last update":"Wed Dec 6 18:27:57 2023","interfaces":[{"interface":"back","average":{"1min":19.742000000000001,"5min":19.742000000000001,"15min":19.742000000000001},"min":{"1min":6.282,"5min":6.282,"15min":6.282},"max":{"1min":74.382999999999996,"5min":74.382999999999996,"15min":74.382999999999996},"last":10.800000000000001},{"interface":"front","average":{"1min":18.751000000000001,"5min":18.751000000000001,"15min":18.751000000000001},"min":{"1min":6.7050000000000001,"5min":6.7050000000000001,"15min":6.7050000000000001},"max":{"1min":70.510000000000005,"5min":70.510000000000005,"15min":70.510000000000005},"last":13.975}]},{"osd":5,"last update":"Wed Dec 6 18:27:57 2023","interfaces":[{"interface":"back","average":{"1min":25.561,"5min":25.561,"15min":25.561},"min":{"1min":7.6790000000000003,"5min":7.6790000000000003,"15min":7.6790000000000003},"max":{"1min":182.82400000000001,"5min":182.82400000000001,"15min":182.82400000000001},"last":17.507999999999999},{"interface":"front","average":{"1min":28.684000000000001,"5min":28.684000000000001,"15min":28.684000000000001},"min":{"1min":7.8490000000000002,"5min":7.8490000000000002,"15min":7.8490000000000002},"max":{"1min":181.136,"5min":181.136,"15min":181.136},"last":15.709}]},{"osd":6,"last update":"Wed Dec 6 18:27:52 2023","interfaces":[{"interface":"back","average":{"1min":21.277999999999999,"5min":21.277999999999999,"15min":21.277999999999999},"min":{"1min":10.207000000000001,"5min":10.207000000000001,"15min":10.207000000000001},"max":{"1min":58.863999999999997,"5min":58.863999999999997,"15min":58.863999999999997},"last":18.300000000000001},{"interface":"front","average":{"1min":25.216999999999999,"5min":25.216999999999999,"15min":25.216999999999999},"min":{"1min":12.420999999999999,"5min":12.420999999999999,"15min":12.420999999999999},"max":{"1min":88.512,"5min":88.512,"15min":88.512},"last":16.683}]}]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27448,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":438272,"data_stored":184803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:28:57 2023","interfaces":[{"interface":"back","average":{"1min":9.5090000000000003,"5min":9.9730000000000008,"15min":10.051},"min":{"1min":3.8519999999999999,"5min":3.8519999999999999,"15min":3.8519999999999999},"max":{"1min":46.947000000000003,"5min":46.947000000000003,"15min":46.947000000000003},"last":6.1520000000000001},{"interface":"front","average":{"1min":13.398999999999999,"5min":10.704000000000001,"15min":10.255000000000001},"min":{"1min":5.0650000000000004,"5min":4.2569999999999997,"15min":4.2569999999999997},"max":{"1min":45.953000000000003,"5min":45.953000000000003,"15min":45.953000000000003},"last":9.8889999999999993}]},{"osd":1,"last update":"Wed Dec 6 18:28:57 2023","interfaces":[{"interface":"back","average":{"1min":9.7799999999999994,"5min":12.332000000000001,"15min":12.757999999999999},"min":{"1min":4.8609999999999998,"5min":4.8609999999999998,"15min":4.8609999999999998},"max":{"1min":36.68,"5min":37.984000000000002,"15min":37.984000000000002},"last":5.0220000000000002},{"interface":"front","average":{"1min":12.550000000000001,"5min":14.35,"15min":14.65},"min":{"1min":5.8540000000000001,"5min":5.8540000000000001,"15min":5.8540000000000001},"max":{"1min":37.831000000000003,"5min":40.420999999999999,"15min":40.420999999999999},"last":10.954000000000001}]},{"osd":2,"last update":"Wed Dec 6 18:28:56 2023","interfaces":[{"interface":"back","average":{"1min":11.728999999999999,"5min":19.626999999999999,"15min":20.943000000000001},"min":{"1min":6.7599999999999998,"5min":6.7599999999999998,"15min":6.7599999999999998},"max":{"1min":23.186,"5min":200.88499999999999,"15min":200.88499999999999},"last":8.8070000000000004},{"interface":"front","average":{"1min":15.923999999999999,"5min":16.411999999999999,"15min":16.494},"min":{"1min":7.4119999999999999,"5min":7.4119999999999999,"15min":7.4119999999999999},"max":{"1min":48.005000000000003,"5min":48.005000000000003,"15min":48.005000000000003},"last":12.56}]},{"osd":3,"last update":"Wed Dec 6 18:29:00 2023","interfaces":[{"interface":"back","average":{"1min":11.968,"5min":13.628,"15min":13.904999999999999},"min":{"1min":7.8970000000000002,"5min":7.4290000000000003,"15min":7.4290000000000003},"max":{"1min":20.722000000000001,"5min":25.190999999999999,"15min":25.190999999999999},"last":7.5819999999999999},{"interface":"front","average":{"1min":16.404,"5min":16.369,"15min":16.363},"min":{"1min":7.6079999999999997,"5min":7.6079999999999997,"15min":7.6079999999999997},"max":{"1min":47.006999999999998,"5min":47.006999999999998,"15min":47.006999999999998},"last":14.036}]},{"osd":4,"last update":"Wed Dec 6 18:28:57 2023","interfaces":[{"interface":"back","average":{"1min":13.778,"5min":15.015000000000001,"15min":15.221},"min":{"1min":7.7169999999999996,"5min":7.0270000000000001,"15min":7.0270000000000001},"max":{"1min":27.632000000000001,"5min":31.193999999999999,"15min":31.193999999999999},"last":11.753},{"interface":"front","average":{"1min":17.359999999999999,"5min":16.782,"15min":16.686},"min":{"1min":8.8889999999999993,"5min":7.7169999999999996,"15min":7.7169999999999996},"max":{"1min":48.988999999999997,"5min":48.988999999999997,"15min":48.988999999999997},"last":14.866}]},{"osd":5,"last update":"Wed Dec 6 18:28:57 2023","interfaces":[{"interface":"back","average":{"1min":18.762,"5min":18.515000000000001,"15min":18.474},"min":{"1min":8.2579999999999991,"5min":8.2579999999999991,"15min":8.2579999999999991},"max":{"1min":57.308,"5min":57.308,"15min":57.308},"last":21.581},{"interface":"front","average":{"1min":10.728,"5min":14.436999999999999,"15min":15.055},"min":{"1min":6.0129999999999999,"5min":6.0129999999999999,"15min":6.0129999999999999},"max":{"1min":38.718000000000004,"5min":68.463999999999999,"15min":68.463999999999999},"last":23.334}]},{"osd":7,"last update":"Wed Dec 6 18:27:54 2023","interfaces":[{"interface":"back","average":{"1min":17.899000000000001,"5min":17.899000000000001,"15min":17.899000000000001},"min":{"1min":10.449999999999999,"5min":10.449999999999999,"15min":10.449999999999999},"max":{"1min":45.374000000000002,"5min":45.374000000000002,"15min":45.374000000000002},"last":11.385999999999999},{"interface":"front","average":{"1min":14.888999999999999,"5min":14.888999999999999,"15min":14.888999999999999},"min":{"1min":8.7219999999999995,"5min":8.7219999999999995,"15min":8.7219999999999995},"max":{"1min":38.709000000000003,"5min":38.709000000000003,"15min":38.709000000000003},"last":13.305999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:29:04.772 INFO:tasks.thrashosds.thrasher:Setting 2.7 to [5, 3] 2023-12-06T18:29:04.772 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.7', '5', '3'] 2023-12-06T18:29:04.772 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.7 5 3 2023-12-06T18:29:04.810 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:04.810 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:04.819 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:29:04.863 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:04.859+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:04.946 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:04.946 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:04.955 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:29:04.998 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:04.993+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:05.062 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:05.058+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:05.125 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:05.125 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:05.134 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:29:05.152 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-12-06T18:29:05.197 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:05.193+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:05.263 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:05.343 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:05.344 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:05.260+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:05.345 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:29:05.389 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:05.385+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:05.465 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:05.462+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:05.500 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":20,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:29:05.528 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:05.529 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:05.537 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:29:05.579 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:05.575+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:05.666 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:05.663+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:05.728 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:05.728 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:05.738 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:29:05.751 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:29:05.779 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:05.775+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:05.868 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:05.865+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:05.888 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:51.820 98294== Warning: unimplemented fcntl command: 1036 2023-12-06T18:29:05.890 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:51.822 98294== Warning: unimplemented fcntl command: 1036 2023-12-06T18:29:05.918 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:51.850 98294== Warning: unimplemented fcntl command: 1036 2023-12-06T18:29:05.938 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:05.938 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:05.938 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:22.982 95127== Warning: unimplemented fcntl command: 1036 2023-12-06T18:29:05.947 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:29:05.948 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:22.992 95127== Warning: unimplemented fcntl command: 1036 2023-12-06T18:29:05.974 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:23.019 95127== Warning: unimplemented fcntl command: 1036 2023-12-06T18:29:05.975 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:05.972+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:06.070 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:06.066+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:06.136 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:06.137 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:06.147 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:29:06.172 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:06.168+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:06.246 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:29:06.246 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:29:06.275 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:06.271+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:06.362 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:29:06.362 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:29:06.363 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:06.363 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:06.363 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:06.363 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:06.363 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:06.373 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:29:06.374 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:06.370+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:06.454 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:29:06.455 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:29:06.474 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:06.471+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:06.548 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:29:06.548 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:29:06.575 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:06.571+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:06.612 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:06.612 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:06.612 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:06.612 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:06.612 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.612 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:29:06.612 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:06.613 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 131.19953143999999, 2023-12-06T18:29:06.613 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:29:06.613 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.613 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.613 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.613 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.613 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.613 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:29:06.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.613 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.613 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.613 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.613 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:06.613 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.613 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.614 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:06.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.614 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:29:06.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:29:06.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.614 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:29:06.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:29:06.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.615 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:29:06.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:29:06.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.615 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:29:06.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:29:06.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.615 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:29:06.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:29:06.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:29:06.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:29:06.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:29:06.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:29:06.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:29:06.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:29:06.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:29:06.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:29:06.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:29:06.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:29:06.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:29:06.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:29:06.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:29:06.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:29:06.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:29:06.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:29:06.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:29:06.618 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.618 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.618 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.618 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:06.618 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:06.618 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 129.09095820100001, 2023-12-06T18:29:06.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:29:06.619 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.619 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.619 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.619 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.619 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.619 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:29:06.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.619 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.619 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.619 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:06.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.619 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.620 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.620 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:06.620 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.620 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.620 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.620 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.620 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:29:06.620 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:29:06.620 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.620 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.620 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.620 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:29:06.620 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:29:06.620 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.621 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.621 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.621 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:29:06.621 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:29:06.621 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.621 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.621 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.621 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:29:06.621 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:29:06.621 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.621 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.621 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.621 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:29:06.621 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:29:06.621 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.622 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.622 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.622 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:29:06.622 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:29:06.622 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.622 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.622 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.622 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:29:06.622 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:29:06.622 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.622 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.622 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.622 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:29:06.622 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:29:06.622 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.623 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.623 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.623 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:29:06.623 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:29:06.623 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.623 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.623 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.623 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:29:06.623 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:29:06.623 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.623 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.623 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.623 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:29:06.623 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:29:06.623 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.624 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.624 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.624 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.624 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.624 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:06.624 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:06.624 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 129.09021296899999, 2023-12-06T18:29:06.624 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:29:06.624 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.624 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.624 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.624 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.624 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.625 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:29:06.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.625 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:06.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:06.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:29:06.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:29:06.626 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.626 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.626 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.626 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:29:06.626 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:29:06.626 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.626 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.626 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.626 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:29:06.626 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:29:06.626 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.626 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.626 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.626 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:29:06.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:29:06.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.627 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:29:06.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:29:06.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.627 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:29:06.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:29:06.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:29:06.628 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:29:06.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.628 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.628 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:29:06.628 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:29:06.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.628 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.628 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:29:06.628 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:29:06.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.628 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.629 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.629 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:29:06.629 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:29:06.629 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.629 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.629 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.629 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:29:06.629 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:29:06.629 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.629 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.629 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.629 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.629 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.629 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:06.629 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:06.629 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 129.08952921400001, 2023-12-06T18:29:06.630 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:29:06.630 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.630 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.630 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.630 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.630 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.630 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:29:06.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.630 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.630 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.630 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.630 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:06.630 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.631 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.631 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.631 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:06.631 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.631 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.631 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.631 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:29:06.631 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:29:06.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.631 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.631 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.631 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:29:06.631 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:29:06.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.631 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.632 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.632 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:29:06.632 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:29:06.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.632 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.632 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:29:06.632 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:29:06.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.632 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.632 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:29:06.632 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:29:06.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.633 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.633 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:29:06.633 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:29:06.633 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.633 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.633 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.633 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:29:06.633 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:29:06.633 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.633 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.633 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.633 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:29:06.633 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:29:06.633 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.633 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.633 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.634 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:29:06.634 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:29:06.634 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.634 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.634 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.634 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:29:06.634 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:29:06.634 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.634 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.634 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.634 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:29:06.634 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:29:06.634 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.634 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.634 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.635 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.635 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.635 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:06.635 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:06.635 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 129.08395595900001, 2023-12-06T18:29:06.635 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:29:06.635 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.635 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.635 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.635 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.635 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.635 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:29:06.635 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.635 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.635 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.635 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:06.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:06.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:29:06.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:29:06.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:29:06.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:29:06.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:29:06.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:29:06.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:29:06.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:29:06.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:29:06.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:29:06.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:29:06.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:29:06.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:29:06.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:29:06.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:29:06.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:29:06.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:29:06.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:29:06.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:29:06.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:29:06.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:29:06.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:29:06.640 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.640 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.640 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.640 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:06.640 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:06.640 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 129.08328231900001, 2023-12-06T18:29:06.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:29:06.640 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.640 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.640 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.640 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.640 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.641 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:29:06.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.641 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:06.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:06.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:29:06.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:29:06.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.642 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.642 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.642 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:29:06.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:29:06.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.642 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.642 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.642 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:29:06.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:29:06.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.642 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.642 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.642 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:29:06.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:29:06.643 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.643 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.643 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.643 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:29:06.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:29:06.643 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.643 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.643 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.643 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:29:06.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:29:06.643 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.643 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.643 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.643 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:29:06.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:29:06.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:29:06.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:29:06.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:29:06.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:29:06.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:29:06.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:29:06.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.645 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.645 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:29:06.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:29:06.645 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.645 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.645 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.645 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:06.645 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:06.645 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 129.08262012, 2023-12-06T18:29:06.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:29:06.645 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.645 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.645 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.645 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.645 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.646 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:29:06.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.646 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:06.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:06.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.647 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:29:06.647 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:29:06.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.647 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:29:06.647 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:29:06.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.647 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:29:06.647 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:29:06.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.648 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:29:06.648 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:29:06.648 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.648 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.648 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.648 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:29:06.648 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:29:06.648 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.648 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.648 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.648 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:29:06.648 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:29:06.648 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.648 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.648 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.649 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:29:06.649 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:29:06.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.649 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.649 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.649 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:29:06.649 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:29:06.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.649 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.649 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.649 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:29:06.649 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:29:06.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.649 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.649 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.650 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:29:06.650 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:29:06.650 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.650 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.650 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:29:06.650 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:29:06.650 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.650 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.650 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.650 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.650 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:06.650 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:06.651 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 129.00991068499999, 2023-12-06T18:29:06.651 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:29:06.651 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.651 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.651 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.651 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.651 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.651 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:29:06.651 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.651 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.651 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.651 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.651 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:06.651 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.652 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:06.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.652 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:29:06.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:29:06.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.652 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:29:06.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:29:06.653 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.653 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.653 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.653 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:29:06.653 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:29:06.653 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.653 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.653 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.653 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:29:06.653 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:29:06.653 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.653 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.653 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.653 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:29:06.653 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:29:06.653 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.654 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.654 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.654 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:29:06.654 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:29:06.654 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.654 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.654 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.654 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:29:06.654 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:29:06.654 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.654 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.654 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.654 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:29:06.654 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:29:06.655 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.655 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.655 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.655 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:29:06.655 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:29:06.655 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.655 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.655 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.655 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:29:06.655 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:29:06.655 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.655 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.655 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.655 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:29:06.655 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:29:06.656 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.656 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.656 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.656 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.656 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.656 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:06.656 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:06.656 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 129.009228976, 2023-12-06T18:29:06.656 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:29:06.656 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.656 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.656 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.656 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.656 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.656 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:29:06.656 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.657 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.657 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.657 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.657 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:06.657 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.657 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.657 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.657 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.657 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:06.657 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.657 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.657 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.657 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.657 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:29:06.657 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:29:06.657 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.658 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:29:06.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:29:06.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.658 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:29:06.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:29:06.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.658 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:29:06.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:29:06.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.659 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:29:06.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:29:06.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.659 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:29:06.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:29:06.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.659 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:29:06.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:29:06.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.660 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.660 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:29:06.660 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:29:06.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.660 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.660 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:29:06.660 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:29:06.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.660 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.660 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:29:06.660 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:29:06.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.661 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.661 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:29:06.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:29:06.661 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.661 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.661 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.661 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.661 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.661 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:06.661 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:06.661 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 127.930424888, 2023-12-06T18:29:06.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:29:06.661 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.661 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.662 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.662 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.662 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.662 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:29:06.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.662 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.662 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.662 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.662 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:06.662 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.662 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.662 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.662 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:06.663 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.663 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.663 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.663 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.663 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:29:06.663 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:29:06.663 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.663 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.663 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.663 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:29:06.663 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:29:06.663 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.663 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.663 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.664 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:29:06.664 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:29:06.664 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.664 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.664 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.664 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:29:06.664 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:29:06.664 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.664 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.664 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.664 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:29:06.664 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:29:06.664 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.664 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.665 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.665 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:29:06.665 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:29:06.665 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.665 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.665 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.665 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:29:06.665 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:29:06.665 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.665 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.665 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.665 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:29:06.665 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:29:06.665 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.665 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.666 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.666 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:29:06.666 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:29:06.666 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.666 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.666 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.666 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:29:06.666 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:29:06.666 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.666 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.666 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.666 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:29:06.666 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:29:06.666 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.667 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.667 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.667 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.667 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:06.667 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:06.667 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 127.929008314, 2023-12-06T18:29:06.667 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:29:06.667 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.667 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.667 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.667 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.667 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.667 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:29:06.668 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.668 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.668 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.668 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:06.668 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.668 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.668 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.668 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:06.668 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.668 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.668 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.669 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:29:06.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:29:06.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.669 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.669 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:29:06.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:29:06.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.669 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.669 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:29:06.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:29:06.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.669 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.670 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:29:06.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:29:06.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.670 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.670 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:29:06.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:29:06.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.670 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.670 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:29:06.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:29:06.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:29:06.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:29:06.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:29:06.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:29:06.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:29:06.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:29:06.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:29:06.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:29:06.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:29:06.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:29:06.672 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.672 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.672 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.672 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:06.672 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:06.673 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 127.927923189, 2023-12-06T18:29:06.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:29:06.673 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.673 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.673 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.673 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.673 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.673 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:29:06.673 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.673 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.673 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.673 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.673 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:06.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:06.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:29:06.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:29:06.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:29:06.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:29:06.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:29:06.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:29:06.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:29:06.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:29:06.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:29:06.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:29:06.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.676 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.676 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.676 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:29:06.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:29:06.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.676 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.676 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.676 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:29:06.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:29:06.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.676 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.676 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.676 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:29:06.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:29:06.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:29:06.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:29:06.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:29:06.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:29:06.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:29:06.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:29:06.678 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.678 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.678 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.678 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:06.678 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:06.678 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 127.89277408700001, 2023-12-06T18:29:06.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:29:06.678 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.678 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.678 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.678 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.678 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.679 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:29:06.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.679 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:06.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:06.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:29:06.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:29:06.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:29:06.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:29:06.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:29:06.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:29:06.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.681 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:29:06.681 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:29:06.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.681 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:29:06.681 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:29:06.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.681 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:29:06.681 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:29:06.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.681 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:29:06.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:29:06.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.682 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.682 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:29:06.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:29:06.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.682 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.682 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:29:06.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:29:06.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.682 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.682 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.683 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:29:06.683 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:29:06.683 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.683 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.683 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.683 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:29:06.683 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:29:06.683 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.683 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.683 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.683 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.683 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.683 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:06.683 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:06.683 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 121.34304356200001, 2023-12-06T18:29:06.683 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:29:06.684 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.684 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.684 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.684 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.684 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.684 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:29:06.684 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.684 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.684 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.684 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.684 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:06.684 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.684 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.684 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.684 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:06.685 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.685 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:29:06.685 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:29:06.685 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:29:06.685 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:29:06.685 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:29:06.686 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:29:06.686 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.686 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.686 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.686 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:29:06.686 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:29:06.686 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.686 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.686 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.686 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:29:06.686 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:29:06.686 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.686 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.686 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.687 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:29:06.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:29:06.687 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.687 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.687 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.687 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:29:06.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:29:06.687 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.687 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.687 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.687 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:29:06.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:29:06.687 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.687 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.688 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.688 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:29:06.688 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:29:06.688 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.688 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.688 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:29:06.688 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:29:06.688 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.688 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.688 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:29:06.688 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:29:06.688 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.688 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.688 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.689 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:06.689 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:06.689 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 121.342237954, 2023-12-06T18:29:06.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:29:06.689 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.689 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.689 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.689 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.689 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.689 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:29:06.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.689 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.689 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:06.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:06.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:29:06.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:29:06.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:29:06.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:29:06.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:29:06.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:29:06.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:29:06.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:29:06.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:29:06.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:29:06.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:29:06.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:29:06.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:29:06.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:29:06.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:29:06.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:29:06.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:29:06.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:29:06.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:29:06.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:29:06.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:29:06.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:29:06.694 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.694 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.694 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.694 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:06.694 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:06.694 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 121.34154714500001, 2023-12-06T18:29:06.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:29:06.694 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.694 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.694 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.695 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.695 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.695 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:29:06.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.695 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:06.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:06.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:29:06.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:29:06.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:29:06.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:29:06.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:29:06.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:29:06.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:29:06.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:29:06.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:29:06.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:29:06.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:29:06.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:29:06.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:29:06.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:29:06.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:29:06.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:29:06.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:29:06.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:29:06.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:29:06.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:29:06.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:29:06.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:29:06.699 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.699 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.699 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.699 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:06.699 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:06.699 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 121.337249637, 2023-12-06T18:29:06.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:29:06.699 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.700 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.700 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.700 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.700 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.700 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:29:06.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.700 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:06.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:06.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:29:06.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:29:06.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:29:06.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:29:06.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:29:06.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:29:06.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:29:06.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:29:06.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:29:06.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:29:06.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:29:06.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:29:06.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:29:06.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:29:06.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:29:06.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:29:06.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:29:06.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:29:06.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:29:06.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:29:06.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:29:06.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:29:06.704 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.704 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.704 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.705 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:06.705 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:06.705 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 121.336581962, 2023-12-06T18:29:06.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:29:06.705 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.705 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.705 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.705 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.705 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.705 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:29:06.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.705 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.705 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.705 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:06.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:06.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:29:06.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:29:06.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:29:06.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:29:06.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:29:06.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:29:06.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:29:06.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:29:06.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:29:06.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:29:06.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:29:06.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:29:06.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:29:06.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:29:06.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:29:06.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:29:06.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:29:06.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:29:06.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:29:06.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:29:06.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:29:06.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:29:06.709 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.709 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.710 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.710 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:06.710 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:06.710 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 121.335926534, 2023-12-06T18:29:06.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:29:06.710 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.710 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.710 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.710 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.710 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.710 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:29:06.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.711 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:06.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:06.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:29:06.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:29:06.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:29:06.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:29:06.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:29:06.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:29:06.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:29:06.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:29:06.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:29:06.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:29:06.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:29:06.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:29:06.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:29:06.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:29:06.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:29:06.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:29:06.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:29:06.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:29:06.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:29:06.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:29:06.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:29:06.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:29:06.715 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.715 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.715 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.715 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:06.715 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:06.715 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 121.29527198700001, 2023-12-06T18:29:06.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:29:06.715 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:06.715 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:06.715 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:06.716 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:06.716 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:06.716 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:29:06.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.716 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:06.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:06.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:06.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:06.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:06.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:06.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:06.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:29:06.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:29:06.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:06.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:29:06.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:29:06.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:06.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:29:06.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:29:06.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:06.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:29:06.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:29:06.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:06.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:29:06.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:29:06.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:06.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:29:06.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:29:06.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:06.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:29:06.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:29:06.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:06.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:29:06.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:29:06.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:06.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:29:06.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:29:06.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:06.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:29:06.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:29:06.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:06.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:06.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:06.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:29:06.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:29:06.720 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.720 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.720 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.720 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:06.720 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:06.720 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:06.722 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:29:06.722 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:29:06.723 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:29:06.723 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:06.718+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:06.723 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:29:06.723 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:29:06.809 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:29:06.809 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:29:06.812 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:29:06.816 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:29:06.817 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:29:06.822 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:06.819+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:06.879 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:06.879 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:06.879 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:06.880 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:06.890 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:29:06.932 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:06.928+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:06.984 INFO:teuthology.orchestra.run.smithi061.stderr:set 2.7 pg_upmap mapping to [5,3] 2023-12-06T18:29:06.989 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is set 2023-12-06T18:29:07.025 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:07.022+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:07.073 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:07.073 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:07.073 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:07.073 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:07.073 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:07.083 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:29:07.127 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:07.123+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:07.222 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:07.222 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:07.222 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:07.222 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:07.222 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:07.222 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:29:07.223 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:07.223 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 105.406430634, 2023-12-06T18:29:07.223 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:29:07.223 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:07.223 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:29:07.223 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:07.223 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:07.223 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:07.223 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:07.223 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:07.223 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:07.223 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:07.223 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:07.223 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:07.223 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:07.224 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:07.224 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:07.224 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:07.224 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:29:07.224 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:29:07.224 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:07.224 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:07.224 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:07.224 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:29:07.224 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:29:07.224 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:07.224 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:07.224 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:07.224 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:29:07.224 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:29:07.224 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:07.224 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:07.225 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:07.225 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:29:07.225 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:29:07.225 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:07.225 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:07.225 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:07.225 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:29:07.225 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:29:07.225 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:07.225 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:07.225 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:07.225 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:29:07.225 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:29:07.225 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:07.225 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:07.225 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:07.226 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:29:07.226 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:29:07.226 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:07.226 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:07.226 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:07.226 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:07.226 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:07.226 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:07.227 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:07.224+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:07.231 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:29:07.328 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:07.325+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:07.410 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:07.410 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:07.410 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:07.410 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:07.419 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:29:07.429 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:07.426+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:07.530 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:07.527+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:07.634 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:07.630+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:07.636 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:07.636 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:07.636 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:07.636 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:07.636 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:07.645 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:29:07.732 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:07.729+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:07.827 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:07.827 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:07.827 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:07.828 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:29:07.828 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:07.833 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:07.830+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:07.836 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:29:07.934 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:07.931+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:07.963 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:07.964 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:07.964 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:07.964 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:07.972 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:29:08.056 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:08.053+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:08.141 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:08.141 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:08.141 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:08.142 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:08.139+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:08.151 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:29:08.238 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:08.234+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:08.322 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:08.322 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:08.322 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:08.322 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:08.322 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.322 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:08.322 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:08.323 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.828939351000002, 2023-12-06T18:29:08.323 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:29:08.323 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.323 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:29:08.323 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:29:08.323 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:29:08.323 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:08.323 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:29:08.323 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.323 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.323 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.323 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.323 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:08.323 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.323 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.324 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.324 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.324 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:08.324 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.324 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.324 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.324 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.324 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:29:08.324 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:29:08.324 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.324 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.324 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.324 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:29:08.324 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:29:08.324 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.324 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.325 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.325 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:29:08.325 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:29:08.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.325 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.325 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.325 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:29:08.325 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:29:08.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.325 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.325 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.325 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:29:08.325 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:29:08.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.325 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.325 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.326 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:29:08.326 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:29:08.326 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.326 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.326 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.326 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.326 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.326 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:08.332 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:29:08.338 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:08.334+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:08.438 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:08.435+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:08.500 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:08.501 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:08.501 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:08.501 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:08.510 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:29:08.514 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:29:08.559 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:08.555+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:08.640 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:08.637+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:08.680 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:08.681 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:08.681 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:08.681 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:08.681 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:08.689 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:29:08.774 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:08.770+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:08.817 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:29:08.862 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:08.858+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:08.898 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:08.898 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:08.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:08.898 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:08.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.898 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:29:08.898 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:08.898 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 132.28297166799999, 2023-12-06T18:29:08.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:29:08.898 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.898 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.898 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:08.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:08.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:29:08.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:29:08.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:29:08.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:29:08.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:29:08.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:29:08.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:29:08.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:29:08.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:29:08.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:29:08.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:29:08.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:29:08.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:29:08.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:29:08.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:29:08.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:29:08.902 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.902 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.902 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.902 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:29:08.902 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:08.902 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 131.26014465899999, 2023-12-06T18:29:08.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:29:08.902 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.902 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.902 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:08.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:08.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:29:08.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:29:08.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:29:08.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:29:08.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:29:08.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:29:08.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:29:08.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:29:08.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:29:08.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:29:08.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:29:08.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:29:08.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:29:08.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:29:08.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:29:08.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:29:08.905 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.905 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.906 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.906 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:29:08.906 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:08.906 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 131.248098021, 2023-12-06T18:29:08.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:29:08.906 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.906 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.906 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:08.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:08.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:29:08.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:29:08.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:29:08.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:29:08.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:29:08.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:29:08.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:29:08.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:29:08.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:29:08.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:29:08.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:29:08.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:29:08.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:29:08.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:29:08.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:29:08.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:29:08.909 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.909 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.909 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.909 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:29:08.909 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:08.910 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 131.226697414, 2023-12-06T18:29:08.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:29:08.910 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.910 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.910 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:08.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:08.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:29:08.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:29:08.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:29:08.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:29:08.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:29:08.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:29:08.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:29:08.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:29:08.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:29:08.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:29:08.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:29:08.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:29:08.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:29:08.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:29:08.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:29:08.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:29:08.913 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.913 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.913 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.913 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:29:08.913 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:08.913 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 131.225518091, 2023-12-06T18:29:08.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:29:08.913 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.913 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.914 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:08.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:08.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:29:08.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:29:08.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:29:08.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:29:08.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:29:08.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:29:08.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:29:08.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:29:08.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:29:08.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:29:08.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:29:08.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:29:08.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:29:08.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:29:08.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:29:08.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:29:08.917 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.917 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.917 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.917 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:29:08.917 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:08.917 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 131.22494610999999, 2023-12-06T18:29:08.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:29:08.917 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.917 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.918 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:08.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:08.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:29:08.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:29:08.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:29:08.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:29:08.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:29:08.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:29:08.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:29:08.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:29:08.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:29:08.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:29:08.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:29:08.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:29:08.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:29:08.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:29:08.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:29:08.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:29:08.921 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.921 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.921 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.921 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:29:08.921 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:08.921 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 130.77911064, 2023-12-06T18:29:08.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:29:08.921 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.921 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.921 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:08.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:08.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:29:08.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:29:08.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:29:08.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:29:08.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:29:08.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:29:08.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:29:08.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:29:08.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:29:08.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:29:08.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:29:08.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:29:08.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:29:08.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:29:08.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:29:08.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:29:08.924 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.924 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.924 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.925 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:29:08.925 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:08.925 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 130.77751078399999, 2023-12-06T18:29:08.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:29:08.925 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.925 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.925 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:08.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:08.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:29:08.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:29:08.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:29:08.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:29:08.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:29:08.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:29:08.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:29:08.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:29:08.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:29:08.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:29:08.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:29:08.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:29:08.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:29:08.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:29:08.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:29:08.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:29:08.928 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.928 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.928 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.928 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:29:08.928 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:08.928 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 130.39830516800001, 2023-12-06T18:29:08.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:29:08.929 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.929 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.929 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:08.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:08.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:29:08.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:29:08.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:29:08.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:29:08.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:29:08.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:29:08.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:29:08.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:29:08.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:29:08.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:29:08.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:29:08.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:29:08.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:29:08.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:29:08.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.932 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:29:08.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:29:08.932 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.932 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.932 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.932 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.932 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:29:08.932 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:08.932 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 130.16080298599999, 2023-12-06T18:29:08.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:29:08.932 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.932 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.932 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:08.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:08.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:29:08.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:29:08.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:29:08.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:29:08.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:29:08.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:29:08.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:29:08.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:29:08.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:29:08.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:29:08.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.935 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.935 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:29:08.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:29:08.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.935 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.935 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:29:08.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:29:08.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.935 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.935 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:29:08.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:29:08.935 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.935 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.936 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.936 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.936 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.936 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:29:08.936 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:08.936 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 130.160074987, 2023-12-06T18:29:08.936 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:29:08.936 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.936 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.936 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.936 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.936 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.936 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:08.936 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.936 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.936 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:08.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:29:08.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:29:08.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:29:08.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:29:08.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:29:08.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:29:08.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:29:08.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:29:08.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:29:08.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:29:08.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:29:08.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:29:08.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.939 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.939 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.939 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:29:08.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:29:08.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.939 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.939 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.939 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:29:08.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:29:08.939 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.939 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.939 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.939 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.939 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:29:08.940 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:08.940 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 130.155062577, 2023-12-06T18:29:08.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:29:08.940 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.940 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.940 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:08.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:08.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:29:08.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:29:08.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:29:08.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:29:08.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:29:08.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:29:08.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:29:08.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:29:08.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:29:08.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:29:08.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:29:08.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:29:08.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:29:08.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:29:08.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.943 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:29:08.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:29:08.943 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.943 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.943 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.943 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:29:08.943 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:08.943 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 130.14953549800001, 2023-12-06T18:29:08.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:29:08.943 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.944 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.944 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.944 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.944 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.944 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:08.944 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.944 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.944 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.944 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.944 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:08.944 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.944 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.944 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.944 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.944 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:29:08.944 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:29:08.945 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.945 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.945 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.945 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:29:08.945 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:29:08.945 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.945 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.945 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.945 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:29:08.945 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:29:08.945 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.945 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.945 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.945 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:29:08.945 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:29:08.946 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.946 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.946 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.946 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:29:08.946 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:29:08.946 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.946 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.946 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.946 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:29:08.946 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:29:08.946 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.946 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.946 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.946 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:29:08.947 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:29:08.947 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.947 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.947 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.947 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:29:08.947 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:29:08.947 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.947 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.947 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.947 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.947 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.947 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:29:08.947 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:08.947 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 130.149024661, 2023-12-06T18:29:08.947 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:29:08.948 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.948 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.948 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.948 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.948 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:08.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.948 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.948 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:08.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.948 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.948 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:29:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:29:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:29:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:29:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:29:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:29:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.949 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:29:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:29:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:29:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:29:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:29:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:29:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.950 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:29:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:29:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:29:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:29:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:29:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:08.951 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 130.1485654, 2023-12-06T18:29:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:29:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.952 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:29:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:29:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:29:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:29:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:29:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:29:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:29:08.953 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:29:08.954 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.954 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.954 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.954 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:29:08.954 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:29:08.954 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.954 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.954 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.954 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:29:08.954 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:29:08.954 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.954 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.954 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.954 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:29:08.954 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:29:08.954 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.955 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.955 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.955 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:29:08.955 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:29:08.955 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.955 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.955 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.955 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.955 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.955 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:29:08.955 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:08.955 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 123.568649593, 2023-12-06T18:29:08.955 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:29:08.955 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.955 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.955 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.955 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.956 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.956 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:08.956 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.956 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.956 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.956 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.956 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:08.956 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.956 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.956 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.956 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.956 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:29:08.956 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:29:08.956 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.956 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.957 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.957 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:29:08.957 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:29:08.957 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.957 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.957 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.957 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:29:08.957 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:29:08.957 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.957 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.957 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.957 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:29:08.957 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:29:08.957 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.957 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.957 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.958 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:29:08.958 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:29:08.958 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.958 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.958 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.958 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:29:08.958 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:29:08.958 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.958 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.958 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.958 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:29:08.958 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:29:08.958 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.958 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.958 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.959 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:29:08.959 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:29:08.959 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.959 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.959 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.959 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.959 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.959 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:29:08.959 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:08.959 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 123.56241178800001, 2023-12-06T18:29:08.959 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:29:08.959 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.959 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.959 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.959 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.959 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.959 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:08.960 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.960 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.960 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.960 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:08.960 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.960 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.960 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.960 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:29:08.960 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:29:08.960 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.960 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.960 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:29:08.961 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:29:08.961 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.961 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.961 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.961 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:29:08.961 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:29:08.961 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.961 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.961 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.961 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:29:08.961 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:29:08.961 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.961 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.961 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.962 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:29:08.962 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:29:08.962 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.962 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.962 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.962 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:29:08.962 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:29:08.962 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.962 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.962 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.962 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:29:08.962 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:29:08.962 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.962 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.963 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.963 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:29:08.963 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:29:08.963 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.963 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.963 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.963 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.963 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.963 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:29:08.963 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:08.963 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 123.55634870900001, 2023-12-06T18:29:08.963 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:29:08.963 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.963 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.963 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.964 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.964 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.964 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:08.964 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.964 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.964 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.964 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.964 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:08.964 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.964 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.964 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.964 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.964 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:29:08.964 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:29:08.964 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.965 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.965 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.965 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:29:08.965 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:29:08.965 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.965 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.965 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.965 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:29:08.965 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:29:08.965 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.965 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.965 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.965 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:29:08.965 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:29:08.965 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.965 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.966 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.966 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:29:08.966 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:29:08.966 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.966 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.966 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.966 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:29:08.966 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:29:08.966 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.966 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.966 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.966 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:29:08.966 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:29:08.966 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.966 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.967 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.967 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:29:08.967 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:29:08.967 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.967 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.967 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.967 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.967 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.967 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:29:08.967 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:08.967 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 123.418961624, 2023-12-06T18:29:08.967 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:29:08.967 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.967 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.967 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.968 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.968 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.968 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:08.968 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.968 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.968 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.968 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.968 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:08.968 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.968 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.968 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.968 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.968 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:29:08.968 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:29:08.968 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.969 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.969 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.969 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:29:08.969 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:29:08.969 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.969 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.969 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.969 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:29:08.969 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:29:08.969 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.969 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.969 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.969 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:29:08.969 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:29:08.969 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.970 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.970 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.970 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:29:08.970 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:29:08.970 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.970 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.970 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.970 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:29:08.970 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:29:08.970 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.970 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.970 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.970 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:29:08.970 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:29:08.971 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.971 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.971 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:29:08.971 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:29:08.971 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.971 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.971 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.971 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.971 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:29:08.971 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:08.971 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 123.41229089700001, 2023-12-06T18:29:08.971 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:29:08.971 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:08.971 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:08.972 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:08.972 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.972 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:08.972 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:08.972 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.972 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.972 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.972 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:08.972 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:08.972 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:08.972 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.972 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.972 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:08.972 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:29:08.972 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:29:08.972 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.973 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.973 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:08.973 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:29:08.973 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:29:08.973 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.973 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.973 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:08.973 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:29:08.973 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:29:08.973 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.973 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.973 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:08.973 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:29:08.973 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:29:08.973 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.973 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.974 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:08.974 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:29:08.974 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:29:08.974 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.974 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.974 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:08.974 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:29:08.974 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:29:08.974 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.974 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.974 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:08.974 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:29:08.974 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:29:08.974 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:08.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:08.975 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:08.975 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:29:08.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:29:08.975 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.975 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.975 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.975 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:08.975 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:08.975 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:08.976 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:29:08.977 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:08.973+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:09.007 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-12-06T18:29:09.077 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:09.073+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:09.153 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:09.153 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:09.154 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:09.154 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:09.163 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:29:09.178 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:09.174+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:09.202 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:29:09.203 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:29:09.270 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":24,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:29:09.296 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:09.296 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:09.296 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:09.296 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:09.296 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:09.304 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:29:09.304 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:29:09.304 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:09.301+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:09.306 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:29:09.380 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:09.377+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:09.397 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:29:09.397 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:29:09.434 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:09.434 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:09.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:09.434 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:29:09.434 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:09.443 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:29:09.488 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:09.484+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:09.491 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:29:09.492 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:29:09.583 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:09.579+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:09.594 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:29:09.595 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:29:09.600 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:09.600 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:09.609 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:29:09.680 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:29:09.680 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:29:09.684 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:09.680+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:09.766 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:29:09.767 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:29:09.769 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:29:09.773 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:29:09.774 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:29:09.784 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:09.781+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:09.789 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:09.789 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:09.799 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:29:09.885 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:09.882+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:09.976 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:09.977 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:09.986 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:29:09.986 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:09.983+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:10.087 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:10.084+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:10.163 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:10.163 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:10.172 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:29:10.217 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:10.213+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:10.288 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:10.284+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:10.310 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:10.310 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:10.319 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:29:10.388 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:10.385+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:10.489 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:10.486+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:10.496 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:10.497 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:10.507 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:29:10.591 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:10.587+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:10.630 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is unset 2023-12-06T18:29:10.656 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:10.656 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:10.666 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:29:10.709 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:10.704+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:10.793 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:10.790+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:10.852 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:10.852 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:10.865 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:29:10.894 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:10.891+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:10.995 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:10.992+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:11.013 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:11.013 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:11.022 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:29:11.063 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:25.125 98186== Warning: unimplemented fcntl command: 1036 2023-12-06T18:29:11.096 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:11.092+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:11.114 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:25.176 98186== Warning: unimplemented fcntl command: 1036 2023-12-06T18:29:11.160 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:11.160 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:11.169 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:29:11.197 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:11.194+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:11.298 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:11.295+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:11.347 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:11.347 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:11.350 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:25.412 98186== Warning: unimplemented fcntl command: 1036 2023-12-06T18:29:11.356 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:29:11.400 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:11.396+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:11.495 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:11.495 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:11.500 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:11.497+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:11.504 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:29:11.601 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:11.597+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:11.635 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:11.636 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:11.646 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:29:11.730 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:11.726+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:11.774 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:29:11.802 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:11.799+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:11.823 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:11.823 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:11.833 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:29:11.904 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:11.900+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:12.001 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [7] live_osds: [0, 1, 2, 3, 4, 5, 6] 2023-12-06T18:29:12.002 INFO:tasks.thrashosds.thrasher:Reviving osd 7 2023-12-06T18:29:12.002 INFO:tasks.ceph.osd.7:Restarting daemon 2023-12-06T18:29:12.002 DEBUG:teuthology.orchestra.run.smithi152:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 2023-12-06T18:29:12.005 INFO:tasks.ceph.osd.7:Started 2023-12-06T18:29:12.005 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:29:12.005 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:12.005 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:12.014 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:29:12.030 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:29:12.030 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:29:12.031 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:12.027+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:12.101 INFO:teuthology.orchestra.run.smithi152.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2023-12-06T18:29:12.104 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-12-06T18:29:12.104 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-7, ['dump_ops_in_flight'] 2023-12-06T18:29:12.104 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:12.101+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:12.123 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:29:12.123 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:29:12.186 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:12.187 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:12.196 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:29:12.205 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:12.202+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:12.214 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:29:12.214 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:29:12.288 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:29:12.306 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:12.302+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:12.309 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:29:12.309 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:29:12.326 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:12.326 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:12.335 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:29:12.401 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:29:12.402 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:29:12.407 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:12.403+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:12.499 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:29:12.499 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:29:12.508 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:12.504+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:12.609 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:12.605+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:12.619 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:12.619 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:12.619 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:12.619 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:12.619 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:29:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 135.97174902699999, 2023-12-06T18:29:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:29:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.620 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:29:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:29:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:29:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:29:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.621 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:29:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:29:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:29:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:29:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:29:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:29:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.622 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:29:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:29:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:29:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:29:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:29:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:29:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:29:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 134.94892201799999, 2023-12-06T18:29:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:29:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.624 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:29:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:29:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:29:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:29:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:29:12.625 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:29:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:29:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:29:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:29:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:29:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:29:12.626 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:29:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:29:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:29:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:29:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:29:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:29:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 134.93687538, 2023-12-06T18:29:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:29:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:29:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:29:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:29:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:29:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.629 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:29:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:29:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:29:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:29:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:29:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:29:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:29:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:29:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:29:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:29:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:29:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:29:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:29:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 134.915474773, 2023-12-06T18:29:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:29:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.632 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:29:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:29:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:29:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:29:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:29:12.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:29:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:29:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:29:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:29:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:29:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:29:12.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:29:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:29:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:29:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:29:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:29:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:29:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:12.635 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 134.91429545, 2023-12-06T18:29:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:29:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:29:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:29:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:29:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:29:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:29:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:29:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:29:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:29:12.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:29:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:29:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:29:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:29:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:29:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:29:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:29:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:29:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:29:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 134.91372346899999, 2023-12-06T18:29:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:29:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:29:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:29:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:29:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:29:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:29:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:29:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:29:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:29:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:29:12.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:29:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:29:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:29:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:29:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:29:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:29:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:29:12.642 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:29:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 134.467887999, 2023-12-06T18:29:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:29:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:29:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:29:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:29:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:29:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:29:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:29:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:29:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:29:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:29:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:29:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:29:12.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:29:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:29:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:29:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:29:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:29:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:29:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 134.46628814300001, 2023-12-06T18:29:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:29:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:29:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:29:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:29:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:29:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:29:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:29:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.648 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:29:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:29:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:29:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:29:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:29:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:29:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.649 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:29:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:29:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:29:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:29:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:29:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 134.08708252700001, 2023-12-06T18:29:12.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:29:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:29:12.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:29:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:29:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:29:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:29:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:29:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:29:12.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:29:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:29:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:29:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:29:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:29:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:29:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:29:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:29:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:29:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:29:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 133.84958034499999, 2023-12-06T18:29:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:29:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:29:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:29:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:29:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:29:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:29:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:29:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:29:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:29:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:29:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:29:12.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:29:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:29:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:29:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:29:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:29:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:29:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.657 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:29:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 133.848852346, 2023-12-06T18:29:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:29:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:29:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:29:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:29:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:29:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:29:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:29:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:29:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:29:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:29:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:29:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:29:12.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:29:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:29:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:29:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:29:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:29:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.661 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:29:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 133.84383993599999, 2023-12-06T18:29:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:29:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:29:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:29:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:29:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:29:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:29:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:29:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:29:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:29:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:29:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:29:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:29:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:29:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:29:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:29:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:29:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:29:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:29:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 133.83831285700001, 2023-12-06T18:29:12.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:29:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:29:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:29:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:29:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:29:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:29:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:29:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:29:12.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:29:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:29:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:29:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:29:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:29:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:29:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:29:12.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:29:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:29:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:29:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 133.83780202, 2023-12-06T18:29:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:29:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.669 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:29:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:29:12.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:29:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:29:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:29:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:29:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:29:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:29:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:29:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:29:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:29:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:29:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:29:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:29:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:29:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:29:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:29:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 133.83734275899999, 2023-12-06T18:29:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:29:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:29:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:29:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:29:12.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:29:12.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:29:12.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:29:12.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:29:12.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:29:12.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:29:12.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:29:12.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:29:12.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:29:12.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:29:12.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:29:12.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:29:12.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:29:12.677 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.677 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.677 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.677 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.677 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.677 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:29:12.677 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:12.677 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 127.257426952, 2023-12-06T18:29:12.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:29:12.677 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.677 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.677 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.677 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.677 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.677 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:12.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.677 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:12.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:29:12.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:29:12.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:29:12.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:29:12.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:29:12.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:29:12.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:29:12.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:29:12.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:29:12.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:29:12.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:29:12.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:29:12.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:29:12.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:29:12.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:29:12.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:29:12.680 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.681 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.681 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.681 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:29:12.681 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:12.681 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 127.25118914700001, 2023-12-06T18:29:12.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:29:12.681 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.681 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.681 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.681 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.681 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:12.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:12.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:29:12.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:29:12.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:29:12.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:29:12.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:29:12.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:29:12.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:29:12.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:29:12.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:29:12.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:29:12.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:29:12.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:29:12.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:29:12.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:29:12.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:29:12.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:29:12.684 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.684 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.684 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.684 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:29:12.685 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:12.685 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 127.245126068, 2023-12-06T18:29:12.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:29:12.685 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.685 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.685 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:12.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:12.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.686 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.686 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:29:12.686 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:29:12.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.686 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.686 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.686 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:29:12.686 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:29:12.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.686 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.686 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.686 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:29:12.686 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:29:12.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.686 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.687 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.687 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:29:12.687 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:29:12.687 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.687 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.687 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.687 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:29:12.687 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:29:12.687 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.687 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.687 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.687 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:29:12.687 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:29:12.687 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.687 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.687 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.688 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:29:12.688 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:29:12.688 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.688 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.688 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.688 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:29:12.688 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:29:12.688 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.688 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.688 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.688 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.688 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.688 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:29:12.688 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:12.688 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 127.107738983, 2023-12-06T18:29:12.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:29:12.689 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.689 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.689 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.689 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:12.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.689 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:12.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.689 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:29:12.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:29:12.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:29:12.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:29:12.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:29:12.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:29:12.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:29:12.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:29:12.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:29:12.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:29:12.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:29:12.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:29:12.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:29:12.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:29:12.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:29:12.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:29:12.692 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.692 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.692 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.692 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:29:12.692 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:12.692 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 127.101068256, 2023-12-06T18:29:12.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:29:12.692 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:12.693 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:12.693 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:12.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.693 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:12.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:12.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.693 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:12.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:12.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:12.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.693 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:12.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:29:12.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:29:12.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:12.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:29:12.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:29:12.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:12.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:29:12.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:29:12.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:12.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:29:12.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:29:12.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:12.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:29:12.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:29:12.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:12.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:29:12.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:29:12.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:12.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:29:12.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:29:12.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:12.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:12.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:12.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:29:12.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:29:12.696 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.696 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.696 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.696 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:12.696 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:12.696 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:12.696 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-12-06T18:29:12.697 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:29:12.698 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:29:12.699 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:29:12.699 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:29:12.699 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:29:12.699 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:29:12.742 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:12.739+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:12.810 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:12.806+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:12.830 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:12.830 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:12.830 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:12.830 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:12.840 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:29:12.911 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:12.907+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:13.012 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:13.008+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:13.014 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:13.014 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:13.014 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:13.014 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:13.014 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:13.025 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:29:13.112 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:13.109+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:13.155 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:13.155 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:13.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:13.156 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:29:13.156 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:13.164 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:29:13.213 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:13.209+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:13.252 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":28,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:29:13.313 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:13.313 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:13.313 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:13.313 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:13.314 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:13.310+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:13.323 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:29:13.415 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:13.411+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:13.459 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:13.459 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:13.459 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:13.459 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:13.459 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:13.468 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:29:13.515 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:13.511+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:13.607 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:13.607 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:29:13.607 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:13.616 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:29:13.616 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:13.612+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:13.716 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:13.713+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:13.792 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:13.792 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:13.792 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:13.792 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:13.801 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:29:13.817 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:13.814+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:13.918 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:13.915+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:13.952 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is unset 2023-12-06T18:29:13.982 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:13.982 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:13.982 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:13.982 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:13.982 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:13.992 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:29:14.019 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:14.015+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:14.120 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:14.117+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:14.220 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:14.217+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:14.232 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:14.232 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:14.232 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:14.232 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:14.232 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.232 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:29:14.232 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:14.232 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 138.82261278499999, 2023-12-06T18:29:14.233 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:29:14.233 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.233 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.233 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.233 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.233 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.233 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:29:14.233 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.233 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.233 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.233 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.233 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:14.233 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.233 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.233 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.233 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.234 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:14.234 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.234 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.234 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.234 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.234 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:29:14.234 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:29:14.234 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.234 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.234 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.234 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:29:14.234 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:29:14.234 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.234 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.234 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.234 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:29:14.235 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:29:14.235 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.235 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.235 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.235 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:29:14.235 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:29:14.235 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.235 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.235 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.235 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:29:14.235 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:29:14.235 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.235 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.235 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:29:14.235 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:29:14.235 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:29:14.236 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.236 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.236 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.236 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:29:14.236 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:29:14.236 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.236 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.236 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.236 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:29:14.236 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:29:14.236 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.236 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.236 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.236 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:29:14.236 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:29:14.236 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.236 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.237 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.237 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:29:14.237 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:29:14.237 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.237 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.237 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.237 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:29:14.237 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:29:14.237 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.237 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.237 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.237 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:29:14.237 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:29:14.237 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.237 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.237 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.237 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:29:14.238 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:29:14.238 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.238 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.238 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.238 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.238 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.238 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:14.238 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:14.238 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 136.71403954600001, 2023-12-06T18:29:14.238 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:29:14.238 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.238 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.238 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.238 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.238 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.239 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:29:14.239 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.239 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.239 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.239 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.239 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:14.239 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.239 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.239 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.239 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.239 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:14.239 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.239 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.239 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.239 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.240 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:29:14.240 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:29:14.240 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.240 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.240 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.240 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:29:14.240 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:29:14.240 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.240 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.240 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.240 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:29:14.240 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:29:14.240 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.240 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.240 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.240 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:29:14.241 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:29:14.241 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.241 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.241 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.241 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:29:14.241 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:29:14.241 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.241 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.241 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.241 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:29:14.241 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:29:14.241 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.241 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.241 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.241 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:29:14.241 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:29:14.241 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.242 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.242 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.242 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:29:14.242 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:29:14.242 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.242 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.242 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.242 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:29:14.242 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:29:14.242 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.242 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.242 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.242 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:29:14.242 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:29:14.243 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.243 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.243 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.243 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:29:14.243 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:29:14.243 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.243 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.243 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.243 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.243 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.243 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:14.243 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:14.243 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 136.713294314, 2023-12-06T18:29:14.243 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:29:14.243 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.243 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.243 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.244 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.244 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.244 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:29:14.244 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.244 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.244 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.244 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.244 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:14.244 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.244 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.244 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.244 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.244 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:14.244 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.244 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.244 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.245 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.245 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:29:14.245 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:29:14.245 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.245 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.245 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.245 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:29:14.245 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:29:14.245 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.245 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.245 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.245 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:29:14.245 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:29:14.245 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.245 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.245 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.246 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:29:14.246 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:29:14.246 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.246 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.246 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.246 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:29:14.246 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:29:14.246 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.246 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.246 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.246 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:29:14.246 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:29:14.246 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.246 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.246 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.246 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:29:14.247 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:29:14.247 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.247 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.247 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.247 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:29:14.247 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:29:14.247 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.247 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.247 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.247 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:29:14.247 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:29:14.247 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.247 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.247 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.247 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:29:14.247 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:29:14.247 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.248 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.248 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.248 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:29:14.248 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:29:14.248 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.248 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.248 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.248 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.248 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.248 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:14.248 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:14.248 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 136.71261055900001, 2023-12-06T18:29:14.248 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:29:14.248 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.248 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.248 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.248 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.249 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.249 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:29:14.249 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.249 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.249 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.249 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.249 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:14.249 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.249 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.249 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.249 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.249 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:14.249 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.249 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.249 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.249 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.249 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:29:14.249 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:29:14.250 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.250 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.250 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.250 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:29:14.250 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:29:14.250 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.250 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.250 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.250 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:29:14.250 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:29:14.250 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.250 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.250 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.250 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:29:14.250 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:29:14.250 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.250 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.251 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.251 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:29:14.251 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:29:14.251 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.251 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.251 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.251 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:29:14.251 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:29:14.251 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.251 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.251 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.251 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:29:14.251 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:29:14.251 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.251 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.251 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.251 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:29:14.252 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:29:14.252 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.252 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.252 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.252 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:29:14.252 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:29:14.252 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.252 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.252 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.252 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:29:14.252 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:29:14.252 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.252 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.252 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.252 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:29:14.252 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:29:14.252 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.253 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.253 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.253 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.253 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.253 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:14.253 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:14.253 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 136.70703730400001, 2023-12-06T18:29:14.253 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:29:14.253 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.253 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.253 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.253 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.253 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.253 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:29:14.253 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.253 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.253 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.254 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.254 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:14.254 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.254 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.254 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.254 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.254 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:14.254 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.254 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.254 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.254 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.254 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:29:14.254 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:29:14.254 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.254 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.254 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.254 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:29:14.255 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:29:14.255 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.255 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.255 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.255 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:29:14.255 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:29:14.255 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.255 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.255 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.255 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:29:14.255 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:29:14.255 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.255 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.255 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.255 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:29:14.255 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:29:14.255 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.256 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.256 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.256 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:29:14.256 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:29:14.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.256 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.256 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.256 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:29:14.256 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:29:14.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.256 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.256 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.256 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:29:14.256 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:29:14.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.256 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.256 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.257 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:29:14.257 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:29:14.257 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.257 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.257 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.257 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:29:14.257 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:29:14.257 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.257 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.257 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.257 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:29:14.257 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:29:14.257 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.257 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.257 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.257 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.257 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.258 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:14.258 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:14.258 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 136.70636366400001, 2023-12-06T18:29:14.258 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:29:14.258 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.258 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.258 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.258 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.258 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.258 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:29:14.258 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.258 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.258 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.258 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.258 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:14.258 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.259 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.259 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.259 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.259 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:14.259 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.259 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.259 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.259 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.259 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:29:14.259 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:29:14.259 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.259 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.259 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.259 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:29:14.259 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:29:14.259 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.260 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.260 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.260 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:29:14.260 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:29:14.260 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.260 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.260 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.260 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:29:14.260 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:29:14.260 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.260 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.260 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.260 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:29:14.260 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:29:14.260 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.260 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.260 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.261 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:29:14.261 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:29:14.261 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.261 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.261 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.261 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:29:14.261 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:29:14.261 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.261 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.261 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.261 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:29:14.261 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:29:14.261 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.261 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.261 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.261 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:29:14.261 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:29:14.262 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.262 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.262 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.262 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:29:14.262 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:29:14.262 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.262 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.262 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.262 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:29:14.262 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:29:14.262 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.262 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.262 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.262 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.262 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.262 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:14.263 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:14.263 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 136.705701465, 2023-12-06T18:29:14.263 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:29:14.263 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.263 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.263 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.263 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.263 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.263 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:29:14.263 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.263 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.263 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.263 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.263 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:14.263 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.263 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.263 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.264 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.264 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:14.264 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.264 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.264 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.264 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.264 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:29:14.264 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:29:14.264 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.264 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.264 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.264 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:29:14.264 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:29:14.264 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.264 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.264 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.265 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:29:14.265 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:29:14.265 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.265 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.265 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.265 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:29:14.265 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:29:14.265 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.265 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.265 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.265 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:29:14.265 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:29:14.265 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.265 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.265 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.265 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:29:14.265 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:29:14.265 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.266 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.266 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.266 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:29:14.266 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:29:14.266 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.266 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.266 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.266 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:29:14.266 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:29:14.266 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.266 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.266 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.266 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:29:14.266 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:29:14.266 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.267 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.267 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.267 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:29:14.267 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:29:14.267 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.267 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.267 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.267 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:29:14.267 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:29:14.267 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.267 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.267 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.267 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.267 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.267 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:14.267 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:14.267 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 136.63299203, 2023-12-06T18:29:14.268 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:29:14.268 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.268 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.268 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.268 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.268 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.268 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:29:14.268 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.268 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.268 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.268 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.268 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:14.268 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.268 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.268 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.268 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.268 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:14.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.269 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.269 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:29:14.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:29:14.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.269 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.269 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:29:14.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:29:14.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.269 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.269 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:29:14.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:29:14.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.270 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.270 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.270 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:29:14.270 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:29:14.270 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.270 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.270 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.270 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:29:14.270 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:29:14.270 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.270 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.270 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.270 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:29:14.270 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:29:14.270 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.270 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.270 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.271 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:29:14.271 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:29:14.271 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.271 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.271 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.271 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:29:14.271 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:29:14.271 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.271 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.271 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.271 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:29:14.271 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:29:14.271 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.271 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.271 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.271 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:29:14.271 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:29:14.272 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.272 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.272 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.272 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:29:14.272 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:29:14.272 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.272 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.272 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.272 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.272 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.272 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:14.272 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:14.272 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 136.63231032100001, 2023-12-06T18:29:14.272 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:29:14.272 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.272 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.273 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.273 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.273 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.273 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:29:14.273 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.273 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.273 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.273 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.273 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:14.273 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.273 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.273 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.273 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.273 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:14.273 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.273 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.273 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.274 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.274 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:29:14.274 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:29:14.274 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.274 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.274 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.274 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:29:14.274 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:29:14.274 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.274 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.274 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.274 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:29:14.274 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:29:14.274 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.274 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.274 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.275 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:29:14.275 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:29:14.275 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.275 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.275 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:29:14.275 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:29:14.275 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.275 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.275 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:29:14.275 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:29:14.275 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.275 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.275 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:29:14.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:29:14.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.276 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:29:14.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:29:14.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.276 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:29:14.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:29:14.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.276 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:29:14.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:29:14.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.277 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.277 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:29:14.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:29:14.277 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.277 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.277 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.277 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.277 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.277 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:14.277 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:14.277 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 135.55350623300001, 2023-12-06T18:29:14.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:29:14.277 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.277 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.277 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.278 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.278 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.278 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:29:14.278 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.278 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.278 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.278 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:14.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.279 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:14.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:29:14.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:29:14.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:29:14.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:29:14.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:29:14.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:29:14.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.281 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:29:14.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:29:14.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.281 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:29:14.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:29:14.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.281 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:29:14.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:29:14.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:29:14.282 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:29:14.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:29:14.282 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:29:14.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:29:14.282 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:29:14.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.283 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:29:14.283 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:29:14.283 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.283 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.283 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.283 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:29:14.283 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:29:14.283 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.283 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.283 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.283 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.283 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.283 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:14.283 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:14.283 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 135.55208965899999, 2023-12-06T18:29:14.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:29:14.284 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.284 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.284 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.284 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.284 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.284 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:29:14.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.284 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.284 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.284 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:14.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.284 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.284 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:14.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.285 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.285 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:29:14.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:29:14.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.285 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.285 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:29:14.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:29:14.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.285 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.285 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:29:14.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:29:14.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.286 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:29:14.286 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:29:14.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.286 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:29:14.286 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:29:14.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.286 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:29:14.286 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:29:14.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:29:14.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:29:14.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:29:14.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:29:14.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:29:14.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:29:14.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.288 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:29:14.288 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:29:14.288 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.288 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.288 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.288 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:29:14.288 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:29:14.288 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.288 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.288 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.288 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.288 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.288 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:14.288 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:14.288 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 135.55100453399999, 2023-12-06T18:29:14.288 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:29:14.288 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.289 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.289 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.289 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.289 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.289 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:29:14.289 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.289 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.289 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.289 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.289 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:14.289 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.289 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.289 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.289 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.289 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:14.289 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.289 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.289 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.290 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.290 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:29:14.290 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:29:14.290 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.290 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.290 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:29:14.290 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:29:14.290 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.290 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.290 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:29:14.290 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:29:14.290 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.290 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.290 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:29:14.291 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:29:14.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.291 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.291 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.291 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:29:14.291 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:29:14.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.291 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.291 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.291 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:29:14.291 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:29:14.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.291 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.291 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.291 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:29:14.291 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:29:14.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.292 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.292 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:29:14.292 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:29:14.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.292 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.292 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:29:14.292 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:29:14.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.292 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.292 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:29:14.292 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:29:14.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.292 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.293 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:29:14.293 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:29:14.293 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.293 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.293 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.293 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.293 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.293 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:14.293 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:14.293 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 135.515855432, 2023-12-06T18:29:14.293 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:29:14.293 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.293 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.293 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.293 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.294 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.294 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:29:14.294 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.294 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.294 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.294 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.294 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:14.294 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.294 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.294 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.294 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.294 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:14.294 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.294 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.294 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.294 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.294 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:29:14.295 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:29:14.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.295 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.295 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.295 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:29:14.295 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:29:14.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.295 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.295 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.295 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:29:14.295 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:29:14.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.295 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.295 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.295 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:29:14.295 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:29:14.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.296 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.296 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.296 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:29:14.296 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:29:14.296 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.296 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.296 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.296 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:29:14.296 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:29:14.296 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.296 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.296 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.296 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:29:14.296 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:29:14.296 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.296 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.297 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.297 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:29:14.297 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:29:14.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.297 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.297 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.297 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:29:14.297 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:29:14.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.297 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.297 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.297 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:29:14.297 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:29:14.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.297 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.297 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.297 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:29:14.298 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:29:14.298 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.298 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.298 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.298 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.298 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.298 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:14.298 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:14.298 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 128.96612490699999, 2023-12-06T18:29:14.298 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:29:14.298 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.298 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.298 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.298 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.298 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.298 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:29:14.298 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.299 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.299 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.299 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.299 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:14.299 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.299 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.299 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.299 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.299 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:14.299 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.299 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.299 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.299 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.299 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:29:14.299 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:29:14.299 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.299 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.300 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.300 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:29:14.300 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:29:14.300 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.300 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.300 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.300 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:29:14.300 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:29:14.300 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.300 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.300 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.300 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:29:14.300 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:29:14.300 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.300 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.300 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.300 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:29:14.301 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:29:14.301 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.301 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.301 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.301 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:29:14.301 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:29:14.301 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.301 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.301 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.301 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:29:14.301 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:29:14.301 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.301 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.301 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.301 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:29:14.301 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:29:14.302 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.302 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.302 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.302 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:29:14.302 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:29:14.302 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.302 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.302 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.302 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:29:14.302 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:29:14.302 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.302 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.302 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:29:14.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:29:14.303 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.303 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.303 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.303 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:14.303 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:14.303 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 128.96531929899999, 2023-12-06T18:29:14.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:29:14.303 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.303 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.303 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.303 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.304 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.304 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:29:14.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.304 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:14.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:14.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:29:14.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:29:14.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:29:14.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:29:14.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:29:14.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:29:14.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:29:14.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:29:14.306 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.306 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:29:14.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:29:14.306 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.306 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:29:14.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:29:14.306 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.306 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:29:14.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:29:14.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.307 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.307 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.307 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:29:14.307 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:29:14.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.307 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.307 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.307 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:29:14.307 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:29:14.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.307 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.307 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.307 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:29:14.307 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:29:14.308 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.308 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.308 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.308 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:29:14.308 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:29:14.308 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.308 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.308 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.308 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.308 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.308 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:14.308 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:14.308 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 128.96462849, 2023-12-06T18:29:14.308 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:29:14.308 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.308 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.309 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.309 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.309 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.309 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:29:14.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.309 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.309 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.309 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:14.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.309 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.309 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:14.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.310 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.310 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:29:14.310 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:29:14.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.310 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.310 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.310 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:29:14.310 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:29:14.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.310 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.310 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.310 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:29:14.310 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:29:14.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.310 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.310 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:29:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:29:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:29:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:29:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:29:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:29:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:29:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:29:14.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:29:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:29:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:29:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:29:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:29:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:29:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:29:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:29:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 128.96033098199999, 2023-12-06T18:29:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:29:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:29:14.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:29:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:29:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:29:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:29:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:29:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:29:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:29:14.315 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:29:14.316 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.316 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.316 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.316 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:29:14.316 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:29:14.316 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.316 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.316 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.316 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:29:14.316 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:29:14.316 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.316 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.316 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.316 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:29:14.316 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:29:14.316 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.317 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.317 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:29:14.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:29:14.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.317 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.317 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:29:14.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:29:14.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.317 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.317 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:29:14.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:29:14.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:29:14.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:29:14.318 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.318 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.318 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.318 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:14.318 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:14.318 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 128.959663307, 2023-12-06T18:29:14.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:29:14.318 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.318 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.318 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.319 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.319 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.319 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:29:14.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.319 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:14.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:14.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:29:14.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:29:14.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:29:14.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:29:14.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:29:14.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:29:14.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.321 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.321 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:29:14.321 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:29:14.321 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.321 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.321 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.321 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:29:14.321 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:29:14.321 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.321 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.321 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.321 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:29:14.321 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:29:14.321 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.321 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.321 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.322 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:29:14.322 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:29:14.322 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.322 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.322 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.322 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:29:14.322 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:29:14.322 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.322 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.322 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.322 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:29:14.322 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:29:14.322 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.322 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.322 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.323 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:29:14.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:29:14.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.323 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.323 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.323 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:29:14.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:29:14.323 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.323 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.323 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.323 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.323 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:14.323 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:14.323 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 128.95900787900001, 2023-12-06T18:29:14.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:29:14.324 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.324 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.324 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.324 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.324 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.324 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:29:14.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.324 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.324 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:14.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.324 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:14.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:29:14.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:29:14.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:29:14.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:29:14.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:29:14.326 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:29:14.326 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.326 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.326 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.326 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:29:14.326 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:29:14.326 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.326 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.326 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.326 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:29:14.326 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:29:14.326 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.326 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.326 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.327 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:29:14.327 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:29:14.327 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.327 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.327 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.327 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:29:14.327 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:29:14.327 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.327 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.327 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.327 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:29:14.327 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:29:14.327 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.327 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.327 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.328 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:29:14.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:29:14.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.328 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.328 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.328 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:29:14.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:29:14.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.328 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.328 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.328 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:29:14.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:29:14.328 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.328 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.328 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.329 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:14.329 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:14.329 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 128.91835333200001, 2023-12-06T18:29:14.329 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:29:14.329 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.329 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:14.329 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:14.329 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:14.329 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:14.329 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:29:14.329 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.329 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.329 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:14.329 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:14.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:29:14.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:29:14.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:29:14.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:29:14.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.331 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.331 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.331 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:29:14.331 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:29:14.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.331 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.331 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.331 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:29:14.331 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:29:14.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.331 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.331 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.331 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:29:14.331 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:29:14.332 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.332 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.332 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.332 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:29:14.332 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:29:14.332 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.332 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.332 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:14.332 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:29:14.332 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:29:14.332 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.332 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.332 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:14.332 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:29:14.332 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:29:14.333 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.333 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.333 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:14.333 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:29:14.333 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:29:14.333 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.333 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.333 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:14.333 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:29:14.333 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:29:14.333 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.333 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.333 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.333 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:29:14.333 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:29:14.333 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.333 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.334 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.334 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.334 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.334 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:14.334 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:29:14.336 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:14.333+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:14.438 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:14.435+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:14.472 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:14.472 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:14.472 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:14.472 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:14.481 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:29:14.565 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:14.562+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:14.642 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:14.639+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:14.659 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:14.660 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:14.660 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:14.660 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:14.660 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:14.668 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:29:14.700 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:29:14.752 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:14.749+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:14.846 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:14.842+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:14.857 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:14.857 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:14.857 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:14.857 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:14.857 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.857 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:29:14.857 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:14.857 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 113.039327109, 2023-12-06T18:29:14.857 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:29:14.858 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:14.858 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:29:14.858 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:14.858 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.858 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:14.858 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:14.858 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.858 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.858 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.858 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:14.858 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:14.858 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:14.858 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.858 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.858 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:14.858 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:29:14.859 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:29:14.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.859 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:14.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:29:14.859 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:29:14.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.859 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:14.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:29:14.859 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:29:14.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.859 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:14.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:29:14.860 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:29:14.860 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.860 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.860 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:14.860 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:29:14.860 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:29:14.860 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.860 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.860 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:14.860 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:29:14.860 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:29:14.860 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:14.860 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:14.860 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:14.861 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:29:14.861 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:29:14.861 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.861 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.861 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.861 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:14.861 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:14.861 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:14.867 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:29:14.947 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:14.944+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:14.996 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:29:14.997 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:29:15.050 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:15.046+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:15.056 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:15.056 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:15.056 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:15.056 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:15.066 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:29:15.090 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:29:15.090 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:29:15.151 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:15.148+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:15.185 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:29:15.185 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:29:15.238 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:15.238 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:15.238 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:15.238 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:15.238 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:15.248 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:29:15.289 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:15.285+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:15.301 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:29:15.301 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:29:15.355 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:15.352+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:15.410 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:29:15.410 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:29:15.437 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:15.437 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:15.451 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:29:15.493 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:15.489+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:15.507 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:29:15.507 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:29:15.590 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:15.590 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:15.599 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:29:15.608 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:29:15.608 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:29:15.609 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:15.605+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:15.611 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:29:15.615 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:29:15.615 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:29:15.660 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:15.656+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:15.739 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:15.739 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:15.749 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:29:15.760 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:15.756+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:15.861 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:15.858+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:15.919 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:15.919 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:15.928 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:29:15.966 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-12-06T18:29:15.972 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:15.969+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:16.058 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:16.058 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:16.062 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:16.059+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:16.067 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:29:16.164 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:16.160+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:16.238 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:16.238 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:16.247 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:29:16.265 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:16.262+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:16.271 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:29:16.367 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:16.363+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:16.376 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:16.376 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:16.385 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:29:16.470 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:16.466+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:16.558 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:16.558 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:16.568 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:29:16.609 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:16.606+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:16.671 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:16.667+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:16.738 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:16.738 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:16.747 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:29:16.790 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:16.787+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:16.873 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:16.870+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:16.877 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:16.877 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:16.886 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:29:16.952 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":32,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:29:16.974 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:16.971+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:16.993 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is set 2023-12-06T18:29:17.034 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:17.034 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:17.045 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:29:17.086 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:17.082+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:17.105 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:29:17.176 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:17.173+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:17.198 INFO:teuthology.orchestra.run.smithi152.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2023-12-06T18:29:17.201 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-12-06T18:29:17.201 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-7, ['dump_ops_in_flight'] 2023-12-06T18:29:17.219 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:17.220 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:17.231 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:29:17.315 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:17.311+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:17.378 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:17.375+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:17.406 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:17.406 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:17.415 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:29:17.479 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:17.476+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:17.580 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:17.577+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:17.592 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:17.592 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:17.601 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:29:17.616 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:29:17.686 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:17.682+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:17.783 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:17.779+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:17.789 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:17.790 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:17.799 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:29:17.884 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:17.880+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:17.895 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:29:17.896 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:29:17.984 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:17.981+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:18.042 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:29:18.042 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:29:18.085 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:18.081+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:18.113 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:18.113 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:18.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:18.113 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:18.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.113 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:29:18.113 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:18.113 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 142.69083646000001, 2023-12-06T18:29:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:29:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:29:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:29:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:29:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:29:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:29:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:29:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:29:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:29:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:29:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:29:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:29:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:29:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:29:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:29:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:29:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:29:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:29:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:29:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:29:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:29:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:29:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:29:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:29:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:29:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:29:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:29:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:29:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:29:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 140.58226322100001, 2023-12-06T18:29:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:29:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:29:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:29:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:29:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:29:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:29:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:29:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:29:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:29:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:29:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:29:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:29:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:29:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:29:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:29:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:29:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:29:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:29:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:29:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:29:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:29:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:29:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:29:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:29:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 140.58151798899999, 2023-12-06T18:29:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:29:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:29:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:29:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:29:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:29:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:29:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:29:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:29:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:29:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:29:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:29:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:29:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:29:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:29:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:29:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:29:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:29:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:29:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:29:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:29:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:29:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:29:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:29:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:29:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 140.58083423400001, 2023-12-06T18:29:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:29:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:29:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:29:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:29:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:29:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:29:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:29:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:29:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:29:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:29:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:29:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:29:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:29:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:29:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:29:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:29:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:29:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:29:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:29:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:29:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:29:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:29:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:29:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:29:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 140.57526097900001, 2023-12-06T18:29:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:29:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:29:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:29:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:29:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:29:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:29:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:29:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:29:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:29:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:29:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:29:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:29:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:29:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:29:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:29:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:29:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:29:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:29:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:29:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:29:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:29:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:29:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:29:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:29:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 140.574587339, 2023-12-06T18:29:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:29:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:29:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:29:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:29:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:29:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:29:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:29:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:29:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:29:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:29:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:29:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:29:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:29:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:29:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:29:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:29:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:29:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:29:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:29:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:29:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:29:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:29:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:29:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:29:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 140.57392514, 2023-12-06T18:29:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:29:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:29:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:29:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:29:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:29:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:29:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:29:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:29:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:29:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:29:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:29:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:29:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:29:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:29:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:29:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:29:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:29:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:29:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:29:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:29:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:29:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:29:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:29:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:29:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 140.50121570499999, 2023-12-06T18:29:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:29:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:29:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:29:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:29:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:29:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:29:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:29:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:29:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:29:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:29:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:29:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:29:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:29:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:29:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:29:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:29:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:29:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:29:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:29:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:29:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:29:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:29:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:29:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:29:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 140.500533996, 2023-12-06T18:29:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:29:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:29:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:29:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:29:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:29:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:29:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:29:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:29:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:29:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:29:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:29:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:29:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:29:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:29:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:29:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:29:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:29:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:29:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:29:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:29:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:29:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:29:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:29:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:29:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 139.421729908, 2023-12-06T18:29:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:29:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:29:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:29:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:29:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:29:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:29:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:29:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:29:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:29:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:29:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:29:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:29:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:29:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:29:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:29:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:29:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:29:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:29:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:29:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:29:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:29:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:29:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:29:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:29:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 139.42031333400001, 2023-12-06T18:29:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:29:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:29:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:29:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:29:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:29:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:29:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:29:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:29:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:29:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:29:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:29:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:29:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:29:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:29:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:29:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:29:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:29:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:29:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:29:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:29:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:29:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:29:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:29:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:29:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 139.41922820900001, 2023-12-06T18:29:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:29:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:29:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:29:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:29:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:29:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:29:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:29:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:29:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:29:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:29:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:29:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:29:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:29:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:29:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:29:18.172 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:29:18.172 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.172 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.172 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.172 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:29:18.172 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:29:18.172 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.172 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.172 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.172 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:29:18.172 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:29:18.172 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.172 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.172 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.172 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:29:18.172 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:29:18.173 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.173 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.173 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.173 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:29:18.173 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:29:18.173 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.173 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.173 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.173 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.173 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.173 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:18.173 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:18.173 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 139.38407910699999, 2023-12-06T18:29:18.173 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:29:18.174 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.174 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.174 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.174 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.174 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.174 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:29:18.174 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.174 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.174 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.174 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.174 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:18.174 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.174 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.174 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.174 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.175 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:18.175 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.175 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.175 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.175 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.175 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:29:18.175 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:29:18.175 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.175 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.175 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.175 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:29:18.175 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:29:18.175 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.175 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.175 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.176 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:29:18.176 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:29:18.176 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.176 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.176 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.176 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:29:18.176 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:29:18.176 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.176 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.176 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.176 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:29:18.176 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:29:18.176 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.176 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.176 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.177 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:29:18.177 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:29:18.177 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.177 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.177 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.177 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:29:18.177 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:29:18.177 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.177 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.177 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.177 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:29:18.177 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:29:18.177 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.177 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.177 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.177 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:29:18.177 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:29:18.178 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.178 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.178 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.178 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:29:18.178 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:29:18.178 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.178 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.178 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.178 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:29:18.178 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:29:18.178 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.178 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.178 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.178 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.178 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.179 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:18.179 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:18.179 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 132.83434858199999, 2023-12-06T18:29:18.179 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:29:18.179 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.179 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.179 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.179 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.179 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.179 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:29:18.179 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.179 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.179 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.179 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.179 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:18.179 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.180 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.180 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.180 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.180 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:18.180 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.180 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.180 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.180 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.180 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:29:18.180 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:29:18.180 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.180 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.180 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.180 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:29:18.180 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:29:18.180 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.180 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.181 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.181 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:29:18.181 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:29:18.181 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.181 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.181 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.181 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:29:18.181 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:29:18.181 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.181 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.181 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.181 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:29:18.181 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:29:18.181 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.181 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.181 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.181 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:29:18.182 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:29:18.182 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.182 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.182 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.182 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:29:18.182 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:29:18.182 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.182 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.182 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.182 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:29:18.182 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:29:18.182 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.182 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.182 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.182 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:29:18.183 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:29:18.183 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.183 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.183 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.183 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:29:18.183 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:29:18.183 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.183 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.183 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.183 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:29:18.183 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:29:18.183 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.183 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.183 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.183 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.183 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.183 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:18.184 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:18.184 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 132.83354297400001, 2023-12-06T18:29:18.184 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:29:18.184 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.184 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.184 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.184 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.184 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.184 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:29:18.184 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.184 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.184 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.184 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.184 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:18.184 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.184 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.184 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.185 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.185 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:18.185 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.185 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.185 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.185 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.185 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:29:18.185 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:29:18.185 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.185 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.185 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.185 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:29:18.185 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:29:18.185 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.186 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.186 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.186 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:29:18.186 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:29:18.186 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.186 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.186 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.186 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:29:18.186 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:29:18.186 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.186 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.186 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.186 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:29:18.186 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:29:18.187 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.187 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.187 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.187 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:29:18.187 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:29:18.187 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.187 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.187 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.187 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:29:18.187 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:29:18.187 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.187 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.187 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.187 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:29:18.187 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:29:18.188 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.188 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.188 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.188 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:29:18.188 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:29:18.188 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.188 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.188 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.188 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:29:18.188 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:29:18.188 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.188 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.188 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.188 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:29:18.188 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:29:18.189 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.189 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.189 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.189 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.189 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:18.189 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:18.189 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 132.83285216499999, 2023-12-06T18:29:18.189 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:29:18.189 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.189 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.189 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.189 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.189 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.189 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:29:18.190 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.190 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.190 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.190 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.190 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:18.190 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.190 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.190 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.190 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.190 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:18.190 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.190 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.190 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.190 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.190 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:29:18.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:29:18.191 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.191 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.191 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.191 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:29:18.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:29:18.191 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.191 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.191 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.191 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:29:18.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:29:18.191 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.191 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.191 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.191 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:29:18.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:29:18.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.192 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.192 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.192 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:29:18.192 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:29:18.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.192 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.192 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.192 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:29:18.192 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:29:18.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.192 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.192 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.192 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:29:18.192 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:29:18.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.192 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.193 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.193 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:29:18.193 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:29:18.193 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.193 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.193 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.193 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:29:18.193 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:29:18.193 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.193 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.193 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.193 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:29:18.193 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:29:18.193 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.193 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.193 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.193 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:29:18.194 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:29:18.194 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.194 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.194 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.194 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.194 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.194 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:18.194 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:18.194 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 132.82855465700001, 2023-12-06T18:29:18.194 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:29:18.194 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.194 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.194 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.194 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.194 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.195 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:29:18.195 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.195 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.195 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.195 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.195 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:18.195 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.195 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.195 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.195 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.195 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:18.195 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.195 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.195 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.195 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.195 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:29:18.196 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:29:18.196 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.196 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.196 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.196 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:29:18.196 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:29:18.196 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.196 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.196 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.196 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:29:18.196 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:29:18.196 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.196 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.196 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.196 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:29:18.197 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:29:18.197 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.197 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.197 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.197 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:29:18.197 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:29:18.197 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.197 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.197 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.197 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:29:18.197 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:29:18.197 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.197 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.197 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.198 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:29:18.198 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:29:18.198 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.198 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.198 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.198 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:29:18.198 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:29:18.198 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.198 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.198 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.198 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:29:18.198 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:29:18.198 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.198 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.198 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.198 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:29:18.199 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:29:18.199 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.199 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.199 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.199 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:29:18.199 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:29:18.199 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.199 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.199 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.199 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.199 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.199 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:18.199 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:18.199 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 132.827886982, 2023-12-06T18:29:18.199 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:29:18.200 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.200 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.200 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.200 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.200 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.200 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:29:18.200 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.200 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.200 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.200 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.200 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:18.200 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.200 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.200 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.200 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.201 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:18.201 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.201 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.201 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.201 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.201 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:29:18.201 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:29:18.201 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.201 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.201 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.201 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:29:18.201 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:29:18.201 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.201 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.201 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.201 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:29:18.202 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:29:18.202 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.202 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.202 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.202 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:29:18.202 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:29:18.202 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.202 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.202 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.202 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:29:18.202 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:29:18.202 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.202 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.202 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.202 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:29:18.203 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:29:18.203 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.203 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.203 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.203 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:29:18.203 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:29:18.203 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.203 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.203 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.203 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:29:18.203 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:29:18.203 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.203 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.203 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.203 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:29:18.203 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:29:18.203 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.204 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.204 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.204 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:29:18.204 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:29:18.204 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.204 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.204 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.204 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:29:18.204 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:29:18.204 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.204 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.204 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.204 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.204 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.204 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:18.204 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:18.205 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 132.82723155400001, 2023-12-06T18:29:18.205 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:29:18.205 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.205 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.205 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.205 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.205 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.205 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:29:18.205 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.205 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.205 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.205 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.205 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:18.205 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.205 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.205 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.206 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.206 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:18.206 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.206 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.206 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.206 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.206 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:29:18.206 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:29:18.206 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.206 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.206 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.206 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:29:18.206 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:29:18.206 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.206 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.207 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.207 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:29:18.207 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:29:18.207 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.207 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.207 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.207 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:29:18.207 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:29:18.207 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.207 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.207 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.207 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:29:18.207 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:29:18.207 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.207 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.207 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.208 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:29:18.208 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:29:18.208 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.208 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.208 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.208 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:29:18.208 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:29:18.208 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.208 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.208 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.208 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:29:18.208 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:29:18.208 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.208 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.208 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.209 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:29:18.209 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:29:18.209 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.209 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.209 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.209 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:29:18.209 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:29:18.209 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.209 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.209 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.209 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:29:18.209 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:29:18.209 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.209 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.209 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.209 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.210 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.210 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:18.210 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:18.210 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 132.78657700700001, 2023-12-06T18:29:18.210 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:29:18.210 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.210 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:18.210 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:18.210 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:18.210 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:18.210 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:29:18.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.210 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.210 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.210 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.211 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:18.211 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.211 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.211 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.211 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.211 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:18.211 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.211 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.211 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.211 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.211 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:29:18.211 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:29:18.211 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.211 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.211 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.211 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:29:18.212 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:29:18.212 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.212 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.212 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.212 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:29:18.212 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:29:18.212 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.212 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.212 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.212 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:29:18.212 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:29:18.212 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.212 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.212 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.212 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:29:18.213 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:29:18.213 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.213 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.213 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.213 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:29:18.213 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:29:18.213 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.213 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.213 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:18.213 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:29:18.213 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:29:18.213 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.213 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.213 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:18.213 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:29:18.213 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:29:18.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.214 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.214 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:18.214 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:29:18.214 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:29:18.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.214 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.214 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:18.214 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:29:18.214 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:29:18.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.214 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.214 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.214 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:29:18.214 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:29:18.215 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.215 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.215 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.215 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.215 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.215 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:18.216 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:29:18.217 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:29:18.217 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:29:18.217 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:18.213+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:18.247 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:29:18.247 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:29:18.317 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:18.313+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:18.396 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:18.396 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:18.397 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:18.397 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:18.397 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:29:18.397 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:29:18.406 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:29:18.418 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:18.414+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:18.490 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:29:18.491 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:29:18.518 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:18.515+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:18.546 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:18.546 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:18.546 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:18.546 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:18.546 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:18.555 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:29:18.597 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:29:18.598 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:29:18.600 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:29:18.605 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:29:18.606 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:29:18.647 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:18.643+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:18.721 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:18.717+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:18.739 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:18.739 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:18.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:18.739 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:18.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.740 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:29:18.740 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:18.740 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 116.923582525, 2023-12-06T18:29:18.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:29:18.740 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:18.740 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:29:18.740 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:18.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:18.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:18.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:18.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:18.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:18.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:18.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:29:18.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:29:18.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:18.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:29:18.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:29:18.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:18.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:29:18.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:29:18.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:18.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:29:18.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:29:18.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:18.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:29:18.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:29:18.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:18.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:29:18.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:29:18.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:18.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:18.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:18.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:29:18.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:29:18.744 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.744 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.744 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.744 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:18.744 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:18.744 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:18.748 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:29:18.833 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:18.829+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:18.923 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:18.920+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:18.926 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:18.927 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:18.927 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:18.927 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:18.936 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:29:19.008 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-12-06T18:29:19.065 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:19.061+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:19.125 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:19.121+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:19.135 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:19.136 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:19.136 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:19.136 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:19.136 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:19.145 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:29:19.230 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:19.227+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:19.324 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:19.324 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:19.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:19.324 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:29:19.325 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:19.326 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:19.322+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:19.333 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:29:19.428 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:19.424+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:19.484 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:19.485 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:19.485 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:19.485 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:19.494 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:29:19.528 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:19.525+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:19.603 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:07.537 99450== Warning: unimplemented fcntl command: 1038 2023-12-06T18:29:19.623 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:19.623 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:19.623 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:19.624 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:19.624 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:19.629 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:19.625+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:19.633 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:29:19.668 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:07.601 99450== Warning: unimplemented fcntl command: 1038 2023-12-06T18:29:19.731 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:19.728+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:19.763 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:19.764 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:19.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:19.764 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:19.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:19.764 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:19.764 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:19.764 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 58.270496626000003, 2023-12-06T18:29:19.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:29:19.764 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:19.764 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:29:19.764 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:29:19.764 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:29:19.764 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:19.764 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:29:19.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:19.765 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:19.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:19.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:19.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:19.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:19.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:19.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:19.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:19.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:19.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:19.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:19.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:19.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:19.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:29:19.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:29:19.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:19.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:19.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:19.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:29:19.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:29:19.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:19.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:19.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:19.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:29:19.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:29:19.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:19.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:19.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:19.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:29:19.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:29:19.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:19.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:19.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:19.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:29:19.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:29:19.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:19.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:19.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:19.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:29:19.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:29:19.767 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:19.767 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:19.768 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:19.768 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:19.768 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:19.768 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:19.773 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:29:19.831 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:19.828+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:19.932 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:19.929+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:19.944 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:19.944 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:19.944 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:19.944 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:19.953 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:29:19.966 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:29:20.034 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:20.030+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:20.085 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:20.085 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:20.085 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:20.085 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:20.085 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:20.094 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:29:20.139 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:20.135+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:20.225 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:29:20.216+0000 5dc2340 -1 Falling back to public interface 2023-12-06T18:29:20.235 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:20.231+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:20.265 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:20.265 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:20.265 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:20.265 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:20.265 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.265 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:29:20.265 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:20.265 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 143.64803434300001, 2023-12-06T18:29:20.266 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:29:20.266 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.266 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.266 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.266 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.266 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.266 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:20.266 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.266 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.266 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.266 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.266 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:20.266 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.266 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.267 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.267 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.267 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:29:20.267 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:29:20.267 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.267 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.267 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.267 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:29:20.267 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:29:20.267 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.267 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.267 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.267 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:29:20.267 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:29:20.267 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.268 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.268 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.268 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:29:20.268 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:29:20.268 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.268 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.268 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.268 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:29:20.268 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:29:20.268 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.268 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.268 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.268 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:29:20.268 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:29:20.268 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.269 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.269 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.269 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:29:20.269 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:29:20.269 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.269 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.269 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.269 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:29:20.269 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:29:20.269 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.269 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.269 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.269 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.269 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.269 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:29:20.269 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:20.270 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 142.62520733400001, 2023-12-06T18:29:20.270 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:29:20.270 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.270 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.270 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.270 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.270 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.270 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:20.270 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.270 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.270 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.270 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.270 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:20.270 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.270 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.271 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.271 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.271 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:29:20.271 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:29:20.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.271 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.271 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.271 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:29:20.271 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:29:20.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.271 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.271 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.271 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:29:20.271 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:29:20.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.271 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.272 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.272 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:29:20.272 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:29:20.272 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.272 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.272 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.272 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:29:20.272 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:29:20.272 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.272 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.272 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.272 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:29:20.272 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:29:20.272 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.272 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.273 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.273 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:29:20.273 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:29:20.273 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.273 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.273 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.273 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:29:20.273 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:29:20.273 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.273 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.273 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.273 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.273 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.273 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:29:20.273 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:20.274 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 142.61316069599999, 2023-12-06T18:29:20.274 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:29:20.274 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.274 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.274 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.274 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.274 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.274 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:20.274 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.274 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.274 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.274 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.274 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:20.274 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.274 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.275 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.275 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.275 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:29:20.275 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:29:20.275 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.275 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.275 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.275 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:29:20.275 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:29:20.275 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.275 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.275 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.275 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:29:20.275 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:29:20.275 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.275 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.276 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.276 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:29:20.276 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:29:20.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.276 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.276 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.276 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:29:20.276 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:29:20.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.276 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.276 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.276 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:29:20.276 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:29:20.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.276 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.277 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.277 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:29:20.277 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:29:20.277 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.277 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.277 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.277 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:29:20.277 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:29:20.277 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.277 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.277 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.277 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.277 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.277 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:29:20.277 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:20.277 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 142.59176008899999, 2023-12-06T18:29:20.278 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:29:20.278 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.278 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.278 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.278 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.278 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.278 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:20.278 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.278 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.278 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.278 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.278 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:20.278 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.279 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.279 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.279 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:29:20.279 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:29:20.279 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.279 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.279 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:29:20.279 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:29:20.279 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.279 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.279 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:29:20.279 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:29:20.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.280 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.280 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:29:20.280 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:29:20.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.280 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.280 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:29:20.280 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:29:20.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.280 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.280 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:29:20.280 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:29:20.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.281 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:29:20.281 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:29:20.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.281 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:29:20.281 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:29:20.281 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.281 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.281 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.281 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:29:20.281 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:20.282 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 142.59058076599999, 2023-12-06T18:29:20.282 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:29:20.282 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.282 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.282 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.282 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.282 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:20.282 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.282 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.282 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:20.282 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.283 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.283 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.283 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:29:20.283 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:29:20.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.283 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.283 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.283 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:29:20.283 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:29:20.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.283 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.283 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.283 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:29:20.283 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:29:20.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.284 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.284 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.284 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:29:20.284 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:29:20.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.284 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.284 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.284 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:29:20.284 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:29:20.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.284 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.284 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.284 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:29:20.284 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:29:20.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.284 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:29:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:29:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:29:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:29:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:29:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:20.285 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 142.59000878500001, 2023-12-06T18:29:20.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:29:20.286 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.286 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.286 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:20.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:20.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:29:20.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:29:20.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:29:20.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:29:20.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:29:20.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:29:20.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.288 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.288 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:29:20.288 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:29:20.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.288 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.288 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.288 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:29:20.288 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:29:20.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.288 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.288 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.288 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:29:20.288 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:29:20.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.288 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.289 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.289 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:29:20.289 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:29:20.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.289 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.289 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.289 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:29:20.289 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:29:20.289 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.289 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.289 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.289 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.289 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:29:20.289 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:20.289 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 142.14417331499999, 2023-12-06T18:29:20.289 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:29:20.290 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.290 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.290 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.290 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.290 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:20.290 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.290 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.290 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.290 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:20.290 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.290 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.290 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:29:20.291 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:29:20.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:29:20.291 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:29:20.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:29:20.291 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:29:20.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:29:20.292 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:29:20.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.292 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.292 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.292 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:29:20.292 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:29:20.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.292 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.292 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.292 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:29:20.292 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:29:20.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.292 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.292 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.292 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:29:20.292 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:29:20.293 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.293 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.293 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.293 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:29:20.293 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:29:20.293 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.293 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.293 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.293 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.293 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.293 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:29:20.293 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:20.293 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 142.142573459, 2023-12-06T18:29:20.293 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:29:20.293 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.293 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.294 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.294 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.294 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.294 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:20.294 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.294 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.294 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.294 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.294 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:20.294 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.294 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.294 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.294 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.294 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:29:20.295 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:29:20.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.295 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.295 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.295 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:29:20.295 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:29:20.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.295 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.295 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.295 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:29:20.295 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:29:20.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.295 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.295 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.295 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:29:20.295 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:29:20.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.296 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.296 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.296 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:29:20.296 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:29:20.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.296 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.296 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.296 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:29:20.296 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:29:20.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.296 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.296 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.296 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:29:20.296 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:29:20.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.297 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.297 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.297 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:29:20.297 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:29:20.297 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.297 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.297 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.297 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.297 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:29:20.297 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:20.297 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 141.763367843, 2023-12-06T18:29:20.297 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:29:20.297 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.297 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.298 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.298 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.298 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.298 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:20.298 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.298 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.298 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.298 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:20.298 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.298 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.298 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.298 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:29:20.298 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:29:20.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.299 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.299 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:29:20.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:29:20.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.299 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.299 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:29:20.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:29:20.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.299 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.299 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:29:20.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:29:20.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:29:20.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:29:20.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:29:20.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:29:20.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:29:20.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:29:20.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.301 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.301 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:29:20.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:29:20.301 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.301 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.301 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.301 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:29:20.301 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:20.301 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 141.52586566100001, 2023-12-06T18:29:20.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:29:20.301 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.301 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.301 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.301 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.302 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:20.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.302 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.302 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:20.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.302 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.302 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:29:20.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:29:20.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.302 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:29:20.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:29:20.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:29:20.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:29:20.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:29:20.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:29:20.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:29:20.304 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:29:20.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.304 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:29:20.304 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:29:20.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.304 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:29:20.304 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:29:20.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.305 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.305 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:29:20.305 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:29:20.305 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.305 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.305 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.305 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.305 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:29:20.305 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:20.305 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 141.52513766199999, 2023-12-06T18:29:20.305 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:29:20.305 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.305 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.305 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.306 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.306 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.306 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:20.306 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.306 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.306 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.306 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:20.306 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.306 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.306 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.306 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:29:20.306 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:29:20.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.306 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.307 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.307 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:29:20.307 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:29:20.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.307 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.307 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:29:20.307 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:29:20.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.307 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.307 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:29:20.307 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:29:20.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.307 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.307 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:29:20.308 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:29:20.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.308 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.308 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:29:20.308 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:29:20.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.308 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.308 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:29:20.308 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:29:20.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.308 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.308 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:29:20.309 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:29:20.309 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.309 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.309 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.309 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.309 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:29:20.309 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:20.309 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 141.52012525200001, 2023-12-06T18:29:20.309 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:29:20.309 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.309 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.309 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.309 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.309 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.309 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:20.310 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.310 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.310 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.310 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:20.310 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.310 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.310 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.310 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:29:20.310 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:29:20.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.310 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.310 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.311 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:29:20.311 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:29:20.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.311 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.311 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:29:20.311 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:29:20.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.311 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.311 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:29:20.311 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:29:20.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.311 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.311 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:29:20.312 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:29:20.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.312 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.312 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.312 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:29:20.312 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:29:20.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.312 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.312 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.312 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:29:20.312 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:29:20.312 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.312 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.312 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.312 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:29:20.312 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:29:20.313 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.313 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.313 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.313 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:29:20.313 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:20.313 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 141.514598173, 2023-12-06T18:29:20.313 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:29:20.313 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.313 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.313 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.313 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.313 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:20.313 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.314 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.314 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:20.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.314 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.314 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:29:20.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:29:20.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.314 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.314 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:29:20.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:29:20.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:29:20.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:29:20.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:29:20.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:29:20.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:29:20.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:29:20.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.316 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:29:20.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:29:20.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.316 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:29:20.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:29:20.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.316 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:29:20.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:29:20.316 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.316 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.316 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.317 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.317 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.317 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:29:20.317 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:20.317 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 141.51408733599999, 2023-12-06T18:29:20.317 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:29:20.317 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.317 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.317 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.317 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.317 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.317 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:20.317 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.317 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.317 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.318 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.318 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:20.318 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.318 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.318 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.318 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.318 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:29:20.318 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:29:20.318 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.318 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.318 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.318 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:29:20.318 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:29:20.318 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.318 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.319 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.319 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:29:20.319 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:29:20.319 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.319 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.319 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.319 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:29:20.319 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:29:20.319 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.319 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.319 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.319 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:29:20.319 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:29:20.319 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.319 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.319 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.320 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:29:20.320 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:29:20.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.320 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.320 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.320 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:29:20.320 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:29:20.320 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.320 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.320 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.320 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:29:20.320 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:29:20.320 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.320 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.320 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.321 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.321 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.321 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:29:20.321 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:20.321 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 141.51362807500001, 2023-12-06T18:29:20.321 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:29:20.321 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.321 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.321 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.321 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.321 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.321 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:20.321 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.321 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.321 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.321 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.322 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:20.322 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.322 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.322 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.322 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.322 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:29:20.322 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:29:20.322 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.322 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.322 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.322 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:29:20.322 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:29:20.322 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.322 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.322 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.323 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:29:20.323 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:29:20.323 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.323 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.323 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.323 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:29:20.323 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:29:20.323 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.323 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.323 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.323 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:29:20.323 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:29:20.323 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.323 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.323 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.323 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:29:20.324 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:29:20.324 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.324 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.324 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.324 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:29:20.324 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:29:20.324 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.324 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.324 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.324 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:29:20.324 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:29:20.324 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.324 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.324 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.324 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.325 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.325 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:29:20.325 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:20.325 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 134.93371226799999, 2023-12-06T18:29:20.325 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:29:20.325 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.325 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.325 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.325 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.325 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.325 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:20.325 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.325 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.325 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.325 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:20.326 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.326 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.326 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.326 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.326 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:29:20.326 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:29:20.326 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.326 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.326 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.326 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:29:20.326 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:29:20.326 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.326 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.326 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.326 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:29:20.326 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:29:20.327 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.327 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.327 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.327 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:29:20.327 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:29:20.327 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.327 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.327 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.327 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:29:20.327 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:29:20.327 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.327 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.327 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.327 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:29:20.327 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:29:20.327 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.327 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.328 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.328 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:29:20.328 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:29:20.328 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.328 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.328 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.328 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:29:20.328 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:29:20.328 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.328 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.328 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.328 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.328 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.328 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:29:20.328 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:20.329 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 134.92747446300001, 2023-12-06T18:29:20.329 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:29:20.329 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.329 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.329 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.329 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.329 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.329 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:20.329 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.329 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.329 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.329 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.329 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:20.329 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:29:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:29:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:29:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:29:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:29:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:29:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:29:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:29:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:29:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:29:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:29:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:29:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:29:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:29:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:29:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:29:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:29:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 134.92141138400001, 2023-12-06T18:29:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:29:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:29:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:29:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:29:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:29:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:29:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:29:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.335 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.335 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:29:20.335 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:29:20.335 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.335 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.335 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.335 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:29:20.335 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:29:20.335 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.335 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.335 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.335 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:29:20.335 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:29:20.335 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.335 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.336 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.336 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:29:20.336 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:29:20.336 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.336 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.336 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.336 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:29:20.336 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:29:20.336 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.336 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.336 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.336 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.336 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.336 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:29:20.336 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:20.336 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 134.78402429900001, 2023-12-06T18:29:20.336 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:29:20.337 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.337 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.337 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.337 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.337 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.337 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:20.337 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.337 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.337 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.337 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.337 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:20.337 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.337 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.337 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.337 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.338 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:29:20.338 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:29:20.338 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.338 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.338 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.338 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:29:20.338 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:29:20.338 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.338 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.338 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.338 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:29:20.338 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:29:20.338 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.338 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.338 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.339 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:29:20.339 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:29:20.339 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.339 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.339 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.339 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:29:20.339 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:29:20.339 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.339 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.339 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.339 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:29:20.339 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:29:20.339 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.339 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.339 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.339 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:29:20.340 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:29:20.340 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.340 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.340 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.340 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:29:20.340 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:29:20.340 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.340 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.340 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.340 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.340 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.340 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:29:20.340 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:20.340 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 134.77735357200001, 2023-12-06T18:29:20.340 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:29:20.341 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:20.341 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:20.341 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:20.341 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.341 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:20.341 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:20.341 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.341 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.341 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.341 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:20.341 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:20.341 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:20.341 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.341 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.341 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:20.342 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:29:20.342 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:29:20.342 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.342 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.342 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:20.342 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:29:20.342 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:29:20.342 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.342 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.342 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:20.342 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:29:20.342 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:29:20.342 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.342 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.342 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:20.343 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:29:20.343 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:29:20.343 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.343 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.343 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:20.343 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:29:20.343 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:29:20.343 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.343 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.343 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:20.343 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:29:20.343 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:29:20.343 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.343 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.343 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:20.343 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:29:20.344 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:29:20.344 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:20.344 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:20.344 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:20.344 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:29:20.344 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:29:20.344 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.344 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.344 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.344 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:20.344 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:20.344 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:20.346 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:29:20.346 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:20.342+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:20.446 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:20.442+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:20.514 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:20.514 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:20.515 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:20.515 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:20.523 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:29:20.545 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is set 2023-12-06T18:29:20.546 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:20.543+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:20.607 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:29:20.627 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:08.561 99450== Warning: unimplemented fcntl command: 1038 2023-12-06T18:29:20.647 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:20.644+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:20.650 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":22,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":35,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:29:20.655 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:20.655 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:20.655 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:20.655 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:20.655 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:20.666 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:29:20.748 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:20.745+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:20.835 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:20.835 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:20.835 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:20.835 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:29:20.835 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:20.845 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:29:20.870 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:29:20.871 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:29:20.871 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:20.868+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:20.951 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:20.947+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:20.967 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:29:20.968 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:29:20.985 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:08.919 99450== Warning: unimplemented fcntl command: 1038 2023-12-06T18:29:21.011 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:21.011 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:21.021 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:29:21.052 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:21.049+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:21.066 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:29:21.066 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:29:21.152 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:21.148+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:21.165 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:29:21.165 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:29:21.202 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:21.202 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:21.211 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:29:21.253 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:29:21.253 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:29:21.253 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:21.250+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:21.304 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:09.238 99450== Warning: unimplemented fcntl command: 1038 2023-12-06T18:29:21.340 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:29:21.340 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:29:21.353 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:21.350+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:21.411 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:21.411 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:21.420 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:29:21.431 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:29:21.431 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:29:21.434 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:29:21.439 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:29:21.440 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:29:21.454 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:21.451+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:21.556 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:21.553+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:21.567 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:21.567 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:21.571 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:09.505 99450== Warning: unimplemented fcntl command: 1038 2023-12-06T18:29:21.578 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:29:21.657 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:21.653+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:21.754 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:21.754 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:21.758 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:21.754+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:21.764 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:29:21.835 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:09.769 99450== Warning: unimplemented fcntl command: 1038 2023-12-06T18:29:21.859 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:21.855+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:21.902 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:21.903 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:21.912 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:29:21.959 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:21.956+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:22.049 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:22.049 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:22.058 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:29:22.060 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:22.057+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:22.116 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:10.050 99450== Warning: unimplemented fcntl command: 1038 2023-12-06T18:29:22.161 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:22.158+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:22.194 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:22.194 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:22.202 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:29:22.203 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:29:22.288 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:22.285+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:22.362 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:22.359+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:22.383 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:22.383 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:22.386 INFO:teuthology.orchestra.run.smithi152.stderr:no valid command found; 10 closest matches: 2023-12-06T18:29:22.386 INFO:teuthology.orchestra.run.smithi152.stderr:0 2023-12-06T18:29:22.386 INFO:teuthology.orchestra.run.smithi152.stderr:1 2023-12-06T18:29:22.386 INFO:teuthology.orchestra.run.smithi152.stderr:2 2023-12-06T18:29:22.386 INFO:teuthology.orchestra.run.smithi152.stderr:abort 2023-12-06T18:29:22.386 INFO:teuthology.orchestra.run.smithi152.stderr:assert 2023-12-06T18:29:22.386 INFO:teuthology.orchestra.run.smithi152.stderr:config diff 2023-12-06T18:29:22.386 INFO:teuthology.orchestra.run.smithi152.stderr:config diff get 2023-12-06T18:29:22.386 INFO:teuthology.orchestra.run.smithi152.stderr:config get 2023-12-06T18:29:22.387 INFO:teuthology.orchestra.run.smithi152.stderr:config help [] 2023-12-06T18:29:22.387 INFO:teuthology.orchestra.run.smithi152.stderr:config set ... 2023-12-06T18:29:22.387 INFO:teuthology.orchestra.run.smithi152.stderr:admin_socket: invalid command 2023-12-06T18:29:22.388 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-12-06T18:29:22.389 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-7, ['dump_ops_in_flight'] 2023-12-06T18:29:22.393 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:29:22.412 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:10.345 99450== Warning: unimplemented fcntl command: 1038 2023-12-06T18:29:22.463 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:22.460+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:22.564 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-12-06T18:29:22.565 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:22.562+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:22.573 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:22.573 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:22.583 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:29:22.665 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:22.662+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:22.700 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:10.634 99450== Warning: unimplemented fcntl command: 1038 2023-12-06T18:29:22.767 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:22.764+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:22.771 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:22.771 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:22.780 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:29:22.868 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:22.865+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:22.969 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:22.965+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:22.977 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:22.977 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:22.987 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:29:22.995 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:10.929 99450== Warning: unimplemented fcntl command: 1038 2023-12-06T18:29:23.070 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:23.067+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:23.115 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:23.115 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:23.125 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:29:23.209 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:23.206+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:23.272 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:23.269+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:23.295 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:23.296 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:23.305 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:29:23.389 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:23.386+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:23.441 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:29:23.474 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:23.471+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:23.487 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:23.488 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:23.499 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:29:23.539 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:11.473 99450== Warning: unimplemented fcntl command: 1038 2023-12-06T18:29:23.575 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:23.573+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:23.594 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:11.528 99450== Warning: unimplemented fcntl command: 1038 2023-12-06T18:29:23.666 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:29:23.670 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:23.670 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:23.676 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:23.672+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:23.679 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:29:23.776 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:23.773+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:23.871 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:23.871 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:23.877 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:23.874+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:23.881 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:29:24.008 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:24.005+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:24.038 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:24.038 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:24.051 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:29:24.078 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:24.076+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:24.179 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:24.176+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:24.239 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:24.239 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:24.249 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:29:24.280 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:24.277+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:24.293 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:29:24.293 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:29:24.381 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:24.379+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:24.385 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":28,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"c","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:29:24.402 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2023-12-06T18:29:24.402 INFO:tasks.mon_recovery:stopping all monitors 2023-12-06T18:29:24.402 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-12-06T18:29:24.402 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:29:24.403 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:29:24.403 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:29:24.417 INFO:tasks.ceph.mon.b.smithi152.stderr:2023-12-06T18:29:24.403+0000 9f54640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 98182) UID: 0 2023-12-06T18:29:24.419 INFO:tasks.ceph.mon.b.smithi152.stderr:2023-12-06T18:29:24.404+0000 9f54640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2023-12-06T18:29:24.428 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:24.428 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:24.440 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:29:24.482 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:24.478+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:24.497 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:29:24.497 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:29:24.582 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:24.579+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:24.588 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:29:24.588 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:29:24.612 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:29:24.613 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:29:24.622 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:29:24.683 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:24.679+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:24.703 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:29:24.704 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:29:24.763 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:24.763 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:24.763 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:24.763 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:24.772 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:29:24.792 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:29:24.792 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:29:24.793 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:24.790+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:24.886 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:24.882+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:24.899 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:29:24.899 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:29:24.922 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:24.922 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:24.922 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:24.922 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:24.922 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:24.932 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:29:24.986 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:24.982+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:25.059 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:39.120 98186== 2023-12-06T18:29:25.059 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:39.120 98186== Exit program on first error (--exit-on-first-error=yes) 2023-12-06T18:29:25.069 INFO:tasks.ceph.mon.b.smithi152.stderr:daemon-helper: command failed with exit status 42 2023-12-06T18:29:25.110 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:25.111 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:25.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:25.111 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:29:25.111 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:25.117 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:25.114+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:25.120 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:29:25.188 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:25.185+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:25.257 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:25.257 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:25.257 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:25.257 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:25.267 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:29:25.308 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:25.305+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:25.389 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:25.387+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:25.443 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:25.443 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:25.444 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:25.444 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:25.444 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:25.452 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:29:25.495 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:25.491+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:25.590 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:25.588+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:25.688 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:25.688 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:25.688 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:25.688 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:25.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.689 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:29:25.689 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:25.689 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 150.27824186199999, 2023-12-06T18:29:25.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:29:25.689 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.689 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.689 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.689 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.689 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.689 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:29:25.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.689 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.689 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.689 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:25.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:25.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:29:25.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:29:25.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:29:25.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:29:25.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:29:25.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:29:25.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:29:25.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:29:25.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:29:25.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:29:25.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:29:25.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:29:25.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:29:25.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:29:25.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:29:25.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:29:25.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:29:25.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:29:25.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:29:25.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:29:25.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:29:25.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:29:25.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:29:25.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:29:25.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:29:25.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:29:25.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:29:25.694 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.694 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.694 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.694 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:25.694 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:25.694 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 148.16966862300001, 2023-12-06T18:29:25.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:29:25.694 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.694 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.694 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.695 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.695 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.695 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:29:25.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.695 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:25.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:25.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:29:25.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:29:25.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:29:25.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:29:25.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:29:25.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:29:25.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:29:25.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:29:25.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:29:25.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:29:25.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:29:25.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:29:25.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:29:25.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:29:25.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:29:25.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:29:25.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:29:25.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:29:25.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:29:25.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:29:25.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:29:25.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:29:25.699 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.699 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.699 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.700 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:25.700 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:25.700 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 148.16892339099999, 2023-12-06T18:29:25.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:29:25.700 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.700 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.700 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.700 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.700 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.700 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:29:25.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.700 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:25.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:25.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:29:25.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:29:25.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:29:25.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:29:25.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:29:25.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:29:25.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:29:25.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:29:25.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:29:25.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:29:25.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:29:25.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:29:25.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:29:25.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:29:25.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:29:25.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:29:25.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:29:25.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:29:25.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:29:25.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:29:25.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:29:25.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:29:25.704 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.704 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.704 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.705 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:25.705 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:25.705 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 148.16823963600001, 2023-12-06T18:29:25.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:29:25.705 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.705 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.705 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.705 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.705 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.705 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:29:25.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.705 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:25.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:25.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:29:25.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:29:25.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:29:25.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:29:25.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:29:25.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:29:25.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:29:25.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:29:25.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:29:25.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:29:25.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:29:25.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:29:25.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:29:25.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:29:25.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:29:25.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:29:25.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:29:25.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:29:25.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:29:25.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:29:25.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:29:25.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:29:25.710 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.710 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.710 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.710 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:25.710 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:25.710 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 148.16266638100001, 2023-12-06T18:29:25.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:29:25.710 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.710 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.710 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.711 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.711 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.711 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:29:25.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.711 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:25.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:25.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:29:25.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:29:25.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:29:25.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:29:25.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:29:25.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:29:25.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:29:25.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:29:25.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:29:25.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:29:25.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:29:25.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:29:25.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:29:25.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:29:25.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:29:25.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:29:25.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:29:25.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:29:25.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:29:25.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:29:25.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:29:25.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:29:25.715 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.715 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.715 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.715 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:25.715 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:25.715 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 148.16199274100001, 2023-12-06T18:29:25.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:29:25.715 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.716 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.716 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.716 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.716 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.716 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:29:25.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.716 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:25.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:25.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:29:25.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:29:25.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:29:25.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:29:25.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:29:25.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:29:25.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:29:25.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:29:25.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:29:25.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:29:25.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:29:25.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:29:25.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:29:25.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:29:25.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:29:25.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:29:25.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:29:25.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:29:25.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:29:25.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:29:25.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:29:25.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:29:25.720 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.720 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.720 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.721 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:25.721 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:25.721 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 148.161330542, 2023-12-06T18:29:25.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:29:25.721 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.721 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.721 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.721 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.721 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.721 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:29:25.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.722 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:25.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:25.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:29:25.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:29:25.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:29:25.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:29:25.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:29:25.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:29:25.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:29:25.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:29:25.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:29:25.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:29:25.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:29:25.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:29:25.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:29:25.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:29:25.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:29:25.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:29:25.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:29:25.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:29:25.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:29:25.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:29:25.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.726 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:29:25.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:29:25.726 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.726 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.726 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.726 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:25.726 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:25.726 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 148.08862110699999, 2023-12-06T18:29:25.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:29:25.726 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.726 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.727 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.727 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.727 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.727 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:29:25.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.727 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:25.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:25.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:29:25.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:29:25.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:29:25.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:29:25.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:29:25.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:29:25.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:29:25.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:29:25.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:29:25.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:29:25.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:29:25.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:29:25.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:29:25.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:29:25.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:29:25.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:29:25.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:29:25.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:29:25.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:29:25.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:29:25.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.772 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:29:25.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:29:25.772 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.772 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.772 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.772 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:25.772 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:25.772 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 148.087939398, 2023-12-06T18:29:25.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:29:25.773 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.773 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.773 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.773 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.773 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.773 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:29:25.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.773 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:25.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:25.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:29:25.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:29:25.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:29:25.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:29:25.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:29:25.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:29:25.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:29:25.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:29:25.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.776 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:29:25.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:29:25.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.776 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:29:25.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:29:25.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:29:25.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:29:25.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:29:25.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:29:25.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:29:25.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:29:25.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:29:25.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:29:25.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:29:25.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:29:25.778 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.778 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.778 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.778 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:25.779 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:25.779 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 147.00913531, 2023-12-06T18:29:25.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:29:25.779 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.779 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.779 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.779 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.779 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.779 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:29:25.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.779 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:25.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:25.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:29:25.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:29:25.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:29:25.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:29:25.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:29:25.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:29:25.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.782 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.782 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:29:25.782 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:29:25.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.782 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.782 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:29:25.782 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:29:25.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.782 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.782 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:29:25.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:29:25.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:29:25.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:29:25.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:29:25.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:29:25.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:29:25.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:29:25.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:29:25.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:29:25.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:29:25.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:29:25.785 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.785 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.785 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.785 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:25.785 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:25.785 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 147.00771873599999, 2023-12-06T18:29:25.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:29:25.785 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.785 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.785 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.785 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.785 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.786 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:29:25.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.786 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:25.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:25.787 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.787 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.787 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:29:25.787 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:29:25.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.787 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.787 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:29:25.787 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:29:25.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.787 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.787 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:29:25.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:29:25.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.788 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:29:25.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:29:25.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.788 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:29:25.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:29:25.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:29:25.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:29:25.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:29:25.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:29:25.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:29:25.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:29:25.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.790 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.790 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:29:25.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:29:25.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.790 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.790 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:29:25.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:29:25.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:29:25.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:29:25.791 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.791 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.791 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.791 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:25.791 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:25.791 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 147.00663361100001, 2023-12-06T18:29:25.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:29:25.791 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.791 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.791 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.791 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.792 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.792 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:29:25.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.792 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.792 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.792 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:25.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.792 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.792 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:25.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:29:25.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:29:25.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:29:25.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:29:25.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:29:25.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:29:25.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:29:25.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:29:25.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:29:25.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:29:25.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:29:25.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:29:25.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:29:25.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:29:25.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:29:25.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:29:25.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:29:25.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:29:25.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:29:25.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:29:25.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:29:25.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:29:25.796 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.796 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.797 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.797 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:25.797 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:25.797 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 146.97148450899999, 2023-12-06T18:29:25.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:29:25.797 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.797 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.797 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.797 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.797 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.798 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:29:25.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.798 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:25.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:25.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:29:25.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:29:25.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:29:25.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:29:25.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:29:25.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:29:25.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:29:25.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:29:25.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:29:25.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:29:25.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:29:25.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:29:25.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:29:25.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:29:25.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:29:25.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:29:25.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.802 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:29:25.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:29:25.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.802 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:29:25.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:29:25.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:29:25.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:29:25.803 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.803 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.803 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.803 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:25.803 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:25.803 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 140.42175398399999, 2023-12-06T18:29:25.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:29:25.803 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.803 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.803 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.803 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.804 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.804 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:29:25.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.804 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:25.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:25.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:29:25.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:29:25.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:29:25.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:29:25.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:29:25.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:29:25.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:29:25.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:29:25.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:29:25.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:29:25.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:29:25.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:29:25.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:29:25.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:29:25.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:29:25.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:29:25.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:29:25.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:29:25.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:29:25.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:29:25.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.808 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.808 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:29:25.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:29:25.808 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.808 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.808 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.808 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:25.808 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:25.808 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 140.42094837600001, 2023-12-06T18:29:25.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:29:25.808 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.808 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.808 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.809 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.809 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.809 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:29:25.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.809 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:25.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:25.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:29:25.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:29:25.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:29:25.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:29:25.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:29:25.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:29:25.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:29:25.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:29:25.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:29:25.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:29:25.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:29:25.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:29:25.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:29:25.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:29:25.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:29:25.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:29:25.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:29:25.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:29:25.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:29:25.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:29:25.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.814 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:29:25.814 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:29:25.814 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.814 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.814 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.814 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:25.814 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:25.814 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 140.42025756699999, 2023-12-06T18:29:25.814 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:29:25.814 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.814 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.814 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.815 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.815 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.815 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:29:25.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.815 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.815 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.815 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.815 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:25.815 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.815 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.815 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:25.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:29:25.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:29:25.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:29:25.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:29:25.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:29:25.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:29:25.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:29:25.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:29:25.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:29:25.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:29:25.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:29:25.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:29:25.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:29:25.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:29:25.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:29:25.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:29:25.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.819 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:29:25.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:29:25.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.819 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.819 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.819 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:29:25.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:29:25.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.819 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.819 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.819 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:29:25.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:29:25.819 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.819 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.819 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.820 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.820 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:25.820 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:25.820 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 140.41596005900001, 2023-12-06T18:29:25.820 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:29:25.820 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.820 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.820 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.820 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.820 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.820 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:29:25.820 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.820 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.821 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.821 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.821 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:25.821 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.821 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.821 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.821 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.821 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:25.821 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.821 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.821 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.821 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.822 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:29:25.822 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:29:25.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.822 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.822 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.822 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:29:25.822 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:29:25.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.822 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.822 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.822 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:29:25.822 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:29:25.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.822 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.822 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:29:25.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:29:25.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:29:25.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:29:25.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:29:25.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:29:25.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:29:25.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:29:25.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.824 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:29:25.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:29:25.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.824 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:29:25.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:29:25.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.824 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:29:25.825 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:29:25.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.825 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.825 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.825 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:29:25.825 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:29:25.825 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.825 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.825 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.825 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.825 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:25.825 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:25.825 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 140.415292384, 2023-12-06T18:29:25.825 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:29:25.825 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.826 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.826 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.826 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.826 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.826 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:29:25.826 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.826 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.826 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.826 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.826 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:25.826 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.826 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.826 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.826 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.826 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:25.827 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.827 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.827 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.827 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.827 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:29:25.827 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:29:25.827 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.827 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.827 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.827 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:29:25.827 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:29:25.827 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.827 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.827 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.827 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:29:25.827 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:29:25.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.828 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.828 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:29:25.828 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:29:25.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.828 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.828 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:29:25.828 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:29:25.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.828 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.828 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:29:25.828 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:29:25.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.829 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:29:25.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:29:25.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.829 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.829 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:29:25.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:29:25.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.829 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.829 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:29:25.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:29:25.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.829 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.829 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:29:25.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:29:25.830 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.830 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.830 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.830 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:29:25.830 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:29:25.830 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.830 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.830 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.830 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.830 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.830 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:25.830 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:25.830 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 140.41463695600001, 2023-12-06T18:29:25.830 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:29:25.830 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.831 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.831 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.831 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.831 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.831 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:29:25.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.831 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.831 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.831 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.831 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:25.831 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.831 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.831 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.831 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:25.831 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.832 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:29:25.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:29:25.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.832 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:29:25.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:29:25.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.832 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:29:25.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:29:25.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.833 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.833 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.833 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:29:25.833 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:29:25.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.833 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.833 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.833 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:29:25.833 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:29:25.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.833 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.833 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.833 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:29:25.833 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:29:25.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.833 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.834 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.834 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:29:25.834 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:29:25.834 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.834 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.834 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.834 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:29:25.834 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:29:25.834 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.834 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.834 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.834 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:29:25.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:29:25.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.835 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.835 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.835 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:29:25.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:29:25.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.835 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.835 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.835 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:29:25.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:29:25.835 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.835 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.835 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.836 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.836 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:25.836 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:25.836 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 140.37398240900001, 2023-12-06T18:29:25.836 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:29:25.836 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:25.836 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:25.836 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:25.836 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:25.836 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:25.836 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:29:25.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.836 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:25.836 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.836 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:25.836 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:25.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.837 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:25.837 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:25.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:25.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.837 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:25.837 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:29:25.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:29:25.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.837 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:25.837 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:29:25.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:29:25.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:25.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:29:25.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:29:25.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:25.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:29:25.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:29:25.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:25.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:29:25.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:29:25.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:25.839 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:29:25.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:29:25.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:25.839 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:29:25.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:29:25.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:25.839 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:29:25.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:29:25.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.840 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:25.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:29:25.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:29:25.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.840 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:25.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:29:25.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:29:25.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:25.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:25.840 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:25.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:29:25.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:29:25.840 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.840 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.840 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.841 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:25.841 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:25.841 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:25.842 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:29:25.843 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:25.839+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:25.943 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:25.940+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:26.019 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:26.019 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:26.019 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:26.019 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:26.028 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:29:26.072 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:26.069+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:26.146 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:26.143+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:26.165 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:26.165 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:26.165 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:26.165 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:26.166 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:26.174 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:29:26.246 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:26.243+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:26.348 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:26.345+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:26.354 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:29:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 124.538073938, 2023-12-06T18:29:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:29:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:29:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:29:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:29:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:29:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:29:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:29:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:29:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:29:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:29:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:29:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:29:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:29:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:29:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:29:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:29:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:26.358 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:26.364 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:29:26.449 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:26.446+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:26.540 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:26.540 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:26.541 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:26.541 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:26.549 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:29:26.550 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:26.547+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:26.650 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:26.648+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:26.729 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:26.729 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:26.729 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:26.729 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:26.730 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:26.739 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:29:26.782 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:26.778+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:26.852 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:26.849+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:26.938 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:26.939 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:26.939 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:26.939 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:26.939 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.939 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:26.939 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:26.939 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 5.8309431250000001, 2023-12-06T18:29:26.939 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:29:26.939 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:26.939 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:29:26.939 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:26.939 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:29:26.939 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:26.939 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:29:26.939 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:26.940 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:26.940 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.940 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:26.940 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:26.940 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:26.940 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:26.940 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.940 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:26.940 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:26.940 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:26.940 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:26.940 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.940 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:26.940 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:29:26.940 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:29:26.940 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:29:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:29:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:29:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:29:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:29:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:29:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:29:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:29:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:29:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:29:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:26.943 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:26.943 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:26.947 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:29:26.991 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:26.988+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:27.053 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:27.050+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:27.079 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:27.079 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:27.079 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:27.079 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:27.089 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:29:27.154 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:27.152+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:27.255 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:27.251+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:27.259 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:27.259 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:27.259 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:27.259 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:27.259 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:27.268 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:29:27.355 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:27.352+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:27.390 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:29:27.441 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:27.441 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:27.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:27.441 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:27.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.441 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:27.441 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:27.441 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 65.947625403000004, 2023-12-06T18:29:27.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:29:27.442 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:27.442 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:29:27.442 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:29:27.442 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:29:27.442 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:27.442 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:29:27.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.442 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:27.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:27.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:27.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:27.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:27.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:27.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:27.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:27.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:29:27.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:29:27.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:27.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:29:27.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:29:27.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:27.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:29:27.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:29:27.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:27.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:29:27.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:29:27.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:27.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:29:27.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:29:27.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:27.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:29:27.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:29:27.445 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:27.445 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:27.445 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:27.445 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:27.445 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:27.445 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:27.451 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:29:27.494 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:27.491+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:27.558 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:27.555+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:27.583 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:27.583 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:27.583 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:27.583 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:27.594 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:29:27.678 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:27.675+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:27.760 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:27.757+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:27.763 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:27.763 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:27.763 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:27.763 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:27.763 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:27.772 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:29:27.861 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:27.858+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:27.980 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:27.980 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:27.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:27.980 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:27.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.980 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:29:27.980 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:27.980 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 151.36465193500001, 2023-12-06T18:29:27.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:29:27.980 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:27.980 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:27.980 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:27.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:27.981 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:27.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:27.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:27.981 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:27.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:27.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:27.981 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:29:27.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:29:27.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:27.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:29:27.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:29:27.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:27.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:29:27.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:29:27.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:27.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:29:27.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:29:27.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:27.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:29:27.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:29:27.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:27.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:29:27.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:29:27.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:27.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:29:27.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:29:27.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:27.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:29:27.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:29:27.984 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:27.984 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:27.984 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:27.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.985 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:29:27.985 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:27.985 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 150.34182492599999, 2023-12-06T18:29:27.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:29:27.985 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:27.985 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:27.985 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:27.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:27.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:27.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:27.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:27.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:27.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:27.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:27.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:29:27.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:29:27.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:27.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:29:27.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:29:27.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:27.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:29:27.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:29:27.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:27.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:29:27.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:29:27.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:27.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:29:27.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:29:27.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:27.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:29:27.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:29:27.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:27.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:29:27.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:29:27.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:27.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:29:27.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:29:27.989 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:27.989 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:27.989 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:27.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.989 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:29:27.989 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:27.989 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 150.329778288, 2023-12-06T18:29:27.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:29:27.989 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:27.989 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:27.989 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:27.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:27.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:27.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:27.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:27.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:27.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:27.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:27.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:29:27.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:29:27.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:27.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:29:27.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:29:27.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:27.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:29:27.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:29:27.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:27.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:29:27.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:29:27.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:27.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:29:27.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:29:27.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:27.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:29:27.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:29:27.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:27.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:29:27.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:29:27.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:27.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:29:27.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:29:27.993 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:27.993 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:27.993 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:27.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.994 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:29:27.994 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:27.994 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 150.308377681, 2023-12-06T18:29:27.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:29:27.994 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:27.994 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:27.994 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:27.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:27.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:27.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:27.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:27.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:27.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:27.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:27.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:29:27.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:29:27.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:27.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:29:27.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:29:27.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:27.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:29:27.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:29:27.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:27.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:29:27.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:29:27.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:27.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:29:27.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:29:27.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:27.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:29:27.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:29:27.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:27.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:29:27.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:29:27.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:27.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:29:27.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:29:27.998 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:27.998 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:27.998 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:27.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:27.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.999 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:29:27.999 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:27.999 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 150.30719835799999, 2023-12-06T18:29:27.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:29:27.999 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:27.999 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:27.999 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:27.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:27.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:27.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:28.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:28.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:28.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:28.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:29:28.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:29:28.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:28.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:29:28.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:29:28.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:28.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:29:28.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:29:28.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:28.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:29:28.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:29:28.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:28.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:29:28.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:29:28.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:28.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:29:28.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:29:28.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:28.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:29:28.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:29:28.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:28.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:29:28.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:29:28.003 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.003 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:28.003 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.003 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:29:28.003 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:28.003 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 150.30662637699999, 2023-12-06T18:29:28.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:29:28.003 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:28.003 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:28.003 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:28.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:28.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:28.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:28.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:28.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:28.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:29:28.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:29:28.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:28.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:29:28.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:29:28.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:28.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:29:28.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:29:28.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:28.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:29:28.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:29:28.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:28.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:29:28.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:29:28.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:28.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:29:28.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:29:28.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:28.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:29:28.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:29:28.008 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.008 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.008 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:28.008 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:29:28.008 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:29:28.008 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.008 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:28.008 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.008 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.008 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.009 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:29:28.009 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:28.009 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 149.86079090699999, 2023-12-06T18:29:28.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:29:28.009 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:28.009 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:28.009 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:28.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:28.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:28.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:28.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:28.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:28.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:29:28.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:29:28.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:28.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:29:28.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:29:28.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:28.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:29:28.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:29:28.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:28.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:29:28.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:29:28.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:28.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:29:28.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:29:28.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:28.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:29:28.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:29:28.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:28.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:29:28.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:29:28.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:28.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:29:28.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:29:28.012 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.012 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:28.012 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.013 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:29:28.013 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:28.013 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 149.85919105100001, 2023-12-06T18:29:28.013 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:29:28.013 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:28.013 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:28.013 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:28.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.013 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:28.013 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:28.013 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.014 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:28.014 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:28.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.014 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:28.014 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:29:28.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:29:28.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.014 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:28.014 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:29:28.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:29:28.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:28.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:29:28.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:29:28.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:28.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:29:28.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:29:28.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:28.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:29:28.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:29:28.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:28.016 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:29:28.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:29:28.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:28.016 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:29:28.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:29:28.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:28.016 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:29:28.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:29:28.016 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.017 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:28.017 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.017 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:29:28.017 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:28.017 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 149.479985435, 2023-12-06T18:29:28.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:29:28.017 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:28.017 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:28.017 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:28.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:28.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:28.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:28.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:28.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:28.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:29:28.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:29:28.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:28.019 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:29:28.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:29:28.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:28.019 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:29:28.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:29:28.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:28.019 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:29:28.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:29:28.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:28.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:29:28.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:29:28.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:28.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:29:28.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:29:28.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:28.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:29:28.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:29:28.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.021 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:28.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:29:28.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:29:28.021 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.021 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:28.021 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.021 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:29:28.021 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:28.021 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 149.24248325299999, 2023-12-06T18:29:28.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:29:28.021 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:28.021 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:28.021 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:28.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:28.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:28.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:28.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:28.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:28.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:29:28.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:29:28.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:28.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:29:28.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:29:28.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:28.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:29:28.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:29:28.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:28.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:29:28.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:29:28.024 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.024 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:28.024 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:29:28.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:29:28.024 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.024 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:28.024 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:29:28.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:29:28.024 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.024 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:28.024 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:29:28.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:29:28.025 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.025 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:28.025 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:29:28.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:29:28.025 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.025 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:28.025 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.025 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.025 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:29:28.025 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:28.025 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 149.241755254, 2023-12-06T18:29:28.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:29:28.025 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:28.026 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:28.026 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:28.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:28.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:28.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:28.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:28.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:28.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:29:28.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:29:28.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:28.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:29:28.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:29:28.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:28.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:29:28.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:29:28.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:28.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:29:28.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:29:28.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:28.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:29:28.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:29:28.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:28.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:29:28.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:29:28.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.029 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:28.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:29:28.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:29:28.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.029 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:28.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:29:28.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:29:28.029 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.029 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:28.029 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.030 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:29:28.030 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:28.030 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 149.23674284399999, 2023-12-06T18:29:28.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:29:28.030 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:28.030 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:28.030 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:28.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:28.030 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:28.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:28.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:28.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:28.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:29:28.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:29:28.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:28.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:29:28.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:29:28.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:28.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:29:28.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:29:28.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:28.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:29:28.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:29:28.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:28.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:29:28.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:29:28.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:28.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:29:28.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:29:28.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:28.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:29:28.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:29:28.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:28.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:29:28.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:29:28.033 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.033 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:28.033 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.034 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:29:28.034 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:28.034 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 149.231215765, 2023-12-06T18:29:28.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:29:28.034 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:28.034 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:28.034 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:28.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.034 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:28.034 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:28.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.034 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:28.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:28.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:28.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:29:28.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:29:28.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:28.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:29:28.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:29:28.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:28.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:29:28.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:29:28.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:28.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:29:28.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:29:28.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:28.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:29:28.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:29:28.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:28.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:29:28.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:29:28.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:28.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:29:28.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:29:28.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:28.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:29:28.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:29:28.037 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.037 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:28.037 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.038 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:29:28.038 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:28.038 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 149.23070492799999, 2023-12-06T18:29:28.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:29:28.038 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:28.038 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:28.038 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:28.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:28.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:28.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:28.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:28.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:28.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:29:28.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:29:28.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:28.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:29:28.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:29:28.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:28.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:29:28.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:29:28.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:28.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:29:28.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:29:28.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:28.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:29:28.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:29:28.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:28.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:29:28.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:29:28.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:28.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:29:28.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:29:28.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:28.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:29:28.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:29:28.042 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.042 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:28.042 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.042 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:29:28.042 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:28.042 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 149.23024566699999, 2023-12-06T18:29:28.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:29:28.042 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:28.042 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:28.042 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:28.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:28.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:28.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:28.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:28.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:28.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:29:28.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:29:28.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:28.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:29:28.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:29:28.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:28.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:29:28.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:29:28.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:28.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:29:28.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:29:28.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:28.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:29:28.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:29:28.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:28.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:29:28.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:29:28.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:28.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:29:28.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:29:28.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:28.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:29:28.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:29:28.046 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.046 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:28.046 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.046 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:29:28.046 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:28.046 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 142.65032986, 2023-12-06T18:29:28.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:29:28.046 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:28.046 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:28.046 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:28.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:28.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:28.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:28.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:28.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:28.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:29:28.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:29:28.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:28.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:29:28.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:29:28.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:28.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:29:28.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:29:28.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:28.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:29:28.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:29:28.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:28.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:29:28.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:29:28.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:28.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:29:28.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:29:28.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:28.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:29:28.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:29:28.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:28.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:29:28.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:29:28.050 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.050 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:28.050 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.050 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:29:28.050 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:28.051 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 142.64409205499999, 2023-12-06T18:29:28.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:29:28.051 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:28.051 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:28.051 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:28.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:28.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:28.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:28.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:28.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:28.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:29:28.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:29:28.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:28.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:29:28.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:29:28.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:28.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:29:28.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:29:28.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:28.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:29:28.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:29:28.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:28.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:29:28.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:29:28.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:28.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:29:28.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:29:28.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:28.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:29:28.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:29:28.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:28.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:29:28.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:29:28.054 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.054 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:28.054 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.054 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:29:28.054 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:28.054 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 142.63802897599999, 2023-12-06T18:29:28.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:29:28.054 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:28.054 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:28.054 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:28.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:28.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:28.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:28.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:28.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:28.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:29:28.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:29:28.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:28.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:29:28.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:29:28.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:28.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:29:28.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:29:28.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:28.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:29:28.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:29:28.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:28.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:29:28.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:29:28.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:28.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:29:28.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:29:28.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:28.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:29:28.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:29:28.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:28.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:29:28.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:29:28.058 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.058 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:28.058 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.058 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:29:28.058 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:28.058 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 142.50064189099999, 2023-12-06T18:29:28.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:29:28.058 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:28.058 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:28.058 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:28.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:28.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:28.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:28.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:28.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:28.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:29:28.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:29:28.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:28.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:29:28.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:29:28.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:28.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:29:28.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:29:28.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:28.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:29:28.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:29:28.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:28.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:29:28.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:29:28.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:28.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:29:28.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:29:28.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:28.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:29:28.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:29:28.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:28.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:29:28.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:29:28.061 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.061 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:28.062 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.062 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:29:28.062 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:28.062 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 142.49397116399999, 2023-12-06T18:29:28.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:29:28.062 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:28.062 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:28.062 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:28.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:28.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:28.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:28.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:28.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:28.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:28.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:29:28.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:29:28.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:28.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:29:28.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:29:28.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:28.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:29:28.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:29:28.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:28.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:29:28.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:29:28.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:28.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:29:28.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:29:28.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:28.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:29:28.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:29:28.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:28.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:29:28.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:29:28.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:28.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:28.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:28.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:29:28.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:29:28.065 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.065 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:28.065 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.065 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:28.065 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:28.066 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:28.066 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:27.976+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:28.067 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:29:28.111 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:28.106+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:28.169 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:28.166+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:28.237 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:28.237 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:28.237 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:28.237 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:28.248 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:29:28.271 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:28.268+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:28.371 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:28.368+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:28.418 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:28.418 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:28.418 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:28.418 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:28.418 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:28.428 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:29:28.472 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:28.469+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:28.573 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:28.571+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:28.599 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:28.600 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:28.600 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:28.600 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:29:28.600 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:28.609 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:29:28.674 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:28.671+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:28.695 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:16.629 99450== Warning: unimplemented fcntl command: 1038 2023-12-06T18:29:28.703 INFO:teuthology.orchestra.run.smithi152.stderr:no valid command found; 10 closest matches: 2023-12-06T18:29:28.703 INFO:teuthology.orchestra.run.smithi152.stderr:0 2023-12-06T18:29:28.703 INFO:teuthology.orchestra.run.smithi152.stderr:1 2023-12-06T18:29:28.703 INFO:teuthology.orchestra.run.smithi152.stderr:2 2023-12-06T18:29:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:abort 2023-12-06T18:29:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:assert 2023-12-06T18:29:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:bluestore allocator dump block 2023-12-06T18:29:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:bluestore allocator fragmentation block 2023-12-06T18:29:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:bluestore allocator fragmentation histogram block [] [] 2023-12-06T18:29:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:bluestore allocator score block 2023-12-06T18:29:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:config diff 2023-12-06T18:29:28.704 INFO:teuthology.orchestra.run.smithi152.stderr:admin_socket: invalid command 2023-12-06T18:29:28.706 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-12-06T18:29:28.706 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-7, ['dump_ops_in_flight'] 2023-12-06T18:29:28.758 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:28.759 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:28.759 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:28.759 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:28.768 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:29:28.774 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-12-06T18:29:28.774 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:28.771+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:28.875 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:28.873+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:28.945 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:28.945 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:28.945 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:28.945 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:28.945 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:28.955 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:29:28.998 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:28.994+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:29.077 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:29.074+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:29.132 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:29.133 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:29.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:29.133 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:29:29.133 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:29.141 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:29:29.178 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:29.175+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:29.279 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:29.276+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:29.280 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:29.280 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:29.280 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:29.280 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:29.289 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:29:29.380 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:29.377+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:29.467 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:29.467 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:29.467 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:29.467 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:29.468 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:29.476 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:29:29.518 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:29.515+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:29.582 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:29.579+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:29.683 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:29.680+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:29.713 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:29.713 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:29.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:29.713 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:29.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.714 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:29:29.714 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:29.714 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 154.303078656, 2023-12-06T18:29:29.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:29:29.714 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.714 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.714 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.714 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.714 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.714 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:29:29.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.714 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:29.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:29.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:29:29.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:29:29.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:29:29.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:29:29.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:29:29.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:29:29.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:29:29.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:29:29.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:29:29.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:29:29.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:29:29.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:29:29.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:29:29.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:29:29.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:29:29.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:29:29.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:29:29.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:29:29.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:29:29.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:29:29.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:29:29.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:29:29.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:29:29.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:29:29.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:29:29.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:29:29.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:29:29.719 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.719 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.719 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.720 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:29.720 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:29.720 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 152.19450541699999, 2023-12-06T18:29:29.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:29:29.720 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.720 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.720 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.720 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.720 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.720 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:29:29.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.720 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:29.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:29.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:29:29.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:29:29.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:29:29.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:29:29.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:29:29.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:29:29.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:29:29.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:29:29.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:29:29.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:29:29.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:29:29.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:29:29.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:29:29.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:29:29.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:29:29.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:29:29.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:29:29.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:29:29.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:29:29.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:29:29.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:29:29.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:29:29.725 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.725 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.725 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.725 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:29.725 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:29.725 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 152.193760185, 2023-12-06T18:29:29.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:29:29.725 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.725 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.725 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.725 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.726 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.726 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:29:29.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.726 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.726 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:29.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:29.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:29:29.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:29:29.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:29:29.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:29:29.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:29:29.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:29:29.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:29:29.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:29:29.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:29:29.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:29:29.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:29:29.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:29:29.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:29:29.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:29:29.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:29:29.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:29:29.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:29:29.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:29:29.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:29:29.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:29:29.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:29:29.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:29:29.730 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.730 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.731 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.731 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:29.731 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:29.731 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 152.19307642999999, 2023-12-06T18:29:29.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:29:29.731 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.731 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.731 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.731 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.731 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.731 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:29:29.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.732 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:29.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:29.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:29:29.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:29:29.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:29:29.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:29:29.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:29:29.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:29:29.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:29:29.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:29:29.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:29:29.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:29:29.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:29:29.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:29:29.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:29:29.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:29:29.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:29:29.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:29:29.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:29:29.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:29:29.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:29:29.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:29:29.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:29:29.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:29:29.736 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.736 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.736 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.737 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:29.737 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:29.737 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 152.18750317499999, 2023-12-06T18:29:29.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:29:29.737 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.737 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.737 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.737 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.737 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.737 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:29:29.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.737 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:29.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:29.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:29:29.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:29:29.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:29:29.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:29:29.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:29:29.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:29:29.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:29:29.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:29:29.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:29:29.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:29:29.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:29:29.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:29:29.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:29:29.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:29:29.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:29:29.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:29:29.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:29:29.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:29:29.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:29:29.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:29:29.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:29:29.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:29:29.742 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.742 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.742 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.742 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:29.742 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:29.742 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 152.18682953499999, 2023-12-06T18:29:29.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:29:29.742 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.742 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.742 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.742 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.743 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.743 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:29:29.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.743 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:29.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:29.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:29:29.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:29:29.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:29:29.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:29:29.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:29:29.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:29:29.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:29:29.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:29:29.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:29:29.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:29:29.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:29:29.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:29:29.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:29:29.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:29:29.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:29:29.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:29:29.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:29:29.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:29:29.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:29:29.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:29:29.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:29:29.748 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:29:29.748 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.748 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.748 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.748 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.748 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.748 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:29.748 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:29.748 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 152.18616733600001, 2023-12-06T18:29:29.748 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:29:29.748 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.748 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.748 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.748 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.748 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.748 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:29:29.748 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:29:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:29:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.749 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.750 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.750 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:29:29.750 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:29:29.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.750 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.750 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.750 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:29:29.750 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:29:29.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.750 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.750 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.750 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:29:29.750 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:29:29.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.751 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.751 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:29:29.751 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:29:29.751 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.751 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.751 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:29:29.751 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:29:29.751 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.751 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.751 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:29:29.751 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:29:29.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:29:29.752 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:29:29.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:29:29.752 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:29:29.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.753 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.753 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.753 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:29:29.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:29:29.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.753 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.753 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.753 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:29:29.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:29:29.753 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.753 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.754 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.754 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.754 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:29.754 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:29.754 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 152.113457901, 2023-12-06T18:29:29.754 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:29:29.754 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.754 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.755 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.755 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.755 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.755 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:29:29.755 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.755 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.755 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.755 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.755 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:29.755 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.756 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.756 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.756 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:29.756 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.756 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.756 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.756 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:29:29.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:29:29.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:29:29.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:29:29.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:29:29.758 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:29:29.758 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.758 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.758 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.758 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:29:29.758 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:29:29.758 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.758 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.758 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.758 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:29:29.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:29:29.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:29:29.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:29:29.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:29:29.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:29:29.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:29:29.760 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:29:29.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.760 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.760 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.760 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:29:29.760 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:29:29.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.760 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.760 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.760 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:29:29.760 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:29:29.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.760 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.760 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.760 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:29:29.761 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:29:29.761 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.761 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.761 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.761 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.761 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:29.761 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:29.761 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 152.11277619200001, 2023-12-06T18:29:29.761 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:29:29.761 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.761 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.761 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.761 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.761 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.761 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:29:29.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.762 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.762 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.762 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:29.762 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.762 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.762 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:29.762 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.762 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.762 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:29:29.763 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:29:29.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.763 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:29:29.763 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:29:29.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.763 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:29:29.763 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:29:29.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.764 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:29:29.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:29:29.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.764 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:29:29.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:29:29.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.764 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:29:29.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:29:29.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:29:29.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:29:29.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.765 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:29:29.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:29:29.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.765 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:29:29.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:29:29.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.766 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.766 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:29:29.766 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:29:29.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.766 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.766 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.766 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:29:29.766 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:29:29.766 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.766 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.766 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.766 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.766 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:29.767 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:29.767 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 151.03397210399999, 2023-12-06T18:29:29.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:29:29.767 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.767 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.767 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.767 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.767 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:29:29.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.767 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.767 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:29.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:29.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:29:29.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:29:29.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:29:29.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:29:29.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:29:29.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:29:29.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:29:29.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:29:29.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:29:29.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:29:29.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:29:29.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:29:29.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:29:29.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:29:29.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:29:29.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:29:29.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:29:29.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:29:29.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:29:29.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:29:29.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:29:29.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:29:29.772 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.772 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.772 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.772 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:29.772 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:29.772 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 151.03255553, 2023-12-06T18:29:29.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:29:29.772 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.772 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.772 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.772 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.773 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.773 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:29:29.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.773 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:29.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:29.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:29:29.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:29:29.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:29:29.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:29:29.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:29:29.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:29:29.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:29:29.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:29:29.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:29:29.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:29:29.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:29:29.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:29:29.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.776 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:29:29.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:29:29.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.776 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:29:29.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:29:29.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:29:29.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:29:29.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:29:29.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:29:29.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:29:29.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:29:29.778 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.778 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.778 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.778 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:29.778 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:29.778 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 151.03147040499999, 2023-12-06T18:29:29.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:29:29.779 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.779 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.779 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.779 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.779 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.779 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:29:29.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.779 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:29.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:29.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:29:29.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:29:29.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:29:29.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:29:29.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:29:29.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:29:29.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:29:29.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:29:29.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:29:29.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:29:29.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.782 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:29:29.782 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:29:29.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.782 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.782 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:29:29.782 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:29:29.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.782 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.782 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:29:29.782 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:29:29.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:29:29.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:29:29.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:29:29.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:29:29.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:29:29.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:29:29.783 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.783 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.783 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.784 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:29.784 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:29.784 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 150.996321303, 2023-12-06T18:29:29.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:29:29.784 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.784 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.784 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.784 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.784 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.784 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:29:29.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.784 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.785 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:29.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.785 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:29.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.785 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:29:29.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:29:29.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:29:29.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:29:29.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:29:29.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:29:29.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:29:29.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:29:29.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.787 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.787 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:29:29.787 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:29:29.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.787 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.787 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:29:29.787 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:29:29.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.788 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:29:29.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:29:29.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.788 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:29:29.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:29:29.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.788 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:29:29.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:29:29.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:29:29.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:29:29.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:29:29.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:29:29.789 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.790 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.790 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.790 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:29.790 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:29.790 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 144.446590778, 2023-12-06T18:29:29.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:29:29.790 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.790 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.790 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.790 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.790 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.790 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:29:29.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.791 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:29.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:29.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:29:29.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:29:29.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.792 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.792 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:29:29.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:29:29.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.792 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.792 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:29:29.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:29:29.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.792 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.792 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:29:29.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:29:29.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:29:29.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:29:29.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:29:29.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:29:29.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:29:29.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:29:29.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:29:29.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:29:29.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:29:29.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:29:29.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:29:29.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:29:29.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:29:29.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:29:29.795 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.795 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.795 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.795 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:29.795 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:29.795 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 144.44578516999999, 2023-12-06T18:29:29.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:29:29.796 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.796 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.796 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.796 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.796 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.796 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:29:29.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.796 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:29.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:29.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:29:29.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:29:29.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:29:29.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:29:29.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:29:29.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:29:29.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:29:29.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:29:29.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:29:29.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:29:29.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:29:29.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:29:29.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:29:29.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:29:29.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:29:29.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:29:29.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:29:29.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:29:29.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:29:29.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:29:29.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:29:29.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:29:29.800 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.800 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.800 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.800 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:29.800 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:29.800 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 144.445094361, 2023-12-06T18:29:29.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:29:29.801 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.801 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.801 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.801 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.801 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.801 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:29:29.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.801 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:29.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:29.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.802 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:29:29.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:29:29.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.802 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:29:29.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:29:29.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.802 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:29:29.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:29:29.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:29:29.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:29:29.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:29:29.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:29:29.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:29:29.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:29:29.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:29:29.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:29:29.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:29:29.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:29:29.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:29:29.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:29:29.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:29:29.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:29:29.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:29:29.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:29:29.805 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.805 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.805 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.805 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:29.805 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:29.805 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 144.44079685299999, 2023-12-06T18:29:29.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:29:29.805 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.805 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.806 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.806 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.806 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.806 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:29:29.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.806 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:29.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:29.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:29:29.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:29:29.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:29:29.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:29:29.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:29:29.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:29:29.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.808 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.808 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:29:29.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:29:29.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.808 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.808 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:29:29.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:29:29.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.808 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.808 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:29:29.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:29:29.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:29:29.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:29:29.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:29:29.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:29:29.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:29:29.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:29:29.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:29:29.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:29:29.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:29:29.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:29:29.810 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.810 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.810 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.811 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:29.811 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:29.811 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 144.44012917800001, 2023-12-06T18:29:29.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:29:29.811 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.811 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.811 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.811 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.811 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.811 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:29:29.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.811 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:29.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:29.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:29:29.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:29:29.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:29:29.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:29:29.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:29:29.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:29:29.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:29:29.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:29:29.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:29:29.814 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:29:29.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.814 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.814 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:29:29.814 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:29:29.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.814 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.814 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:29:29.814 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:29:29.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.814 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.815 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:29:29.815 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:29:29.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.815 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.815 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.815 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:29:29.815 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:29:29.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.815 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.815 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.815 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:29:29.815 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:29:29.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.815 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.815 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:29:29.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:29:29.816 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.816 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.816 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.816 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:29.816 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:29.816 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 144.43947374999999, 2023-12-06T18:29:29.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:29:29.816 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.816 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.816 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.816 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.816 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.816 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:29:29.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.817 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:29.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:29.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:29:29.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:29:29.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:29:29.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:29:29.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:29:29.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:29:29.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:29:29.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:29:29.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.819 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.819 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.819 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:29:29.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:29:29.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.819 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.819 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.819 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:29:29.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:29:29.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.819 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.819 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.819 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:29:29.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:29:29.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.820 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.820 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.820 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:29:29.820 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:29:29.820 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.820 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.820 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.820 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:29:29.820 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:29:29.820 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.820 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.820 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.820 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:29:29.820 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:29:29.821 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.821 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.821 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.821 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:29:29.821 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:29:29.821 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.821 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.821 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.821 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.821 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.821 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:29.821 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:29.821 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 144.39881920299999, 2023-12-06T18:29:29.821 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:29:29.821 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:29.821 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:29.822 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:29.822 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:29.822 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:29.822 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:29:29.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.822 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:29.822 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.822 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:29.822 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:29.822 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.822 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.822 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:29.822 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:29.822 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:29.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:29.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:29:29.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:29:29.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:29.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:29:29.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:29:29.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:29.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:29:29.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:29:29.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:29.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:29:29.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:29:29.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.824 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:29.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:29:29.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:29:29.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.824 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:29.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:29:29.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:29:29.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.824 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:29.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:29:29.825 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:29:29.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.825 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.825 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:29.825 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:29:29.825 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:29:29.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.825 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.825 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:29.825 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:29:29.825 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:29:29.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.825 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.825 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:29.825 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:29:29.825 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:29:29.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:29.826 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:29.826 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:29.826 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:29:29.826 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:29:29.826 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.826 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.826 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.826 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:29.826 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:29.826 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:29.828 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:29:29.828 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:29.825+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:29.929 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:29.926+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:29.966 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:29.966 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:29.967 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:29.967 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:29.976 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:29:30.030 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:30.026+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:30.131 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:30.127+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:30.155 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:30.156 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:30.156 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:30.156 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:30.156 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:30.165 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:29:30.232 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:30.230+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:30.303 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:30.303 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:30.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:30.303 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:30.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.303 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:29:30.303 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:30.303 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 128.487006538, 2023-12-06T18:29:30.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:29:30.303 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:30.304 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:29:30.304 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:30.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:30.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:30.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:30.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:30.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:30.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:30.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:30.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:30.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:30.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:29:30.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:29:30.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:30.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:30.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:29:30.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:29:30.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:30.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:30.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:29:30.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:29:30.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:30.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:30.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:29:30.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:29:30.306 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:30.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.306 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:30.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:29:30.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:29:30.306 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:30.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.306 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:30.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:29:30.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:29:30.306 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:30.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.307 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:30.307 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:29:30.307 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:29:30.307 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:30.307 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:30.307 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:30.307 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:30.307 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:30.307 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:30.311 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:29:30.353 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:30.350+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:30.434 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:30.430+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:30.492 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:30.492 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:30.492 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:30.492 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:30.501 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:29:30.503 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-12-06T18:29:30.503 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi152 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b" 2023-12-06T18:29:30.504 INFO:tasks.ceph.mon.b:Stopped 2023-12-06T18:29:30.504 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-12-06T18:29:30.504 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:29:30.534 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:30.531+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:30.558 INFO:tasks.ceph.mon.c.smithi061.stderr:2023-12-06T18:29:30.545+0000 9f54640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 98289) UID: 0 2023-12-06T18:29:30.558 INFO:tasks.ceph.mon.c.smithi061.stderr:2023-12-06T18:29:30.547+0000 9f54640 -1 mon.c@2(leader) e1 *** Got Signal Terminated *** 2023-12-06T18:29:30.635 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:30.631+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:30.656 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:30.656 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:30.656 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:30.656 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:30.656 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:30.665 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:29:30.750 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:30.747+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:30.839 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:30.834+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:30.847 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:30.847 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:30.847 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:30.847 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:30.847 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.847 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:30.847 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:30.848 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 9.7493658669999999, 2023-12-06T18:29:30.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:29:30.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:30.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:29:30.848 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:30.848 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:29:30.848 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:30.848 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:29:30.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:30.848 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:30.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:30.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:30.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:30.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:30.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:30.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:30.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:30.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:30.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:30.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:29:30.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:29:30.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:30.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:30.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:29:30.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:29:30.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:30.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:30.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:29:30.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:29:30.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:30.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:30.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:29:30.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:29:30.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:30.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:30.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:29:30.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:29:30.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:30.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:30.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:30.851 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:29:30.851 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:29:30.851 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:30.851 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:30.851 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:30.851 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:30.851 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:30.851 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:30.856 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:29:30.937 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:30.935+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:31.001 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:31.001 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:31.001 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:31.002 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:31.011 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:29:31.038 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:31.035+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:31.139 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:31.136+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:31.142 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:31.142 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:31.142 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:31.142 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:31.143 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:31.152 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:29:31.240 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:31.237+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:31.268 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:17.200 98294== 2023-12-06T18:29:31.268 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:17.200 98294== Exit program on first error (--exit-on-first-error=yes) 2023-12-06T18:29:31.279 INFO:tasks.ceph.mon.c.smithi061.stderr:daemon-helper: command failed with exit status 42 2023-12-06T18:29:31.324 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:31.324 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:31.324 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:31.324 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:31.324 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.324 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:31.324 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:31.324 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 69.830847837999997, 2023-12-06T18:29:31.325 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:29:31.325 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.325 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:29:31.325 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:29:31.325 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:29:31.325 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:31.325 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:29:31.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.325 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.325 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.325 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.325 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:31.325 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.325 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.325 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.325 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.326 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:31.326 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.326 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.326 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.326 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.326 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:29:31.326 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:29:31.326 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.326 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.326 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.326 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:29:31.326 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:29:31.326 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.327 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.327 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.327 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:29:31.327 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:29:31.327 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.327 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.327 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.327 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:29:31.327 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:29:31.327 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.327 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.327 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.327 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:29:31.327 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:29:31.327 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.327 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.328 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.328 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:29:31.328 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:29:31.328 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.328 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.328 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.328 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.328 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.328 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:31.334 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:29:31.379 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:31.375+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:31.441 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:31.438+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:31.492 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:31.492 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:31.492 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:31.492 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:31.502 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:29:31.543 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:31.540+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:31.643 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:31.640+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:31.671 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:31.671 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:31.672 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:31.672 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:31.672 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:31.681 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:29:31.744 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:31.740+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:31.845 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:31.842+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:31.853 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:31.853 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:31.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:31.853 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:31.853 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.853 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:29:31.853 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:31.853 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 155.23597513600001, 2023-12-06T18:29:31.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:29:31.853 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.854 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.854 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.854 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.854 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:31.854 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.854 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.854 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:31.854 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.854 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:29:31.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:29:31.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:29:31.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:29:31.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:29:31.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:29:31.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:29:31.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:29:31.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:29:31.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:29:31.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:29:31.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:29:31.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:29:31.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:29:31.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:29:31.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:29:31.857 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.857 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.857 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.857 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:29:31.857 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:31.857 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 154.21314812700001, 2023-12-06T18:29:31.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:29:31.858 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.858 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.858 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:31.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.858 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:31.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.858 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:29:31.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:29:31.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:29:31.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:29:31.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:29:31.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:29:31.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.860 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:29:31.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:29:31.860 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.860 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.860 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:29:31.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:29:31.860 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.860 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.860 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:29:31.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:29:31.860 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:29:31.861 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:29:31.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:29:31.861 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:29:31.861 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.861 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.861 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.861 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:29:31.862 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:31.862 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 154.201101489, 2023-12-06T18:29:31.862 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:29:31.862 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.862 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.862 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.862 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.862 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.862 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:31.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:31.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:29:31.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:29:31.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:29:31.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:29:31.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:29:31.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:29:31.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:29:31.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:29:31.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:29:31.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:29:31.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:29:31.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:29:31.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:29:31.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:29:31.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.866 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:29:31.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:29:31.866 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.866 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.866 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.866 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:29:31.867 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:31.867 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 154.17970088199999, 2023-12-06T18:29:31.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:29:31.867 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.867 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.867 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.867 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:31.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.867 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:31.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:29:31.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:29:31.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:29:31.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:29:31.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:29:31.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:29:31.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:29:31.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:29:31.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:29:31.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:29:31.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:29:31.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:29:31.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:29:31.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:29:31.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.871 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.871 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:29:31.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:29:31.871 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.871 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.871 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.871 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:29:31.871 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:31.871 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 154.17852155899999, 2023-12-06T18:29:31.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:29:31.872 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.872 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.872 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:31.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:31.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:29:31.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:29:31.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:29:31.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:29:31.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:29:31.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:29:31.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:29:31.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:29:31.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:29:31.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:29:31.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:29:31.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:29:31.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:29:31.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:29:31.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:29:31.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:29:31.875 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.875 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.875 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.875 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:29:31.876 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:31.876 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 154.17794957800001, 2023-12-06T18:29:31.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:29:31.876 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.876 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.876 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:31.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:31.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:29:31.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:29:31.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:29:31.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:29:31.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:29:31.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:29:31.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:29:31.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:29:31.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:29:31.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:29:31.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:29:31.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:29:31.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:29:31.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:29:31.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:29:31.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:29:31.880 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.880 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.880 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.880 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:29:31.880 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:31.880 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 153.73211410799999, 2023-12-06T18:29:31.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:29:31.880 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.880 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.880 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:31.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:31.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:29:31.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:29:31.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:29:31.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:29:31.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:29:31.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:29:31.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:29:31.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:29:31.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:29:31.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:29:31.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:29:31.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:29:31.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:29:31.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:29:31.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.884 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.884 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.884 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:29:31.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:29:31.884 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.884 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.884 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.884 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.884 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.884 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:29:31.884 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:31.884 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 153.73051425200001, 2023-12-06T18:29:31.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:29:31.884 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.884 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.884 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:31.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:31.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:29:31.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:29:31.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:29:31.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:29:31.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:29:31.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:29:31.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:29:31.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:29:31.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:29:31.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:29:31.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:29:31.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:29:31.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:29:31.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:29:31.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:29:31.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:29:31.888 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.888 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.888 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.889 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:29:31.889 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:31.889 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 153.351308636, 2023-12-06T18:29:31.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:29:31.889 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.889 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.889 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.889 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.889 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:31.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.889 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.889 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.889 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:31.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:29:31.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:29:31.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:29:31.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:29:31.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:29:31.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:29:31.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:29:31.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:29:31.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:29:31.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:29:31.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:29:31.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:29:31.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:29:31.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:29:31.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:29:31.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:29:31.892 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.892 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.892 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.892 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:29:31.893 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:31.893 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 153.11380645400001, 2023-12-06T18:29:31.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:29:31.893 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.893 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.893 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.893 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.893 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:31.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.893 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.893 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:31.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:29:31.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:29:31.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:29:31.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:29:31.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:29:31.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:29:31.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:29:31.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:29:31.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:29:31.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:29:31.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:29:31.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:29:31.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:29:31.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:29:31.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:29:31.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:29:31.896 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.896 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.896 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.896 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:29:31.896 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:31.897 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 153.11307845499999, 2023-12-06T18:29:31.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:29:31.897 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.897 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.897 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.897 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.897 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:31.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.897 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.897 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:31.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:29:31.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:29:31.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:29:31.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:29:31.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:29:31.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:29:31.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:29:31.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:29:31.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:29:31.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:29:31.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:29:31.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:29:31.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:29:31.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:29:31.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:29:31.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:29:31.900 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.900 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.900 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.900 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:29:31.900 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:31.901 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 153.10806604499999, 2023-12-06T18:29:31.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:29:31.901 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.901 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.901 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:31.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:31.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:29:31.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:29:31.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:29:31.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:29:31.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:29:31.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:29:31.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:29:31.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:29:31.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:29:31.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:29:31.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:29:31.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:29:31.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:29:31.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:29:31.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:29:31.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:29:31.904 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.904 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.904 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.904 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:29:31.904 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:31.904 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 153.102538966, 2023-12-06T18:29:31.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:29:31.905 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.905 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.905 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:31.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:31.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:29:31.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:29:31.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:29:31.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:29:31.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:29:31.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:29:31.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:29:31.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:29:31.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:29:31.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:29:31.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:29:31.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:29:31.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:29:31.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:29:31.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:29:31.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:29:31.908 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.908 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.908 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.908 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:29:31.908 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:31.908 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 153.10202812899999, 2023-12-06T18:29:31.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:29:31.909 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.909 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.909 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:31.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:31.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:29:31.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:29:31.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:29:31.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:29:31.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:29:31.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:29:31.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:29:31.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:29:31.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:29:31.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:29:31.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:29:31.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:29:31.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:29:31.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:29:31.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:29:31.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:29:31.912 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.912 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.912 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.912 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:29:31.912 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:31.912 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 153.10156886799999, 2023-12-06T18:29:31.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:29:31.913 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.913 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.913 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:31.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:31.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:29:31.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:29:31.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:29:31.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:29:31.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:29:31.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:29:31.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:29:31.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:29:31.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:29:31.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:29:31.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:29:31.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:29:31.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:29:31.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:29:31.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:29:31.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:29:31.916 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.916 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.916 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.916 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:29:31.916 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:31.916 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 146.52165306099999, 2023-12-06T18:29:31.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:29:31.917 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.917 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.917 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:31.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:31.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:29:31.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:29:31.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:29:31.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:29:31.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:29:31.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:29:31.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:29:31.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:29:31.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:29:31.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:29:31.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:29:31.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:29:31.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:29:31.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:29:31.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:29:31.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:29:31.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.920 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.920 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:29:31.920 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:31.921 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 146.51541525600001, 2023-12-06T18:29:31.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:29:31.921 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.921 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.921 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:31.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:31.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:29:31.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:29:31.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:29:31.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:29:31.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:29:31.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:29:31.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:29:31.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:29:31.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:29:31.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:29:31.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:29:31.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:29:31.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:29:31.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:29:31.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:29:31.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:29:31.924 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.925 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.925 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.925 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:29:31.925 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:31.925 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 146.50935217700001, 2023-12-06T18:29:31.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:29:31.925 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.925 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.925 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:31.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:31.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:29:31.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:29:31.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:29:31.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:29:31.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:29:31.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:29:31.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:29:31.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:29:31.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:29:31.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:29:31.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:29:31.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:29:31.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:29:31.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:29:31.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:29:31.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:29:31.928 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.929 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.929 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.929 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:29:31.929 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:31.929 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 146.37196509200001, 2023-12-06T18:29:31.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:29:31.929 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.929 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.930 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:31.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:31.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.932 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:29:31.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:29:31.932 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.932 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:29:31.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:29:31.932 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.932 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:29:31.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:29:31.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:29:31.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:29:31.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:29:31.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:29:31.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:29:31.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:29:31.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:29:31.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:29:31.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:29:31.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:29:31.934 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.935 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.935 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.935 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:29:31.935 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:31.935 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 146.36529436500001, 2023-12-06T18:29:31.936 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:29:31.936 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:31.936 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:31.936 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:31.936 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.936 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:31.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:31.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:31.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:31.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:31.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:31.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:29:31.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:29:31.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.939 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.939 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:31.939 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:29:31.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:29:31.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:31.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:29:31.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:29:31.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:31.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:29:31.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:29:31.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:31.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:29:31.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:29:31.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:31.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:29:31.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:29:31.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.943 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:31.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:29:31.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:29:31.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:31.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:31.944 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:31.944 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:29:31.944 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:29:31.944 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.944 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.944 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.944 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:31.945 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:31.945 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:31.947 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:29:31.948 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:31.943+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:32.047 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:32.044+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:32.118 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:32.118 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:32.118 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:32.118 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:32.127 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:29:32.168 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:32.165+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:32.250 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:32.246+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:32.301 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:32.301 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:32.301 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:32.301 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:32.301 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:32.310 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:29:32.355 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:32.350+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:32.442 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:32.442 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:32.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:32.443 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:29:32.443 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:32.454 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:29:32.454 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:32.450+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:32.555 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:32.551+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:32.603 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:32.603 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:32.603 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:32.603 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:32.612 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:29:32.656 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:32.652+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:32.748 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:32.749 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:32.749 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:32.749 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:32.749 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:32.757 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:32.753+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:32.758 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:29:32.858 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:32.854+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:32.939 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:32.939 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:32.939 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:32.939 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:29:32.939 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:32.948 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:20.881 99450== Warning: unimplemented fcntl command: 1038 2023-12-06T18:29:32.949 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:29:32.951 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:20.885 99450== Warning: unimplemented fcntl command: 1038 2023-12-06T18:29:32.990 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:32.986+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:33.060 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:33.055+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:33.125 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:33.125 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:33.125 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:33.125 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:33.134 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:29:33.161 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:33.156+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:33.262 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:33.258+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:33.270 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:33.270 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:33.270 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:33.270 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:33.270 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:33.280 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:29:33.363 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:33.359+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:33.524 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:33.524 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:33.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:33.525 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:33.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.525 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:29:33.525 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:33.525 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 158.11078070400001, 2023-12-06T18:29:33.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:29:33.525 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.525 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.525 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.525 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.525 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.525 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:29:33.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.525 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:33.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:33.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:29:33.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:29:33.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:29:33.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:29:33.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:29:33.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:29:33.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:29:33.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:29:33.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:29:33.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:29:33.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:29:33.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:29:33.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:29:33.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:29:33.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:29:33.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:29:33.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:29:33.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:29:33.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:29:33.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:29:33.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:29:33.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:29:33.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:29:33.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:29:33.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:29:33.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:29:33.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:29:33.531 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.531 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.531 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.531 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:33.531 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:33.531 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 156.002207465, 2023-12-06T18:29:33.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:29:33.531 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.531 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.531 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.531 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.532 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.532 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:29:33.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.532 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:33.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:33.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:29:33.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:29:33.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:29:33.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:29:33.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:29:33.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:29:33.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:29:33.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:29:33.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:29:33.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:29:33.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:29:33.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:29:33.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:29:33.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:29:33.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:29:33.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:29:33.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:29:33.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:29:33.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:29:33.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:29:33.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:29:33.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:29:33.536 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.536 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.536 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.536 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:33.536 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:33.536 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 156.00146223300001, 2023-12-06T18:29:33.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:29:33.537 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.537 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.537 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.537 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.537 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.537 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:29:33.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.537 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:33.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.538 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:33.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.538 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:29:33.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:29:33.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:29:33.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:29:33.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:29:33.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:29:33.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:29:33.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:29:33.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:29:33.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:29:33.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:29:33.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:29:33.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:29:33.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:29:33.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:29:33.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:29:33.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:29:33.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:29:33.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:29:33.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:29:33.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:29:33.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:29:33.542 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.542 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.542 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.542 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:33.542 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:33.542 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 156.000778478, 2023-12-06T18:29:33.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:29:33.542 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.543 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.543 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.543 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.543 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.543 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:29:33.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.543 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:33.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:33.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:29:33.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:29:33.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:29:33.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:29:33.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:29:33.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:29:33.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:29:33.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:29:33.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:29:33.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:29:33.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:29:33.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:29:33.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:29:33.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:29:33.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:29:33.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:29:33.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:29:33.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:29:33.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:29:33.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:29:33.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:29:33.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:29:33.548 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.548 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.548 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.548 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:33.548 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:33.548 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 155.995205223, 2023-12-06T18:29:33.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:29:33.548 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.548 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.548 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.548 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.548 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.549 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:29:33.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.549 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:33.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:33.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:29:33.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:29:33.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:29:33.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:29:33.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:29:33.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:29:33.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:29:33.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:29:33.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:29:33.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:29:33.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:29:33.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:29:33.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:29:33.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:29:33.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:29:33.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:29:33.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:29:33.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:29:33.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:29:33.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:29:33.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:29:33.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:29:33.553 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.553 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.553 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.554 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:33.554 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:33.554 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 155.994531583, 2023-12-06T18:29:33.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:29:33.554 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.554 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.554 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.554 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.554 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.554 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:29:33.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.554 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:33.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:33.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:29:33.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:29:33.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:29:33.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:29:33.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.556 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.556 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:29:33.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:29:33.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.556 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.556 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:29:33.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:29:33.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:29:33.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:29:33.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:29:33.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:29:33.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:29:33.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:29:33.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:29:33.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:29:33.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:29:33.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:29:33.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:29:33.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:29:33.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:29:33.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:29:33.559 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.559 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.559 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.559 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:33.559 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:33.559 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 155.99386938399999, 2023-12-06T18:29:33.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:29:33.559 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.559 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.559 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.559 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.560 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.560 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:29:33.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.560 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:33.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:33.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:29:33.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:29:33.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:29:33.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:29:33.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:29:33.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:29:33.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:29:33.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:29:33.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:29:33.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:29:33.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:29:33.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:29:33.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:29:33.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:29:33.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:29:33.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:29:33.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:29:33.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:29:33.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:29:33.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:29:33.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:29:33.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:29:33.564 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.564 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.564 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.564 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:33.564 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:33.565 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 155.92115994900001, 2023-12-06T18:29:33.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:29:33.565 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.565 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.565 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.565 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.565 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.565 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:29:33.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.565 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:33.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:33.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:29:33.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:29:33.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:29:33.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:29:33.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.567 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.567 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:29:33.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:29:33.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.567 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.567 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:29:33.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:29:33.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.567 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.567 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:29:33.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:29:33.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.568 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.568 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:29:33.568 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:29:33.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.568 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.568 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:29:33.568 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:29:33.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.568 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.568 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:29:33.568 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:29:33.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:29:33.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:29:33.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:29:33.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:29:33.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:29:33.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:29:33.569 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.569 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.570 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.570 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:33.570 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:33.570 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 155.92047823999999, 2023-12-06T18:29:33.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:29:33.570 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.570 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.570 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.570 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.570 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.570 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:29:33.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.571 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:33.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:33.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:29:33.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:29:33.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.572 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:29:33.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:29:33.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.572 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:29:33.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:29:33.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.572 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:29:33.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:29:33.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.573 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.573 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:29:33.573 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:29:33.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.573 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.573 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:29:33.573 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:29:33.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.573 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.573 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:29:33.573 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:29:33.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:29:33.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:29:33.574 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:29:33.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:29:33.574 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:29:33.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:29:33.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.575 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.575 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:29:33.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:29:33.575 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.575 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.575 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.575 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:33.575 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:33.576 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 154.841674152, 2023-12-06T18:29:33.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:29:33.576 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.576 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.576 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.576 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.576 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.577 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:29:33.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.577 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.577 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.577 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:33.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:33.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.579 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.579 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:29:33.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:29:33.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.579 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.579 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.579 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:29:33.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:29:33.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.580 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.580 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:29:33.580 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:29:33.580 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.580 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:29:33.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:29:33.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.581 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.581 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:29:33.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:29:33.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.582 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.582 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:29:33.582 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:29:33.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.583 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:29:33.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:29:33.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.583 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:29:33.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:29:33.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.583 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:29:33.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:29:33.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.584 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:29:33.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:29:33.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.584 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:29:33.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:29:33.584 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.584 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.584 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.584 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:33.585 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:33.585 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 154.84025757800001, 2023-12-06T18:29:33.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:29:33.585 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.585 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.585 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.585 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.585 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.585 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:29:33.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.585 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.585 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.585 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:33.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:33.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:29:33.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:29:33.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:29:33.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:29:33.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:29:33.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:29:33.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:29:33.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:29:33.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.588 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.588 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.588 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:29:33.588 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:29:33.588 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.588 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.588 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.588 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:29:33.588 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:29:33.588 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.588 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.588 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.589 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:29:33.589 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:29:33.589 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.589 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.589 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:29:33.589 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:29:33.589 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.589 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.589 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:29:33.589 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:29:33.589 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.589 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.590 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:29:33.590 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:29:33.590 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.590 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.590 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.590 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:29:33.590 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:29:33.590 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.590 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.590 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.590 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.590 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.590 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:33.590 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:33.590 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 154.839172453, 2023-12-06T18:29:33.591 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:29:33.591 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.591 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.591 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.591 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.591 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.591 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:29:33.591 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.591 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.591 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.591 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.591 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:33.591 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.591 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.591 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.591 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.592 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:33.592 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.592 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.592 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.592 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.592 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:29:33.592 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:29:33.592 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.592 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.592 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.592 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:29:33.592 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:29:33.592 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.592 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.593 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:29:33.593 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:29:33.593 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.593 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.593 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:29:33.593 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:29:33.593 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.593 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.593 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:29:33.593 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:29:33.593 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.593 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.593 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:29:33.594 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:29:33.594 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.594 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.594 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.594 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:29:33.594 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:29:33.594 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.594 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.594 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.594 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:29:33.594 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:29:33.594 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.594 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.594 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.595 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:29:33.595 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:29:33.595 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.595 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.595 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.595 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:29:33.595 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:29:33.595 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.595 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.595 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.595 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:29:33.595 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:29:33.595 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.595 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.595 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.596 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.596 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.596 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:33.596 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:33.596 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 154.80402335100001, 2023-12-06T18:29:33.596 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:29:33.596 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.596 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.596 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.596 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.596 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.596 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:29:33.596 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.596 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.596 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.596 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.597 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:33.597 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.597 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.597 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.597 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.597 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:33.597 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.597 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.597 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.597 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.597 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:29:33.597 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:29:33.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.598 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:29:33.598 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:29:33.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.598 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:29:33.598 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:29:33.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.598 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:29:33.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:29:33.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.599 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:29:33.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:29:33.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.599 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:29:33.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:29:33.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.599 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:29:33.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:29:33.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.600 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.600 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.600 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:29:33.600 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:29:33.600 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.600 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.600 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.600 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:29:33.600 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:29:33.600 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.600 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.600 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.600 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:29:33.600 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:29:33.600 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.600 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.601 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.601 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:29:33.601 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:29:33.601 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.601 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.601 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.601 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.601 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.601 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:33.601 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:33.601 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 148.25429282600001, 2023-12-06T18:29:33.601 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:29:33.601 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.601 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.601 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.601 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.602 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.602 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:29:33.602 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.602 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.602 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.602 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.602 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:33.602 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.602 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.602 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.602 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.602 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:33.602 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.603 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.603 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.603 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.603 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:29:33.603 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:29:33.603 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.603 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.603 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.603 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:29:33.603 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:29:33.603 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.603 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.603 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.603 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:29:33.603 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:29:33.603 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.604 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.604 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.604 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:29:33.604 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:29:33.604 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.604 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.604 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.604 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:29:33.604 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:29:33.604 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.604 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.604 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.604 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:29:33.604 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:29:33.604 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.604 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.605 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.605 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:29:33.605 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:29:33.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.605 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.605 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.605 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:29:33.605 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:29:33.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.605 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.605 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.605 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:29:33.605 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:29:33.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.605 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.606 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.606 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:29:33.606 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:29:33.606 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.606 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.606 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.606 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:29:33.606 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:29:33.606 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.606 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.606 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.606 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.607 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.607 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:33.607 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:33.607 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 148.253487218, 2023-12-06T18:29:33.607 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:29:33.607 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.607 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.607 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.607 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.607 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.607 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:29:33.607 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.607 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.607 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.607 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.608 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:33.608 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.608 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.608 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.608 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.608 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:33.608 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.608 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.608 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.608 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.608 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:29:33.608 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:29:33.608 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.608 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.608 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.608 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:29:33.609 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:29:33.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.609 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.609 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.609 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:29:33.609 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:29:33.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.609 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.609 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.609 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:29:33.609 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:29:33.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.609 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.609 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.609 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:29:33.609 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:29:33.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.610 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.610 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.610 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:29:33.610 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:29:33.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.610 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.610 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.610 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:29:33.610 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:29:33.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.610 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.610 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.610 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:29:33.611 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:29:33.611 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.611 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.611 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.611 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:29:33.611 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:29:33.611 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.611 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.611 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.611 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:29:33.611 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:29:33.611 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.611 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.611 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.611 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:29:33.612 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:29:33.612 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.612 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.612 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.612 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.612 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.612 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:33.612 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:33.612 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 148.25279640900001, 2023-12-06T18:29:33.612 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:29:33.612 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.612 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.612 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.612 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.612 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.613 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:29:33.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.613 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.613 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.613 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.613 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:33.613 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.613 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.613 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.613 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:33.613 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.613 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.613 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.613 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:29:33.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:29:33.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.614 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:29:33.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:29:33.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.614 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:29:33.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:29:33.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.614 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:29:33.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:29:33.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.615 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:29:33.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:29:33.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.615 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:29:33.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:29:33.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.615 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:29:33.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:29:33.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:29:33.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:29:33.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:29:33.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:29:33.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:29:33.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:29:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:29:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:29:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 148.248498901, 2023-12-06T18:29:33.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:29:33.618 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.618 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.618 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.618 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.618 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.618 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:29:33.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.618 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:33.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.619 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.619 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:33.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.619 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.619 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:29:33.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:29:33.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.620 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.620 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.620 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:29:33.620 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:29:33.620 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.620 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.620 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.620 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:29:33.620 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:29:33.620 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.620 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.620 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.620 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:29:33.620 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:29:33.621 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.621 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.621 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.621 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:29:33.621 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:29:33.621 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.621 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.621 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.621 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:29:33.621 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:29:33.621 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.621 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.622 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.622 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:29:33.622 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:29:33.622 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.622 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.622 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.622 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:29:33.622 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:29:33.622 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.622 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.622 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.622 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:29:33.622 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:29:33.622 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.623 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.623 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.623 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:29:33.623 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:29:33.623 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.623 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.623 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.623 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:29:33.623 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:29:33.623 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.623 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.623 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.623 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.623 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.624 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:33.624 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:33.624 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 148.24783122599999, 2023-12-06T18:29:33.624 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:29:33.624 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.624 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.624 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.624 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.624 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.624 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:29:33.624 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.624 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.624 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.624 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.624 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:33.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:33.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:29:33.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:29:33.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:29:33.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:29:33.626 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.626 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.626 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.626 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:29:33.626 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:29:33.626 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.626 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.626 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.626 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:29:33.626 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:29:33.626 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.626 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.626 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.626 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:29:33.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:29:33.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.627 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:29:33.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:29:33.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.627 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:29:33.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:29:33.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:29:33.628 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:29:33.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.628 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.628 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:29:33.628 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:29:33.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.628 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.628 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:29:33.628 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:29:33.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.628 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.628 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:29:33.629 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:29:33.629 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.629 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.629 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.629 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.629 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.629 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:33.629 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:33.629 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 148.247175798, 2023-12-06T18:29:33.629 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:29:33.629 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.629 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.629 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.629 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.629 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.629 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:29:33.629 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.630 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.630 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.630 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.630 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:33.630 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.630 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.630 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.630 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:33.630 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.630 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.630 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.630 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:29:33.631 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:29:33.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.631 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.631 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.631 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:29:33.631 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:29:33.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.631 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.631 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.631 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:29:33.631 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:29:33.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.631 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.631 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.631 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:29:33.632 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:29:33.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.632 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.632 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:29:33.632 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:29:33.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.632 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.632 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:29:33.632 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:29:33.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.632 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.632 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:29:33.632 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:29:33.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.633 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.633 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.633 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:29:33.633 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:29:33.633 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.633 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.633 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.633 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:29:33.633 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:29:33.633 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.633 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.633 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.633 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:29:33.633 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:29:33.633 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.633 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.633 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.634 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:29:33.634 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:29:33.634 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.634 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.634 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.634 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.634 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.634 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:33.634 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:33.634 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 148.206521251, 2023-12-06T18:29:33.634 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:29:33.635 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:33.635 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:33.635 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:33.635 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:33.635 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:33.635 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:29:33.635 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.635 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:33.635 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.635 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:33.635 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:33.635 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.635 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.635 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.635 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:33.635 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:33.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:33.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:33.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:29:33.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:29:33.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:33.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:29:33.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:29:33.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:33.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:29:33.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:29:33.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:33.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:29:33.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:29:33.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:33.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:29:33.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:29:33.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:33.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:29:33.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:29:33.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:33.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:29:33.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:29:33.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:33.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:29:33.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:29:33.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:33.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:29:33.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:29:33.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:33.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:29:33.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:29:33.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:33.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:33.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:33.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:29:33.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:29:33.641 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.641 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.641 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.641 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:33.641 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:33.641 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:33.642 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:33.521+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:33.643 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:29:33.645 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:33.641+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:33.707 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:29:33.751 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:33.748+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:33.781 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:33.781 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:33.781 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:33.781 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:33.790 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:29:33.847 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:33.843+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:33.927 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:33.927 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:33.927 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:33.927 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:33.927 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:33.937 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:29:33.948 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:33.944+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:34.049 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:34.045+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:34.117 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:34.117 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:34.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:34.117 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:34.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.117 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:29:34.117 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:34.117 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 132.30067295399999, 2023-12-06T18:29:34.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:29:34.117 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:34.117 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:29:34.117 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:34.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:34.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:34.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:34.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:34.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:34.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:34.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:34.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:34.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:34.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:29:34.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:29:34.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:34.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:34.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:29:34.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:29:34.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:34.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:34.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:29:34.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:29:34.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:34.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:34.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:29:34.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:29:34.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:34.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:34.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:29:34.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:29:34.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:34.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:34.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:29:34.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:29:34.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:34.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:34.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:29:34.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:29:34.120 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:34.120 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:34.120 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:34.120 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:34.121 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:34.121 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:34.127 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:29:34.149 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:34.146+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:34.250 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:34.247+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:34.305 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:34.305 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:34.305 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:34.305 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:34.315 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:29:34.352 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:34.348+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:34.452 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:34.453 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:34.453 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:34.453 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:34.453 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:34.453 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:34.449+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:34.461 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:29:34.554 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:34.550+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:34.646 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:34.646 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:34.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:34.646 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:34.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.646 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:34.646 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:34.646 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 13.548100322, 2023-12-06T18:29:34.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:29:34.646 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:34.646 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:29:34.647 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:34.647 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:29:34.647 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:34.647 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:29:34.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:34.647 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:34.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:34.647 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:34.647 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:34.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:34.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:34.648 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:34.648 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:34.648 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:34.648 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.648 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:34.648 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:29:34.648 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:29:34.648 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:34.648 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.648 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:34.648 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:29:34.648 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:29:34.648 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:34.649 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.649 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:34.649 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:29:34.649 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:29:34.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:34.649 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.649 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:34.649 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:29:34.649 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:29:34.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:34.649 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.649 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:34.649 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:29:34.649 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:29:34.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:34.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:34.650 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:34.650 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:29:34.650 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:29:34.650 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:34.650 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:34.650 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:34.650 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:34.650 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:34.650 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:34.655 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:34.651+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:34.656 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:29:34.756 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:34.752+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:34.796 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:34.797 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:34.797 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:34.797 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:34.808 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:29:34.856 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:34.852+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:34.882 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-12-06T18:29:34.882 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-12-06T18:29:34.957 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:34.953+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:34.978 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:34.978 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:34.978 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:34.978 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:34.979 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:34.988 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:29:35.058 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:35.054+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:35.119 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:35.120 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:35.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:35.120 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:35.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.120 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:35.120 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:35.120 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 73.626414652999998, 2023-12-06T18:29:35.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:29:35.120 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.120 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:29:35.120 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:29:35.120 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:29:35.121 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:35.121 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:29:35.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.121 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:35.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:35.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.122 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:29:35.122 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:29:35.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.122 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.122 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:29:35.122 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:29:35.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.122 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.122 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:29:35.122 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:29:35.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.122 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:29:35.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:29:35.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:29:35.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:29:35.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:29:35.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:29:35.123 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.123 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.123 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.123 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.124 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.124 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:35.130 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:29:35.172 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:35.168+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:no valid command found; 10 closest matches: 2023-12-06T18:29:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:0 2023-12-06T18:29:35.225 INFO:teuthology.orchestra.run.smithi152.stderr:1 2023-12-06T18:29:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:2 2023-12-06T18:29:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:abort 2023-12-06T18:29:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:assert 2023-12-06T18:29:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:bluefs debug_inject_read_zeros 2023-12-06T18:29:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:bluefs files list 2023-12-06T18:29:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:bluefs stats 2023-12-06T18:29:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:bluestore bluefs device info [] 2023-12-06T18:29:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:config diff 2023-12-06T18:29:35.226 INFO:teuthology.orchestra.run.smithi152.stderr:admin_socket: invalid command 2023-12-06T18:29:35.227 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-12-06T18:29:35.228 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-7, ['dump_ops_in_flight'] 2023-12-06T18:29:35.260 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:35.256+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:35.301 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:35.301 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:35.301 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:35.301 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:35.311 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:29:35.396 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:35.392+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:35.462 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:35.458+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:35.496 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:35.497 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:35.497 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:35.497 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:35.497 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:35.506 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:29:35.563 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:35.559+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:35.663 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:35.660+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:35.719 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:35.719 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:35.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:35.719 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:35.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.719 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:29:35.719 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:35.719 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 159.10133119100001, 2023-12-06T18:29:35.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:29:35.719 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.720 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.720 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:35.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:35.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:29:35.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:29:35.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:29:35.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:29:35.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:29:35.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:29:35.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:29:35.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:29:35.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:29:35.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:29:35.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:29:35.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:29:35.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.723 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:29:35.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:29:35.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.723 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:29:35.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:29:35.723 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.723 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.723 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.723 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:29:35.723 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:35.723 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 158.07850418199999, 2023-12-06T18:29:35.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:29:35.724 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.724 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.724 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:35.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:35.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:29:35.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:29:35.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:29:35.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:29:35.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:29:35.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:29:35.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:29:35.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:29:35.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:29:35.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:29:35.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:29:35.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:29:35.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.727 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.727 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:29:35.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:29:35.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.727 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.727 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:29:35.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:29:35.727 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.727 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.727 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.727 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:29:35.728 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:35.728 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 158.066457544, 2023-12-06T18:29:35.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:29:35.728 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.728 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.728 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:35.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:35.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:29:35.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:29:35.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:29:35.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:29:35.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:29:35.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:29:35.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:29:35.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:29:35.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:29:35.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:29:35.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:29:35.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:29:35.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:29:35.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:29:35.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:29:35.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:29:35.731 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.731 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.732 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.732 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:29:35.732 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:35.732 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 158.045056937, 2023-12-06T18:29:35.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:29:35.732 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.732 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.732 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:35.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:35.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:29:35.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:29:35.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:29:35.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:29:35.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:29:35.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:29:35.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:29:35.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:29:35.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:29:35.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:29:35.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:29:35.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:29:35.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:29:35.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:29:35.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:29:35.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:29:35.736 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.736 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.736 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.736 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:29:35.736 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:35.736 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 158.043877614, 2023-12-06T18:29:35.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:29:35.736 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.736 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.736 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:35.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:35.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:29:35.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:29:35.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:29:35.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:29:35.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:29:35.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:29:35.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:29:35.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:29:35.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:29:35.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:29:35.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:29:35.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:29:35.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:29:35.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:29:35.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:29:35.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:29:35.740 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.740 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.740 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.740 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:29:35.740 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:35.740 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 158.04330563299999, 2023-12-06T18:29:35.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:29:35.740 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.741 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.741 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:35.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:35.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:29:35.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:29:35.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:29:35.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:29:35.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:29:35.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:29:35.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:29:35.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:29:35.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:29:35.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:29:35.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:29:35.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:29:35.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:29:35.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:29:35.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:29:35.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:29:35.744 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.744 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.744 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.745 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:29:35.745 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:35.745 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 157.597470163, 2023-12-06T18:29:35.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:29:35.745 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.745 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.745 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:35.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:35.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:29:35.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:29:35.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:29:35.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:29:35.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:29:35.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:29:35.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:29:35.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:29:35.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:29:35.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:29:35.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:29:35.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:29:35.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:29:35.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:29:35.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:29:35.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:29:35.750 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.750 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.750 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.750 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:29:35.750 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:35.750 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 157.59587030700001, 2023-12-06T18:29:35.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:29:35.751 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.751 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.751 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:35.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:35.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:29:35.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:29:35.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:29:35.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:29:35.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:29:35.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:29:35.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:29:35.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:29:35.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:29:35.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:29:35.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:29:35.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:29:35.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:29:35.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:29:35.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:29:35.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:29:35.755 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.755 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.755 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.755 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:29:35.755 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:35.755 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 157.21666469100001, 2023-12-06T18:29:35.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:29:35.755 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.755 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.755 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:35.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:35.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:29:35.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:29:35.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:29:35.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:29:35.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:29:35.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:29:35.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:29:35.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:29:35.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:29:35.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:29:35.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:29:35.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:29:35.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:29:35.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:29:35.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:29:35.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:29:35.759 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.759 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.759 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.759 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:29:35.760 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:35.760 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 156.97916250899999, 2023-12-06T18:29:35.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:29:35.760 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.760 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.760 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.760 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:35.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:35.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:29:35.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:29:35.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:29:35.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:29:35.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:29:35.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:29:35.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:29:35.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:29:35.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:29:35.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:29:35.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:29:35.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:29:35.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:29:35.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:29:35.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:29:35.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:29:35.764 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.764 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.764 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.764 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:29:35.764 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:35.764 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 156.97843451, 2023-12-06T18:29:35.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:29:35.764 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.764 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.765 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:35.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:35.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:29:35.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:29:35.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:29:35.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:29:35.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:29:35.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:29:35.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:29:35.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:29:35.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:29:35.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:29:35.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:29:35.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:29:35.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.768 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:29:35.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:29:35.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.768 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.768 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:29:35.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:29:35.768 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.768 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.769 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.769 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:29:35.769 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:35.769 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 156.97342209999999, 2023-12-06T18:29:35.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:29:35.769 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.769 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.769 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:35.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:35.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:29:35.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:29:35.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:29:35.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:29:35.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:29:35.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:29:35.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:29:35.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:29:35.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:29:35.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:29:35.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:29:35.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:29:35.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:29:35.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:29:35.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:29:35.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:29:35.774 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.774 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.774 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.774 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:29:35.774 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:35.774 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 156.967895021, 2023-12-06T18:29:35.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:29:35.774 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.774 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.774 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:35.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:35.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:29:35.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:29:35.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:29:35.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:29:35.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:29:35.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:29:35.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:29:35.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:29:35.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:29:35.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:29:35.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:29:35.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:29:35.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:29:35.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:29:35.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:29:35.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:29:35.779 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.779 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.779 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.779 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:29:35.779 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:35.780 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 156.967384184, 2023-12-06T18:29:35.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:29:35.780 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.780 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.780 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:35.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:35.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:29:35.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:29:35.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:29:35.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:29:35.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:29:35.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:29:35.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:29:35.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:29:35.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:29:35.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:29:35.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:29:35.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:29:35.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:29:35.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:29:35.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:29:35.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:29:35.785 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.785 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.785 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.786 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:29:35.786 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:35.786 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 156.96692492299999, 2023-12-06T18:29:35.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:29:35.786 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.786 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.786 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:35.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:35.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:29:35.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:29:35.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:29:35.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:29:35.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:29:35.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:29:35.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:29:35.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:29:35.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:29:35.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:29:35.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:29:35.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:29:35.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:29:35.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:29:35.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:29:35.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:29:35.790 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.790 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.790 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.790 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:29:35.790 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:35.790 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 150.387009116, 2023-12-06T18:29:35.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:29:35.790 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.790 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.790 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:35.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:35.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:29:35.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:29:35.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:29:35.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:29:35.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:29:35.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:29:35.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:29:35.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:29:35.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:29:35.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:29:35.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:29:35.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:29:35.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:29:35.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:29:35.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:29:35.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:29:35.794 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.794 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.794 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.794 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:29:35.794 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:35.794 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 150.38077131099999, 2023-12-06T18:29:35.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:29:35.794 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.794 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.794 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:35.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:35.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:29:35.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:29:35.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:29:35.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:29:35.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:29:35.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:29:35.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:29:35.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:29:35.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:29:35.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:29:35.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:29:35.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:29:35.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:29:35.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:29:35.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:29:35.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:29:35.798 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.798 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.798 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.798 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:29:35.798 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:35.798 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 150.37470823199999, 2023-12-06T18:29:35.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:29:35.798 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.799 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.799 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:35.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:35.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:29:35.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:29:35.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:29:35.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:29:35.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:29:35.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:29:35.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:29:35.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:29:35.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:29:35.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:29:35.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:29:35.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:29:35.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:29:35.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:29:35.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.802 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.802 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:29:35.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:29:35.802 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.802 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.802 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.802 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:29:35.802 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:35.802 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 150.23732114699999, 2023-12-06T18:29:35.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:29:35.802 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.803 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.803 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:35.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:35.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:29:35.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:29:35.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:29:35.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:29:35.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:29:35.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:29:35.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:29:35.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:29:35.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:29:35.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:29:35.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:29:35.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:29:35.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:29:35.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:29:35.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:29:35.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:29:35.806 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.806 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.806 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.806 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:29:35.806 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:35.807 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 150.23065041999999, 2023-12-06T18:29:35.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:29:35.807 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:35.807 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:35.807 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:35.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:35.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:35.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:35.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:35.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:35.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:35.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:29:35.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:29:35.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:35.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:29:35.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:29:35.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:35.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:29:35.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:29:35.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:35.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:29:35.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:29:35.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:35.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:29:35.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:29:35.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:35.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:29:35.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:29:35.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:35.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:29:35.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:29:35.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:35.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:35.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:35.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:29:35.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:29:35.810 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.810 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.810 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.810 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:35.810 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:35.810 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:35.812 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:29:35.812 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:35.808+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:35.913 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:35.909+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:35.947 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:35.948 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:35.948 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:35.948 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:35.957 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:29:36.014 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:36.010+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:36.085 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:36.085 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:36.085 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:36.085 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:36.085 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:36.095 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:29:36.141 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:36.137+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:36.216 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:36.212+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:36.243 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:36.243 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:36.243 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:36.243 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:29:36.243 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:36.253 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:29:36.316 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:36.313+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:36.409 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:36.409 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:36.409 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:36.409 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:36.417 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:36.413+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:36.418 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:29:36.518 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:36.514+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:36.595 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:36.595 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:36.595 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:36.595 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:36.595 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:36.605 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:29:36.605 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-12-06T18:29:36.605 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi061 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c" 2023-12-06T18:29:36.605 INFO:tasks.ceph.mon.c:Stopped 2023-12-06T18:29:36.606 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-12-06T18:29:36.606 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:29:36.618 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:36.615+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:36.661 INFO:tasks.ceph.mon.a.smithi061.stderr:2023-12-06T18:29:36.645+0000 9f54640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 95122) UID: 0 2023-12-06T18:29:36.661 INFO:tasks.ceph.mon.a.smithi061.stderr:2023-12-06T18:29:36.647+0000 9f54640 -1 mon.a@0(probing) e1 *** Got Signal Terminated *** 2023-12-06T18:29:36.678 INFO:tasks.ceph.osd.7.smithi152.stderr:==00:00:00:24.612 99450== Warning: unimplemented fcntl command: 1038 2023-12-06T18:29:36.724 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:36.719+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:36.787 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:36.787 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:36.787 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:36.787 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:29:36.788 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:36.796 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:29:36.836 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:36.833+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:36.921 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:36.918+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:36.976 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:36.976 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:36.976 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:36.976 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:36.985 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:29:37.027 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:37.023+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:37.127 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:37.124+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:37.166 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:37.167 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:37.167 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:37.167 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:37.167 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:37.173 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:29:37.224 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:37.221+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:37.325 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:37.322+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:37.376 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:37.376 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:37.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:37.376 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:37.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.377 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:29:37.377 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:37.377 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 161.96104749400001, 2023-12-06T18:29:37.377 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:29:37.377 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.377 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.378 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.378 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.378 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.378 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:29:37.378 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.378 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.378 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.379 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.379 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:37.379 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.379 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.379 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.379 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.379 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:37.380 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.380 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.380 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.380 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.380 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:29:37.380 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:29:37.380 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.380 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.381 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.381 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:29:37.381 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:29:37.381 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.381 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.381 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.381 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:29:37.381 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:29:37.382 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.382 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.382 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.382 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:29:37.382 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:29:37.383 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.383 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.383 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.383 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:29:37.383 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:29:37.383 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.383 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.384 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:29:37.384 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:29:37.384 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:29:37.384 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.384 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.384 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.384 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:29:37.385 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:29:37.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.385 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.385 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.385 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:29:37.385 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:29:37.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.386 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.386 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.386 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:29:37.386 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:29:37.386 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.386 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.387 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.387 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:29:37.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:29:37.387 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.387 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.387 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.387 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:29:37.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:29:37.388 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.388 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.388 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.388 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:29:37.388 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:29:37.388 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.388 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.389 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.389 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:29:37.389 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:29:37.389 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.389 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.389 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.389 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.390 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.390 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:37.390 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:37.390 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 159.852474255, 2023-12-06T18:29:37.390 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:29:37.391 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.391 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.391 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.391 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.391 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.391 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:29:37.391 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.391 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.392 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.392 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.392 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:37.392 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.392 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.392 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.392 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.393 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:37.393 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.393 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.393 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.393 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.393 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:29:37.393 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:29:37.393 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.394 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.394 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.394 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:29:37.394 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:29:37.394 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.394 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.395 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.395 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:29:37.395 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:29:37.395 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.395 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.395 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.395 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:29:37.395 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:29:37.396 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.396 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.396 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.396 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:29:37.397 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:29:37.397 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.397 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.397 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.397 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:29:37.397 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:29:37.397 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.398 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.398 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.398 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:29:37.398 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:29:37.398 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.398 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.398 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.399 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:29:37.399 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:29:37.399 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.399 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.399 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.399 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:29:37.399 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:29:37.399 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.400 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.400 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.400 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:29:37.400 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:29:37.400 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.400 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.400 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.401 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:29:37.401 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:29:37.401 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.401 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.401 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.401 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.401 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.401 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:37.402 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:37.402 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 159.85172902299999, 2023-12-06T18:29:37.402 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:29:37.402 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.402 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.402 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.403 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.403 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.403 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:29:37.403 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.403 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.404 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.404 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:37.404 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.404 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.405 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:37.405 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.405 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.405 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.405 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.405 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:29:37.405 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:29:37.406 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.406 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.406 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.406 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:29:37.406 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:29:37.406 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.406 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:29:37.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:29:37.407 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.407 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:29:37.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:29:37.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.408 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.408 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:29:37.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:29:37.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.409 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.409 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:29:37.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:29:37.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.409 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.409 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.409 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:29:37.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:29:37.410 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.410 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.410 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.410 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:29:37.410 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:29:37.410 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.410 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.411 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.411 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:29:37.411 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:29:37.411 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.411 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.411 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:29:37.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:29:37.412 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:29:37.413 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:29:37.413 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.413 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.413 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.413 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.413 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.414 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:37.414 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:37.414 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 159.85104526800001, 2023-12-06T18:29:37.414 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:29:37.414 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.414 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.414 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.415 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.415 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.415 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:29:37.415 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.415 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.415 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.416 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.416 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:37.416 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.416 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.416 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.416 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.417 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:37.417 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.417 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.417 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.417 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.417 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:29:37.418 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:29:37.418 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.418 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.418 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.418 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:29:37.418 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:29:37.418 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.419 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.419 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.419 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:29:37.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:29:37.419 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.419 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.419 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.419 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:29:37.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:29:37.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:29:37.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:29:37.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.421 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.421 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.421 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:29:37.421 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:29:37.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.421 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.422 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.422 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:29:37.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:29:37.422 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.422 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.422 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.422 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:29:37.423 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:29:37.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.423 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.423 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:29:37.423 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:29:37.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.424 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.424 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:29:37.424 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:29:37.424 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.424 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.424 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:29:37.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:29:37.425 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.425 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.425 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.425 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:37.425 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:37.426 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 159.84547201300001, 2023-12-06T18:29:37.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:29:37.426 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.426 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.426 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.426 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.426 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.427 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:29:37.427 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.427 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.427 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.427 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.427 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:37.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.428 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.428 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.428 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:37.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.429 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.429 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.429 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:29:37.429 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:29:37.429 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.429 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.429 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.430 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:29:37.430 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:29:37.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.430 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.430 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.430 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:29:37.430 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:29:37.431 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.431 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.431 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.431 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:29:37.431 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:29:37.431 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.431 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.432 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.432 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:29:37.432 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:29:37.432 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.432 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.432 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.432 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:29:37.432 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:29:37.433 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.433 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.433 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.433 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:29:37.433 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:29:37.433 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.433 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.434 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.434 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:29:37.434 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:29:37.434 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.434 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.434 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.434 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:29:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:29:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:29:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:29:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:29:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:29:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.435 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 159.844798373, 2023-12-06T18:29:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:29:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:29:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:29:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:29:37.437 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.438 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.438 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.438 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:29:37.438 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:29:37.438 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.438 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.438 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.438 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:29:37.438 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:29:37.438 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.438 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.438 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.438 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:29:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:29:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:29:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:29:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:29:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:29:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.439 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:29:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:29:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:29:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:29:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:29:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:29:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.440 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:29:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:29:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:29:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:29:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 159.844136174, 2023-12-06T18:29:37.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:29:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:29:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:29:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:29:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:29:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:29:37.443 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:29:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:29:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:29:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:29:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:29:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:29:37.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:29:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:29:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:29:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:29:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:29:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:29:37.445 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:29:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:29:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:29:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:29:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:29:37.446 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:29:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 159.77142673899999, 2023-12-06T18:29:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:29:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.447 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:29:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.448 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:29:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:29:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:29:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:29:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:29:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:29:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:29:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:29:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:29:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:29:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:29:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:29:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.450 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:29:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:29:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:29:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:29:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:29:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:29:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.451 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:29:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:29:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:29:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:29:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:37.452 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 159.77074503, 2023-12-06T18:29:37.453 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:29:37.453 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.453 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.453 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.453 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.453 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.453 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:29:37.453 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.453 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.453 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.453 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.453 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:37.453 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.453 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:29:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:29:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:29:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:29:37.454 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:29:37.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:29:37.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:29:37.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:29:37.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:29:37.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:29:37.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:29:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:29:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:29:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:29:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:29:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:29:37.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:29:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:29:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:29:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:29:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:29:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:29:37.457 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 158.691940942, 2023-12-06T18:29:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:29:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:29:37.458 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.459 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:29:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:29:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:29:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:29:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:29:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:29:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:29:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:29:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:29:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:29:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:29:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:29:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:29:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:29:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:29:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:29:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:29:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:29:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:29:37.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:29:37.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:29:37.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:29:37.463 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.463 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.463 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.463 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:37.463 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:37.463 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 158.69052436800001, 2023-12-06T18:29:37.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:29:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:29:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.464 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:29:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:29:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:29:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:29:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:29:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:29:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:29:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:29:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:29:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:29:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:29:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:29:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:29:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:29:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:29:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:29:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:29:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:29:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:29:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:29:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:29:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:29:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 158.68943924300001, 2023-12-06T18:29:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:29:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:29:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:29:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:29:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:29:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:29:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:29:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:29:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:29:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:29:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:29:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:29:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:29:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:29:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:29:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:29:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:29:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:29:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:29:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:29:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:29:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:29:37.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:29:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:29:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 158.65429014099999, 2023-12-06T18:29:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:29:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.474 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:29:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:37.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:29:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:29:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:29:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:29:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:29:37.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:29:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:29:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:29:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:29:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:29:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:29:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:29:37.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:29:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:29:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:29:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:29:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:29:37.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:29:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:29:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:29:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:29:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:29:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.479 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 152.10455961599999, 2023-12-06T18:29:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:29:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:29:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:37.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:29:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:29:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:29:37.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:29:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:29:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:29:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:29:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:29:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:29:37.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:29:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:29:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:29:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:29:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:29:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:29:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:29:37.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:29:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:29:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:29:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:29:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:29:37.484 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:29:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 152.10375400800001, 2023-12-06T18:29:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:29:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.485 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:29:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:29:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:29:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:29:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:29:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:29:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:29:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:29:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:29:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:29:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:29:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:29:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:29:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:29:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:29:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:29:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:29:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:29:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:29:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:29:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:29:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:29:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:29:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:37.490 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 152.10306319899999, 2023-12-06T18:29:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:29:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:29:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:29:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:29:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:29:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:29:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:29:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:29:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:29:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:29:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:29:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:29:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:29:37.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:29:37.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:29:37.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:29:37.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:29:37.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:29:37.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:29:37.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:29:37.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:29:37.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:29:37.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:29:37.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:29:37.495 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.495 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.495 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.496 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:37.496 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:37.496 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 152.09876569100001, 2023-12-06T18:29:37.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:29:37.496 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.496 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.496 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.496 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.496 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.496 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:29:37.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.496 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:37.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:37.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:29:37.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:29:37.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:29:37.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:29:37.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:29:37.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:29:37.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:29:37.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:29:37.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:29:37.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:29:37.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:29:37.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:29:37.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:29:37.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:29:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:29:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:29:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:29:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:29:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:29:37.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:29:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:29:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:29:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 152.09809801599999, 2023-12-06T18:29:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:29:37.501 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:29:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:29:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:29:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:29:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:29:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:29:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:29:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:29:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:29:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:29:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:29:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:29:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:29:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:29:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:29:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:29:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:29:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:29:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:29:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:29:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:29:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:29:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:29:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.506 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 152.09744258800001, 2023-12-06T18:29:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:29:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:29:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.507 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:29:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:29:37.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:29:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:29:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:29:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:29:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:29:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:29:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.510 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.510 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:29:37.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:29:37.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.510 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.510 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:29:37.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:29:37.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.510 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.510 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:29:37.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:29:37.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:29:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:29:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:29:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:29:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:29:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:29:37.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:29:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:29:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 152.056788041, 2023-12-06T18:29:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:29:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:37.512 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:29:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:37.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:29:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:29:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:29:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:29:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:37.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:29:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:29:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:29:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:29:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:29:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:29:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:29:37.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:29:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:29:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:29:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:29:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:29:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:29:37.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:29:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:29:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:29:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:29:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:29:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:37.517 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:37.518 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:37.519 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:29:37.520 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:54.442 95127== 2023-12-06T18:29:37.520 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:54.442 95127== Exit program on first error (--exit-on-first-error=yes) 2023-12-06T18:29:37.520 INFO:tasks.ceph.mon.a.smithi061.stderr:daemon-helper: command failed with exit status 42 2023-12-06T18:29:37.521 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:37.515+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:37.620 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:37.616+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:37.697 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:37.697 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:37.697 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:37.697 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:37.707 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:29:37.721 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:37.717+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:37.822 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:37.819+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:37.885 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:37.886 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:37.886 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:37.886 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:37.886 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:37.895 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:29:37.923 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:37.920+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:38.024 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:38.021+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:38.035 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:38.035 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:38.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:38.035 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:38.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.035 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:29:38.035 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:38.035 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 136.21905898700001, 2023-12-06T18:29:38.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:29:38.035 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:38.035 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:29:38.035 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:38.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:29:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:29:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:29:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:29:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:29:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:29:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:29:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:29:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.037 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:29:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:29:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:29:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:29:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:29:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:29:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:38.038 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:38.039 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:38.039 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:38.039 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:38.039 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:38.044 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:29:38.126 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:38.122+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:38.182 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:38.182 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:38.183 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:38.183 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:38.194 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:29:38.239 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:38.236+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:38.327 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:38.324+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:38.333 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:38.333 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:38.333 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:38.333 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:38.333 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:38.342 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:29:38.428 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:38.425+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:38.523 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 17.426074101000001, 2023-12-06T18:29:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:29:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:29:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:29:38.524 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:38.525 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:29:38.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:38.525 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:38.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:38.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:38.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:38.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:38.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:38.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:38.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:38.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:38.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:38.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:29:38.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:29:38.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:38.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:38.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:29:38.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:29:38.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:38.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:38.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:29:38.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:29:38.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:29:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:29:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:29:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:29:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:29:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:29:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:38.528 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:38.528 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:38.528 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:38.528 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:38.528 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:38.529 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:38.526+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:38.534 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:29:38.630 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:38.626+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:38.664 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:38.664 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:38.664 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:38.664 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:38.673 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:29:38.731 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:38.728+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:38.805 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:38.806 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:38.806 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:38.806 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:38.806 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:38.815 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:29:38.832 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:38.829+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:38.933 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:38.930+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:38.945 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:38.945 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:38.945 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:38.946 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:38.946 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:38.946 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:38.946 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:38.946 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 77.452108937000006, 2023-12-06T18:29:38.946 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:29:38.946 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:38.946 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:29:38.946 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:29:38.946 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:29:38.946 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:38.946 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:29:38.946 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:38.946 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:38.947 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:38.947 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:38.947 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:38.947 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:38.947 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:38.947 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:38.947 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:38.947 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:38.947 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:38.947 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:38.947 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:38.947 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:38.947 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:29:38.947 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:29:38.947 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:38.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:38.948 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:38.948 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:29:38.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:29:38.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:38.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:38.948 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:38.948 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:29:38.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:29:38.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:38.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:38.948 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:38.948 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:29:38.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:29:38.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:38.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:29:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:29:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:29:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:29:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:38.949 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:38.955 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:29:39.040 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:39.036+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:39.135 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:39.132+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:39.143 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:39.143 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:39.143 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:39.143 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:39.152 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:29:39.236 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:39.232+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:39.321 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:39.321 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:39.321 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:39.321 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:39.321 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:39.330 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:29:39.375 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:39.371+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:39.438 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:39.434+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:39.509 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:39.509 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:39.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:39.509 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:39.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.509 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:29:39.509 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:39.510 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 162.88925427199999, 2023-12-06T18:29:39.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:29:39.510 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.510 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.510 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:39.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:39.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:29:39.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:29:39.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:29:39.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:29:39.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:29:39.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:29:39.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:29:39.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:29:39.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:29:39.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:29:39.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:29:39.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:29:39.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:29:39.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:29:39.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:29:39.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:29:39.513 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.514 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.514 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.514 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:29:39.514 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:39.514 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 161.86642726299999, 2023-12-06T18:29:39.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:29:39.514 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.514 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.514 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:39.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:39.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:29:39.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:29:39.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:29:39.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:29:39.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:29:39.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:29:39.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:29:39.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:29:39.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:29:39.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:29:39.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:29:39.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:29:39.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:29:39.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:29:39.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:29:39.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:29:39.518 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.518 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.518 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.518 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:29:39.518 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:39.518 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 161.854380625, 2023-12-06T18:29:39.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:29:39.519 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.519 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.519 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:39.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:39.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:29:39.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:29:39.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:29:39.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:29:39.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:29:39.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:29:39.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:29:39.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:29:39.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:29:39.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:29:39.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:29:39.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:29:39.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.522 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:29:39.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:29:39.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.522 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.522 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:29:39.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:29:39.522 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.522 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.522 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.522 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:29:39.522 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:39.522 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 161.832980018, 2023-12-06T18:29:39.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:29:39.523 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.523 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.523 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:39.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:39.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:29:39.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:29:39.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:29:39.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:29:39.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:29:39.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:29:39.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:29:39.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:29:39.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:29:39.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:29:39.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:29:39.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:29:39.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.526 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.526 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:29:39.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:29:39.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.526 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.526 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:29:39.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:29:39.526 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.526 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.526 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.526 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:29:39.527 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:39.527 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 161.831800695, 2023-12-06T18:29:39.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:29:39.527 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.527 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.527 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:39.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:39.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:29:39.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:29:39.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:29:39.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:29:39.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:29:39.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:29:39.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:29:39.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:29:39.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:29:39.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:29:39.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:29:39.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:29:39.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.530 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:29:39.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:29:39.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.530 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:29:39.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:29:39.530 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.530 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.531 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.531 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:29:39.531 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:39.531 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 161.83122871399999, 2023-12-06T18:29:39.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:29:39.531 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.531 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.531 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:39.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:39.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:29:39.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:29:39.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:29:39.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:29:39.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:29:39.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:29:39.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.533 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.533 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:29:39.533 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:29:39.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.533 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.533 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:29:39.533 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:29:39.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.533 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.533 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:29:39.533 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:29:39.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.533 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.534 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:29:39.534 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:29:39.534 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.534 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.534 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.534 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:29:39.534 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:29:39.534 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.534 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.534 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.534 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.534 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.534 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:29:39.534 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:39.534 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 161.385393244, 2023-12-06T18:29:39.535 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:29:39.535 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.535 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.535 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.535 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.535 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.535 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:39.535 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.535 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.535 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.535 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.535 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:39.535 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.535 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.535 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.535 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:29:39.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:29:39.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:29:39.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:29:39.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:29:39.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:29:39.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:29:39.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:29:39.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.537 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:29:39.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:29:39.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.537 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:29:39.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:29:39.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.537 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:29:39.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:29:39.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.538 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.538 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.538 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:29:39.538 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:29:39.538 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.538 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.538 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.538 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.538 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:29:39.538 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:39.538 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 161.38379338799999, 2023-12-06T18:29:39.538 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:29:39.538 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.538 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.539 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.539 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.539 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.539 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:39.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.539 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.539 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.539 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:39.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.539 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.539 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.539 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:29:39.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:29:39.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:29:39.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:29:39.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:29:39.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:29:39.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:29:39.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:29:39.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:29:39.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:29:39.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:29:39.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:29:39.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:29:39.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:29:39.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.542 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.542 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.542 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:29:39.542 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:29:39.542 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.542 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.542 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.542 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.542 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:29:39.542 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:39.542 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 161.00458777200001, 2023-12-06T18:29:39.542 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:29:39.542 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.543 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.543 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.543 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.543 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:39.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.543 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.543 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:39.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.543 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.543 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:29:39.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:29:39.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.544 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.544 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.544 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:29:39.544 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:29:39.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.544 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.544 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.544 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:29:39.544 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:29:39.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.544 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.544 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.544 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:29:39.544 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:29:39.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.544 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.545 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:29:39.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:29:39.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.545 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:29:39.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:29:39.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.545 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:29:39.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:29:39.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.546 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:29:39.546 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:29:39.546 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.546 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.546 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.546 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.546 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.546 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:29:39.546 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:39.546 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 160.76708558999999, 2023-12-06T18:29:39.546 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:29:39.546 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.546 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.546 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.546 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.546 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.547 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:39.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.547 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.547 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:39.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.547 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.547 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:29:39.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:29:39.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.548 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:29:39.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:29:39.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.548 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:29:39.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:29:39.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.548 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:29:39.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:29:39.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:29:39.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:29:39.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:29:39.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:29:39.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:29:39.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:29:39.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.550 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.550 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.550 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:29:39.550 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:29:39.550 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.550 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.550 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.550 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.550 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:29:39.550 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:39.550 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 160.766357591, 2023-12-06T18:29:39.550 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:29:39.550 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.551 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.551 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.551 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.551 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.551 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:39.551 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.551 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.551 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.551 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.551 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:39.551 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.551 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.551 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.552 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.552 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:29:39.552 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:29:39.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.552 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.552 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:29:39.552 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:29:39.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.552 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.552 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:29:39.552 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:29:39.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:29:39.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:29:39.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:29:39.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:29:39.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:29:39.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:29:39.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.554 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.554 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:29:39.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:29:39.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.554 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.554 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:29:39.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:29:39.554 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.555 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.555 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.555 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.555 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:29:39.555 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:39.555 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 160.761345181, 2023-12-06T18:29:39.555 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:29:39.555 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.555 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.555 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.555 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.555 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.556 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:39.556 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.556 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.556 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.556 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:39.556 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.556 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.556 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.556 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:29:39.556 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:29:39.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.556 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.557 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:29:39.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:29:39.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.557 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:29:39.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:29:39.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.557 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:29:39.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:29:39.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.558 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.558 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:29:39.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:29:39.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.558 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.558 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.558 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:29:39.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:29:39.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.558 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.558 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.558 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:29:39.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:29:39.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.559 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.559 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.559 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:29:39.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:29:39.559 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.559 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.559 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.559 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.559 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:29:39.559 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:39.559 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 160.75581810200001, 2023-12-06T18:29:39.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:29:39.559 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.559 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.560 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.560 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.560 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.560 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:39.560 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.560 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.560 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.560 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.560 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:39.560 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.560 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.560 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.560 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.560 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:29:39.561 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:29:39.561 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.561 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.561 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.561 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:29:39.561 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:29:39.561 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.561 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.561 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.561 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:29:39.561 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:29:39.561 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.561 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.561 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.561 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:29:39.561 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:29:39.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.562 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.562 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.562 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:29:39.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:29:39.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.562 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.562 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.562 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:29:39.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:29:39.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.562 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.562 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.562 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:29:39.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:29:39.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.563 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.563 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.563 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:29:39.563 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:29:39.563 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.563 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.563 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.563 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.563 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:29:39.563 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:39.563 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 160.755307265, 2023-12-06T18:29:39.563 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:29:39.564 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.564 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.564 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.564 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.564 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:39.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.564 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.564 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:39.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.565 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:29:39.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:29:39.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.565 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:29:39.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:29:39.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.565 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:29:39.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:29:39.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:29:39.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:29:39.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:29:39.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:29:39.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:29:39.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:29:39.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:29:39.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:29:39.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:29:39.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:29:39.567 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.567 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.567 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.567 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:29:39.568 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:39.568 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 160.754848004, 2023-12-06T18:29:39.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:29:39.568 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.568 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.568 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.568 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.568 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:39.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.568 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.568 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:39.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:29:39.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:29:39.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:29:39.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:29:39.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:29:39.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:29:39.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:29:39.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:29:39.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:29:39.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:29:39.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:29:39.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:29:39.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.571 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:29:39.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:29:39.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.571 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:29:39.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:29:39.571 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.571 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.571 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.572 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:29:39.572 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:39.572 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 154.174932197, 2023-12-06T18:29:39.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:29:39.572 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.572 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.572 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.572 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.572 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:39.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.572 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.572 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:39.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.573 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.573 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.573 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:29:39.573 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:29:39.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.573 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.573 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.573 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:29:39.573 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:29:39.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.573 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.573 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.573 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:29:39.573 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:29:39.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.573 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:29:39.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:29:39.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:29:39.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:29:39.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:29:39.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:29:39.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.575 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.575 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:29:39.575 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:29:39.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.575 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.575 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.575 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:29:39.575 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:29:39.575 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.575 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.575 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.575 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.575 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:29:39.575 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:39.575 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 154.16869439199999, 2023-12-06T18:29:39.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:29:39.576 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.576 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.576 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.576 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:39.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.576 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:39.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.577 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:29:39.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:29:39.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.577 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.577 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:29:39.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:29:39.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.577 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.577 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:29:39.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:29:39.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.577 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.577 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:29:39.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:29:39.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.578 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.578 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.578 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:29:39.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:29:39.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.578 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.578 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.578 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:29:39.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:29:39.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.578 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.578 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.578 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:29:39.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:29:39.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.579 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.579 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:29:39.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:29:39.579 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.579 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.579 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.579 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:29:39.579 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:39.579 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 154.16263131299999, 2023-12-06T18:29:39.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:29:39.579 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.579 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.579 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:39.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:39.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:29:39.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:29:39.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:29:39.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:29:39.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:29:39.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:29:39.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:29:39.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:29:39.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:29:39.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:29:39.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:29:39.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:29:39.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:29:39.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:29:39.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:29:39.583 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:29:39.583 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.583 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.583 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.583 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.583 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.583 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:29:39.583 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:39.583 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 154.02524422799999, 2023-12-06T18:29:39.583 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:29:39.583 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.583 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.583 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.583 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.583 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.583 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:39.583 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.584 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.584 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.584 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:39.584 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.584 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.584 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.584 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:29:39.584 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:29:39.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.584 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.584 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:29:39.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:29:39.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:29:39.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:29:39.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:29:39.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:29:39.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.586 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.586 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.586 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:29:39.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:29:39.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.586 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.586 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.586 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:29:39.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:29:39.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.586 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.586 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.586 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:29:39.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:29:39.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.587 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.587 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.587 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:29:39.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:29:39.587 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.587 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.587 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.587 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.587 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:29:39.587 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:39.587 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 154.01857350099999, 2023-12-06T18:29:39.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:29:39.587 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:39.587 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:39.587 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:39.588 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.588 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:39.588 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:39.588 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.588 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.588 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.588 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:39.588 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:39.588 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:39.588 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.588 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.588 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:39.588 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:29:39.588 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:29:39.588 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.589 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.589 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:39.589 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:29:39.589 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:29:39.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.589 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.589 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:39.589 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:29:39.589 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:29:39.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.589 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.589 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:39.589 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:29:39.589 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:29:39.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.590 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.590 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:39.590 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:29:39.590 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:29:39.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.590 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.590 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:39.590 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:29:39.590 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:29:39.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.590 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.590 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:39.590 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:29:39.591 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:29:39.591 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:39.591 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:39.591 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:39.591 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:29:39.591 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:29:39.591 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.591 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.591 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.591 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:39.591 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:39.591 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:39.593 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:29:39.593 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:39.589+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:39.693 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:39.690+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:39.780 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:39.781 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:39.781 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:39.781 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:39.790 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:29:39.795 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:39.791+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:39.895 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:39.892+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:39.972 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:39.972 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:39.972 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:39.972 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:39.972 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:39.981 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:29:39.996 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:39.992+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:40.097 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:40.093+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:40.125 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:40.126 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:40.126 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:40.126 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:29:40.126 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:40.135 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:29:40.197 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:40.194+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:40.229 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:29:40.281 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:40.281 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:40.281 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:40.282 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:40.291 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:29:40.332 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:40.328+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:40.400 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:40.396+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:40.463 INFO:teuthology.orchestra.run.smithi152.stderr:no valid command found; 10 closest matches: 2023-12-06T18:29:40.463 INFO:teuthology.orchestra.run.smithi152.stderr:0 2023-12-06T18:29:40.463 INFO:teuthology.orchestra.run.smithi152.stderr:1 2023-12-06T18:29:40.463 INFO:teuthology.orchestra.run.smithi152.stderr:2 2023-12-06T18:29:40.463 INFO:teuthology.orchestra.run.smithi152.stderr:abort 2023-12-06T18:29:40.463 INFO:teuthology.orchestra.run.smithi152.stderr:assert 2023-12-06T18:29:40.463 INFO:teuthology.orchestra.run.smithi152.stderr:bluefs debug_inject_read_zeros 2023-12-06T18:29:40.463 INFO:teuthology.orchestra.run.smithi152.stderr:bluefs files list 2023-12-06T18:29:40.463 INFO:teuthology.orchestra.run.smithi152.stderr:bluefs stats 2023-12-06T18:29:40.463 INFO:teuthology.orchestra.run.smithi152.stderr:bluestore allocator dump block 2023-12-06T18:29:40.463 INFO:teuthology.orchestra.run.smithi152.stderr:bluestore allocator fragmentation block 2023-12-06T18:29:40.464 INFO:teuthology.orchestra.run.smithi152.stderr:admin_socket: invalid command 2023-12-06T18:29:40.465 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-12-06T18:29:40.466 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-7, ['dump_ops_in_flight'] 2023-12-06T18:29:40.468 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:40.468 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:40.468 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:40.468 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:40.468 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:40.477 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:29:40.520 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:40.515+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:40.601 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:40.598+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:40.658 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:40.658 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:40.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:40.658 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:29:40.658 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:40.667 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:29:40.702 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:40.698+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:40.803 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:40.799+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:40.845 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:40.845 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:40.845 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:40.845 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:40.855 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:29:40.888 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-12-06T18:29:40.888 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-12-06T18:29:40.940 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:40.936+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:41.004 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:41.001+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:41.031 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:41.031 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:41.031 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:41.031 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:41.031 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:41.040 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:29:41.105 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:41.102+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:41.206 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:41.203+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:41.275 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:41.275 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:41.275 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:41.275 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:41.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.275 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:29:41.275 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:41.275 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 165.86445206499999, 2023-12-06T18:29:41.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:29:41.276 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.276 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.276 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.276 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.276 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.276 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:29:41.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.276 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.276 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:41.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.277 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:41.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.277 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.277 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.277 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.277 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:29:41.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:29:41.277 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.277 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.277 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.277 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:29:41.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:29:41.277 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.277 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.277 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.277 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:29:41.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:29:41.278 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.278 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.278 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.278 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:29:41.278 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:29:41.278 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.278 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.278 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.278 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:29:41.278 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:29:41.278 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.278 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.278 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:29:41.278 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:29:41.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:29:41.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.279 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:29:41.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:29:41.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.279 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:29:41.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:29:41.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.279 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:29:41.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:29:41.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:29:41.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:29:41.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:29:41.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:29:41.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:29:41.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:29:41.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.281 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:29:41.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:29:41.281 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.281 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.281 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.281 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:41.281 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:41.281 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 163.75587882600001, 2023-12-06T18:29:41.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:29:41.281 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.281 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.281 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.281 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.282 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.282 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:29:41.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.282 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:41.282 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:41.282 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.283 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.283 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.283 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.283 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:29:41.283 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:29:41.283 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.283 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.283 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.283 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:29:41.283 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:29:41.283 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.283 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.283 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.283 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:29:41.283 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:29:41.283 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.284 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.284 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:29:41.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:29:41.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.284 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.284 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:29:41.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:29:41.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.284 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.284 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:29:41.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:29:41.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.284 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.284 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:29:41.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:29:41.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.285 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.285 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:29:41.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:29:41.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.285 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.285 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:29:41.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:29:41.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.285 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.285 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:29:41.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:29:41.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.286 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:29:41.286 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:29:41.286 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.286 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.286 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.286 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:41.286 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:41.286 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 163.755133594, 2023-12-06T18:29:41.286 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:29:41.286 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.286 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.287 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.287 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.287 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.287 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:29:41.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.287 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:41.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:41.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.288 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.288 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.288 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:29:41.288 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:29:41.288 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.288 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.288 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.288 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:29:41.288 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:29:41.288 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.288 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.288 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.288 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:29:41.288 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:29:41.288 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.288 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.288 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.289 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:29:41.289 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:29:41.289 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.289 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.289 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.289 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:29:41.289 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:29:41.289 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.289 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.289 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.289 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:29:41.289 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:29:41.289 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.289 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.289 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.289 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:29:41.290 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:29:41.290 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.290 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.290 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:29:41.290 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:29:41.290 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.290 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.290 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:29:41.290 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:29:41.290 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.290 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.290 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:29:41.291 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:29:41.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.291 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.291 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.291 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:29:41.291 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:29:41.291 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.291 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.291 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.291 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.291 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:41.291 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:41.291 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 163.75444983899999, 2023-12-06T18:29:41.291 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:29:41.291 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.291 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.292 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.292 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.292 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.292 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:29:41.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.292 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.292 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.292 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:41.292 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.292 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.292 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:41.292 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.293 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.293 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:29:41.293 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:29:41.293 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.293 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.293 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.293 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:29:41.293 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:29:41.293 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.293 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.293 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.293 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:29:41.293 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:29:41.293 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.293 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.293 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.294 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:29:41.294 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:29:41.294 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.294 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.294 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.294 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:29:41.294 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:29:41.294 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.294 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.294 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.294 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:29:41.294 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:29:41.294 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.294 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.294 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.294 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:29:41.295 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:29:41.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.295 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.295 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.295 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:29:41.295 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:29:41.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.295 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.295 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.295 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:29:41.295 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:29:41.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.295 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.295 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.295 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:29:41.295 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:29:41.296 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.296 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.296 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.296 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:29:41.296 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:29:41.296 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.296 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.296 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.296 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.296 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.296 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:41.296 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:41.296 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 163.74887658399999, 2023-12-06T18:29:41.296 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:29:41.296 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.296 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.297 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.297 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.297 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.297 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:29:41.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.297 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.297 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.297 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.297 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:41.297 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.297 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.297 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.297 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:41.297 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.298 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.298 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.298 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:29:41.298 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:29:41.298 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.298 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.298 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.298 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:29:41.298 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:29:41.298 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.298 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.298 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.298 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:29:41.298 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:29:41.298 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.298 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.299 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.299 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:29:41.299 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:29:41.299 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.299 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.299 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.299 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:29:41.299 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:29:41.299 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.299 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.299 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.299 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:29:41.299 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:29:41.299 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.299 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.299 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.300 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:29:41.300 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:29:41.300 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.300 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.300 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.300 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:29:41.300 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:29:41.300 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.300 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.300 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.300 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:29:41.300 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:29:41.300 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.300 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.300 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.300 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:29:41.301 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:29:41.301 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.301 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.301 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.301 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:29:41.301 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:29:41.301 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.301 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.301 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.301 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.301 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.301 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:41.301 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:41.301 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 163.74820294400001, 2023-12-06T18:29:41.301 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:29:41.301 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.301 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.302 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.302 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.302 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.302 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:29:41.302 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.302 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.302 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.302 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.302 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:41.302 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.302 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.302 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.302 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.302 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:41.302 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.303 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:29:41.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:29:41.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.303 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:29:41.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:29:41.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.303 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:29:41.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:29:41.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:29:41.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:29:41.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:29:41.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:29:41.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:29:41.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:29:41.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:29:41.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:29:41.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:29:41.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:29:41.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:29:41.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:29:41.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:29:41.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:29:41.306 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.306 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:29:41.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:29:41.306 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.306 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.306 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.306 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.306 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:41.306 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:41.306 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 163.74754074500001, 2023-12-06T18:29:41.307 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:29:41.307 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.307 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.307 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.307 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.307 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.307 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:29:41.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.307 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.307 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.307 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.307 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:41.307 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.307 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.307 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.307 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:41.308 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.308 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.308 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.308 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.308 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:29:41.308 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:29:41.308 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.308 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.308 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.308 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:29:41.308 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:29:41.308 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.308 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.308 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.308 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:29:41.308 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:29:41.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.309 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.309 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:29:41.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:29:41.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.309 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.309 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:29:41.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:29:41.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.309 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.309 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:29:41.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:29:41.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.310 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.310 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:29:41.310 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:29:41.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.310 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.310 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.310 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:29:41.310 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:29:41.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.310 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.310 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.310 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:29:41.310 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:29:41.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.310 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.310 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:29:41.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:29:41.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.311 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:29:41.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:29:41.311 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.311 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.311 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.311 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:41.311 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:41.311 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 163.67483131, 2023-12-06T18:29:41.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:29:41.311 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.312 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.312 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.312 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.312 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.312 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:29:41.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.312 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:41.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:41.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.313 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:29:41.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:29:41.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.313 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:29:41.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:29:41.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.313 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:29:41.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:29:41.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:29:41.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:29:41.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:29:41.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:29:41.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:29:41.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:29:41.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.315 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.315 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:29:41.315 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:29:41.315 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.315 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.315 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.315 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:29:41.315 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:29:41.315 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.315 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.315 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.315 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:29:41.315 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:29:41.315 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.315 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.315 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.315 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:29:41.316 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:29:41.316 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.316 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.316 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.316 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:29:41.316 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:29:41.316 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.316 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.316 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.316 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.316 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.316 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:41.316 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:41.316 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 163.67414960100001, 2023-12-06T18:29:41.316 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:29:41.316 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.316 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.317 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.317 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.317 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.317 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:29:41.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.317 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.317 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.317 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:41.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.317 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.317 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:41.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:29:41.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:29:41.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:29:41.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:29:41.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:29:41.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:29:41.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:29:41.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:29:41.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:29:41.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:29:41.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:29:41.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:29:41.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:29:41.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:29:41.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:29:41.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:29:41.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:29:41.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:29:41.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:29:41.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:29:41.321 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.321 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.321 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.321 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:29:41.321 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:29:41.321 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.321 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.321 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.321 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.321 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.321 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:41.321 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:41.321 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 162.59534551300001, 2023-12-06T18:29:41.321 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:29:41.321 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.321 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.321 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.321 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.322 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.322 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:29:41.322 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.322 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.322 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.322 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.322 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:41.322 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.322 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.322 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.322 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.322 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:41.322 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.322 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.322 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.323 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.323 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:29:41.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:29:41.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.323 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.323 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.323 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:29:41.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:29:41.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.323 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.323 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.323 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:29:41.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:29:41.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.323 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.323 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.323 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:29:41.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:29:41.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.324 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:29:41.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:29:41.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.324 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:29:41.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:29:41.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.324 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:29:41.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:29:41.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:29:41.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:29:41.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:29:41.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:29:41.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:29:41.326 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:29:41.326 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.326 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.326 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.326 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:29:41.326 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:29:41.326 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.326 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.326 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.326 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.326 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.326 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:41.326 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:41.326 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 162.59392893899999, 2023-12-06T18:29:41.326 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:29:41.327 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.327 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.327 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.327 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.327 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.327 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:29:41.327 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.327 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.327 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.327 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.327 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:41.327 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.327 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.327 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.327 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.327 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:41.327 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.328 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.328 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.328 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:29:41.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:29:41.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.328 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.328 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.328 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:29:41.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:29:41.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.328 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.328 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.328 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:29:41.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:29:41.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.328 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.329 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:29:41.329 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:29:41.329 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.329 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:29:41.329 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:29:41.329 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.329 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:29:41.329 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:29:41.329 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.329 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:29:41.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:29:41.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:29:41.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:29:41.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:29:41.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:29:41.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:29:41.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:29:41.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.331 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.331 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.331 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:29:41.331 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:29:41.331 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.331 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.331 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.331 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.331 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:41.331 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:41.331 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 162.59284381399999, 2023-12-06T18:29:41.331 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:29:41.331 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.331 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.332 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.332 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.332 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.332 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:29:41.332 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.332 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.332 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.332 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.332 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:41.332 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.332 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.332 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.332 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.332 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:41.332 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.332 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.332 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.333 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.333 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:29:41.333 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:29:41.333 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.333 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.333 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.333 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:29:41.333 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:29:41.333 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.333 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.333 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.333 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:29:41.333 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:29:41.333 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.333 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.333 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.333 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:29:41.333 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:29:41.334 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.334 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.334 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.334 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:29:41.334 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:29:41.334 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.334 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.334 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.334 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:29:41.334 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:29:41.334 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.334 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.334 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.334 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:29:41.335 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:29:41.335 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.335 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.335 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.335 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:29:41.335 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:29:41.335 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.335 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.335 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.335 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:29:41.335 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:29:41.335 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.335 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.335 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.335 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:29:41.336 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:29:41.336 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.336 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.336 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.336 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:29:41.336 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:29:41.336 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.336 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.336 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.336 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.336 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.336 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:41.336 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:41.336 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 162.557694712, 2023-12-06T18:29:41.336 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:29:41.336 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.336 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.337 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.337 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.337 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.337 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:29:41.337 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.337 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.337 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.337 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.337 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:41.337 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.337 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.337 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.337 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.337 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:41.337 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.337 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.338 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.338 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.338 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:29:41.338 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:29:41.338 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.338 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.338 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.338 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:29:41.338 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:29:41.338 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.338 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.338 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.338 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:29:41.338 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:29:41.338 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.339 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.339 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.339 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:29:41.339 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:29:41.339 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.339 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.339 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.339 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:29:41.339 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:29:41.339 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.339 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.339 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.339 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:29:41.339 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:29:41.339 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.339 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.340 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.340 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:29:41.340 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:29:41.340 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.340 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.340 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.340 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:29:41.340 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:29:41.340 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.340 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.340 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.340 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:29:41.340 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:29:41.340 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.340 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.340 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.340 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:29:41.341 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:29:41.341 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.341 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.341 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.341 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:29:41.341 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:29:41.341 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.341 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.341 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.341 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.341 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.341 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:41.341 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:41.341 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 156.007964187, 2023-12-06T18:29:41.341 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:29:41.341 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.341 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.341 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.342 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.342 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.342 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:29:41.342 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.342 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.342 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.342 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.342 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:41.342 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.342 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.342 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.342 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.342 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:41.342 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.342 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.343 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.343 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.343 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:29:41.343 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:29:41.343 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.343 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.343 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.343 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:29:41.343 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:29:41.343 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.343 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.343 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.343 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:29:41.343 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:29:41.343 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.343 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.343 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.344 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:29:41.344 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:29:41.344 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.344 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.344 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.344 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:29:41.344 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:29:41.344 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.344 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.344 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.344 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:29:41.344 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:29:41.344 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.344 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.344 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.344 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:29:41.345 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:29:41.345 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.345 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.345 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.345 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:29:41.345 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:29:41.345 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.345 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.345 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.345 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:29:41.345 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:29:41.345 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.345 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.345 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.345 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:29:41.345 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:29:41.345 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.346 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.346 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.346 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:29:41.346 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:29:41.346 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.346 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.346 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.346 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.346 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.346 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:41.346 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:41.346 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 156.00715857899999, 2023-12-06T18:29:41.346 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:29:41.346 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.346 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.347 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.347 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.347 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.347 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:29:41.347 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.347 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.347 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.347 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.347 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:41.347 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.347 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.347 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.347 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.347 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:41.347 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.347 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.348 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.348 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.348 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:29:41.348 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:29:41.348 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.348 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.348 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.348 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:29:41.348 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:29:41.348 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.348 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.348 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.348 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:29:41.348 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:29:41.348 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.348 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.348 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.349 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:29:41.349 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:29:41.349 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.349 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.349 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.349 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:29:41.349 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:29:41.349 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.349 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.349 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.349 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:29:41.349 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:29:41.349 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.349 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.349 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.349 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:29:41.349 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:29:41.349 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.350 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.350 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.350 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:29:41.350 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:29:41.350 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.350 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.350 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.350 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:29:41.350 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:29:41.350 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.350 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.350 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.350 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:29:41.350 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:29:41.350 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.351 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:29:41.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:29:41.351 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.351 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.351 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.351 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:41.351 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:41.351 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 156.00646777, 2023-12-06T18:29:41.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:29:41.351 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.351 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.351 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.351 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.352 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.352 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:29:41.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.352 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:41.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:41.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:29:41.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:29:41.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:29:41.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:29:41.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:29:41.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:29:41.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:29:41.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:29:41.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.354 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.354 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:29:41.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:29:41.354 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.354 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.354 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.354 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:29:41.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:29:41.354 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.354 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.354 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.354 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:29:41.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:29:41.354 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.354 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.354 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.355 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:29:41.355 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:29:41.355 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.355 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.355 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.355 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:29:41.355 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:29:41.355 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.355 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.355 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.355 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:29:41.355 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:29:41.355 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.355 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.355 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.355 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:29:41.355 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:29:41.355 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.356 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.356 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.356 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.356 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.356 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:41.356 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:41.356 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 156.00217026199999, 2023-12-06T18:29:41.356 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:29:41.356 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.356 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.356 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.356 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.356 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.356 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:29:41.356 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.356 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.356 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:41.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.357 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:41.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.357 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:29:41.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:29:41.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.357 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:29:41.358 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:29:41.358 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.358 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.358 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.358 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:29:41.358 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:29:41.358 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.358 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.358 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.358 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:29:41.358 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:29:41.358 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.358 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.358 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.358 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:29:41.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:29:41.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.359 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.359 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.359 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:29:41.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:29:41.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.359 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.359 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.359 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:29:41.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:29:41.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.359 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.359 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.359 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:29:41.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:29:41.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.360 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.360 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.360 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:29:41.360 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:29:41.360 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.360 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.360 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.360 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:29:41.360 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:29:41.360 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.360 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.360 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.360 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:29:41.360 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:29:41.360 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.360 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.360 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.361 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.361 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.361 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:41.361 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:41.361 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 156.001502587, 2023-12-06T18:29:41.361 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:29:41.361 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.361 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.361 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.361 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.361 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.361 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:29:41.361 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.361 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.361 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.361 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.361 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:41.361 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.362 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.362 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.362 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:41.362 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.362 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.362 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.362 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:29:41.362 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:29:41.362 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.362 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.362 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:29:41.362 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:29:41.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:29:41.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:29:41.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:29:41.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:29:41.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:29:41.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:29:41.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:29:41.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:29:41.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:29:41.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:29:41.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:29:41.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:29:41.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:29:41.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:29:41.365 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.365 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:29:41.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:29:41.365 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.365 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:29:41.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:29:41.365 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.365 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.365 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.365 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.366 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.366 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:41.366 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:41.366 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 156.00084715899999, 2023-12-06T18:29:41.366 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:29:41.366 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.366 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.366 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.366 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.366 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.366 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:29:41.366 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.366 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.366 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.366 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.367 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:41.367 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.367 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.367 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.367 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.367 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:41.367 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.367 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.367 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.367 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.367 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:29:41.367 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:29:41.367 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.367 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.367 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.367 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:29:41.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:29:41.368 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.368 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.368 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:29:41.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:29:41.368 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.368 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.368 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:29:41.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:29:41.368 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.368 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.368 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:29:41.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:29:41.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.369 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.369 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.369 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:29:41.369 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:29:41.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.369 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.369 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.369 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:29:41.369 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:29:41.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.369 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.369 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.369 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:29:41.369 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:29:41.370 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.370 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.370 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.370 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:29:41.370 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:29:41.370 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.370 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.370 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.370 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:29:41.370 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:29:41.370 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.370 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.370 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.370 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:29:41.370 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:29:41.371 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.371 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.371 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.371 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.371 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.371 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:41.371 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:41.371 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 155.96019261199999, 2023-12-06T18:29:41.371 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:29:41.371 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.371 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:41.371 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:41.371 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:41.371 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:41.371 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:29:41.371 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.372 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.372 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.372 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.372 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:41.372 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.372 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.372 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.372 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.372 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:41.372 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.372 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.372 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.372 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.372 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:29:41.372 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:29:41.372 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.373 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.373 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:29:41.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:29:41.373 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.373 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.373 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:29:41.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:29:41.373 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.373 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.373 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:29:41.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:29:41.373 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.373 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:29:41.374 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:29:41.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:29:41.374 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:29:41.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:41.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:29:41.374 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:29:41.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:41.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:29:41.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:29:41.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:41.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:29:41.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:29:41.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:41.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:29:41.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:29:41.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:29:41.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:29:41.375 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.376 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.376 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.376 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.376 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.376 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:41.377 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:29:41.377 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:41.374+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:41.478 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:41.475+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:41.554 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:41.555 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:41.555 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:41.555 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:41.564 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:29:41.579 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:41.576+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:41.680 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:41.677+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:41.700 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:41.700 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:41.700 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:41.700 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:41.700 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:41.709 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:29:41.781 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:41.778+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:41.847 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:41.847 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:41.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:41.848 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:41.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.848 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:29:41.848 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:41.848 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 140.03165732299999, 2023-12-06T18:29:41.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:29:41.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:41.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:29:41.848 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:41.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:41.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:41.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:41.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:41.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:41.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:41.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:29:41.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:29:41.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:41.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:29:41.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:29:41.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:41.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:29:41.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:29:41.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:41.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:29:41.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:29:41.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:41.851 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:29:41.851 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:29:41.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:41.851 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:29:41.851 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:29:41.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:41.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:41.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:41.851 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:29:41.851 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:29:41.851 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.851 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.852 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.852 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:41.852 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:41.852 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:41.857 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:29:41.902 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:41.898+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:41.983 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:41.979+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:41.994 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:41.994 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:41.994 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:41.994 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:42.004 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:29:42.089 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:42.085+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:42.182 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:42.182 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:42.182 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:42.182 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:42.182 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:42.185 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:42.181+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:42.191 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:29:42.286 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:42.283+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:42.373 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:42.373 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:42.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:42.373 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:42.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:42.373 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:42.373 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:42.373 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 21.275251281999999, 2023-12-06T18:29:42.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:29:42.373 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:42.373 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:29:42.373 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:42.374 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:29:42.374 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:42.374 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:29:42.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:42.374 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:42.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:42.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:42.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:42.374 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:42.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:42.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:42.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:42.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:42.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:42.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:42.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:42.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:42.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:29:42.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:29:42.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:42.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:42.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:42.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:29:42.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:29:42.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:42.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:42.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:42.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:29:42.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:29:42.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:42.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:42.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:42.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:29:42.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:29:42.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:42.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:42.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:42.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:29:42.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:29:42.377 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:42.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:42.377 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:42.377 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:29:42.377 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:29:42.377 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:42.377 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:42.377 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:42.377 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:42.377 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:42.377 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:42.382 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:29:42.426 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:42.422+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:42.488 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:42.485+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:42.511 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:42.511 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:42.511 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:42.511 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:42.520 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:29:42.589 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:42.586+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:42.691 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:42.687+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:42.691 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:42.691 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:42.691 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:42.691 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:42.691 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:42.700 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:29:42.708 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-12-06T18:29:42.708 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi061 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a" 2023-12-06T18:29:42.708 INFO:tasks.ceph.mon.a:Stopped 2023-12-06T18:29:42.708 INFO:tasks.mon_recovery:forming a minimal quorum for ['b', 'c', 'a'], then adding monitors 2023-12-06T18:29:42.708 INFO:tasks.ceph.mon.b:Restarting daemon 2023-12-06T18:29:42.708 DEBUG:teuthology.orchestra.run.smithi152:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2023-12-06T18:29:42.745 INFO:tasks.ceph.mon.b:Started 2023-12-06T18:29:42.745 INFO:tasks.ceph.mon.c:Restarting daemon 2023-12-06T18:29:42.746 DEBUG:teuthology.orchestra.run.smithi061:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2023-12-06T18:29:42.748 INFO:tasks.ceph.mon.c:Started 2023-12-06T18:29:42.748 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2023-12-06T18:29:42.748 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:29:42.792 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:42.788+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:42.841 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:42.841 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:42.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:42.841 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:42.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:42.841 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:42.841 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:42.841 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 81.347324756999996, 2023-12-06T18:29:42.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:29:42.841 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:42.842 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:29:42.842 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:29:42.842 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:29:42.842 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:42.842 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:29:42.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:42.842 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:42.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:42.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:42.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:42.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:42.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:42.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:42.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:42.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:42.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:42.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:42.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:42.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:42.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:29:42.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:29:42.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:42.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:42.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:42.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:29:42.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:29:42.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:42.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:42.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:42.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:29:42.844 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:29:42.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:42.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:42.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:42.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:29:42.844 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:29:42.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:42.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:42.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:42.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:29:42.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:29:42.845 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:42.845 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:42.845 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:42.845 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:29:42.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:29:42.845 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:42.845 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:42.845 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:42.845 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:42.845 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:42.845 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:42.850 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:29:42.894 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:42.889+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:43.009 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:43.006+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:43.024 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:43.024 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:43.024 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:43.024 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:43.034 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:29:43.094 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:43.091+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:43.195 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:43.192+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:43.204 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:43.204 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:43.204 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:43.204 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:43.204 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:43.214 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:29:43.299 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:43.295+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:43.399 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:43.396+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:43.429 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:43.429 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:43.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:43.430 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:43.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.430 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:29:43.430 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:43.430 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 166.81192696299999, 2023-12-06T18:29:43.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:29:43.430 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.430 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.430 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:43.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:43.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:29:43.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:29:43.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:29:43.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:29:43.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.432 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.432 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:29:43.432 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:29:43.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.432 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.432 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:29:43.432 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:29:43.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.432 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.432 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:29:43.432 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:29:43.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:29:43.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:29:43.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:29:43.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:29:43.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.434 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:29:43.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:29:43.434 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.434 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.434 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.434 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:29:43.434 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:43.434 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 165.78909995399999, 2023-12-06T18:29:43.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:29:43.434 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.434 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.434 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:43.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:43.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:29:43.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:29:43.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.436 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:29:43.436 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:29:43.436 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.436 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.436 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:29:43.436 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:29:43.436 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.436 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.436 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:29:43.436 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:29:43.436 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.436 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:29:43.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:29:43.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:29:43.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:29:43.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:29:43.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:29:43.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:29:43.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:29:43.438 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.438 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.438 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.438 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:29:43.438 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:43.438 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 165.77705331600001, 2023-12-06T18:29:43.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:29:43.438 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.438 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.438 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:43.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:43.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:29:43.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:29:43.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:29:43.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:29:43.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:29:43.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:29:43.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:29:43.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:29:43.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:29:43.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:29:43.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:29:43.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:29:43.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:29:43.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:29:43.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:29:43.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:29:43.442 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.442 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.442 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.442 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:29:43.442 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:43.443 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 165.755652709, 2023-12-06T18:29:43.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:29:43.443 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.443 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.443 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:43.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:43.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:29:43.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:29:43.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:29:43.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:29:43.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:29:43.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:29:43.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:29:43.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:29:43.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:29:43.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:29:43.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:29:43.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:29:43.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:29:43.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:29:43.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:29:43.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:29:43.447 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.447 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.447 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.447 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:29:43.447 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:43.447 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 165.754473386, 2023-12-06T18:29:43.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:29:43.447 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.447 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.447 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.448 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:43.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.448 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:43.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.448 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:29:43.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:29:43.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:29:43.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:29:43.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:29:43.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:29:43.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:29:43.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:29:43.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:29:43.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:29:43.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:29:43.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:29:43.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:29:43.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:29:43.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:29:43.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:29:43.451 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.451 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.451 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.451 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:29:43.451 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:43.451 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 165.75390140499999, 2023-12-06T18:29:43.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:29:43.451 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.451 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.451 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:43.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:43.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:29:43.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:29:43.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:29:43.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:29:43.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:29:43.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:29:43.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:29:43.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:29:43.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:29:43.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:29:43.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:29:43.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:29:43.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:29:43.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:29:43.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:29:43.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:29:43.455 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.455 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.455 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.456 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:29:43.456 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:43.456 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 165.308065935, 2023-12-06T18:29:43.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:29:43.456 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.456 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.456 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:43.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:43.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:29:43.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:29:43.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:29:43.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:29:43.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:29:43.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:29:43.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:29:43.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:29:43.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:29:43.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:29:43.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:29:43.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:29:43.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:29:43.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:29:43.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:29:43.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:29:43.460 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.460 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.460 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.460 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:29:43.460 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:43.460 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 165.30646607899999, 2023-12-06T18:29:43.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:29:43.461 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.461 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.461 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:43.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:43.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:29:43.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:29:43.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:29:43.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:29:43.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:29:43.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:29:43.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:29:43.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:29:43.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:29:43.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:29:43.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:29:43.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:29:43.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:29:43.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:29:43.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:29:43.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:29:43.464 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.464 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.465 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.465 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:29:43.465 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:43.465 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 164.92726046300001, 2023-12-06T18:29:43.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:29:43.465 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.465 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.465 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:43.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:43.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:29:43.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:29:43.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:29:43.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:29:43.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:29:43.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:29:43.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:29:43.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:29:43.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:29:43.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:29:43.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:29:43.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:29:43.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:29:43.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:29:43.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:29:43.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:29:43.470 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.470 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.470 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.470 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:29:43.471 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:43.471 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 164.689758281, 2023-12-06T18:29:43.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:29:43.471 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.471 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.471 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:43.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:43.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:29:43.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:29:43.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:29:43.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:29:43.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:29:43.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:29:43.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:29:43.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:29:43.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:29:43.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:29:43.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:29:43.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:29:43.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:29:43.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:29:43.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:29:43.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:29:43.476 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.476 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.476 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.477 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:29:43.477 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:43.477 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 164.689030282, 2023-12-06T18:29:43.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:29:43.477 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.477 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.477 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:43.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:43.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:29:43.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:29:43.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:29:43.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:29:43.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:29:43.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:29:43.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:29:43.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:29:43.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:29:43.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:29:43.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:29:43.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:29:43.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:29:43.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:29:43.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:29:43.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:29:43.480 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.480 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.480 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.481 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:29:43.481 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:43.481 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 164.684017872, 2023-12-06T18:29:43.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:29:43.481 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.481 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.481 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:43.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:43.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:29:43.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:29:43.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:29:43.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:29:43.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:29:43.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:29:43.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:29:43.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:29:43.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:29:43.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:29:43.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:29:43.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:29:43.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:29:43.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:29:43.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:29:43.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:29:43.484 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.484 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.484 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.485 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:29:43.485 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:43.485 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 164.67849079300001, 2023-12-06T18:29:43.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:29:43.485 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.485 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.485 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:43.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:43.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:29:43.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:29:43.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:29:43.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:29:43.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.487 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:29:43.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:29:43.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.487 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.487 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:29:43.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:29:43.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.487 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.487 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:29:43.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:29:43.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:29:43.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:29:43.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:29:43.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:29:43.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:29:43.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:29:43.488 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.488 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.488 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.489 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:29:43.489 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:43.489 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 164.677979956, 2023-12-06T18:29:43.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:29:43.489 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.489 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.489 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:43.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:43.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:29:43.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:29:43.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:29:43.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:29:43.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.491 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.491 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:29:43.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:29:43.491 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.491 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.491 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:29:43.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:29:43.491 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.491 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.491 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:29:43.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:29:43.491 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.491 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:29:43.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:29:43.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:29:43.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:29:43.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:29:43.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:29:43.492 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.492 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.492 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:29:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 164.677520695, 2023-12-06T18:29:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:29:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:29:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:29:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:29:43.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:29:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:29:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:29:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:29:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:29:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:29:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:29:43.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:29:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:29:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:29:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:29:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:29:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:29:43.496 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:29:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 158.09760488800001, 2023-12-06T18:29:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:29:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:29:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:29:43.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:29:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:29:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:29:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:29:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:29:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:29:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:29:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:29:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:29:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:29:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:29:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:29:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:29:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:29:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:29:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 158.09136708299999, 2023-12-06T18:29:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:29:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:29:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:29:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:29:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:29:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:29:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:29:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:29:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:29:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:29:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:29:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:29:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:29:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:29:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:29:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:29:43.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:29:43.505 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.505 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.505 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.505 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:29:43.505 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:43.505 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 158.08530400399999, 2023-12-06T18:29:43.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:29:43.505 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.505 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.505 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:43.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:43.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:29:43.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:29:43.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:29:43.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:29:43.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:29:43.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:29:43.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:29:43.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:29:43.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:29:43.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:29:43.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:29:43.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:29:43.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:29:43.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:29:43.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:29:43.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:29:43.509 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.509 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.509 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.509 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:29:43.509 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:43.509 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 157.94791691899999, 2023-12-06T18:29:43.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:29:43.509 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.509 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.509 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:43.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:43.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:29:43.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:29:43.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:29:43.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:29:43.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:29:43.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:29:43.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:29:43.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:29:43.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:29:43.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:29:43.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:29:43.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:29:43.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:29:43.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:29:43.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:29:43.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:29:43.512 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.512 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.512 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.513 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:29:43.513 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:43.513 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 157.94124619199999, 2023-12-06T18:29:43.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:29:43.513 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:43.513 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:43.513 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:43.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:43.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:43.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:43.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:43.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:43.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:43.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:29:43.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:29:43.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:43.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:29:43.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:29:43.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:43.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:29:43.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:29:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:29:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:29:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:29:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:29:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:29:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:29:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:43.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:29:43.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:29:43.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:43.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:43.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:43.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:29:43.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:29:43.516 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.516 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.516 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.516 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:43.516 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:43.516 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:43.518 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:29:43.518 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:43.514+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:43.619 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:43.615+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:43.652 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:43.652 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:43.652 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:43.652 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:43.661 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:29:43.720 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:43.716+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:43.792 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:43.792 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:43.792 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:43.792 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:43.792 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:43.801 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:29:43.821 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:43.817+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:43.922 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:43.918+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:43.923 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:29:43.917+0000 5dc2340 -1 osd.7 37 log_to_monitors true 2023-12-06T18:29:43.974 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:43.974 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:43.974 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:43.974 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:29:43.974 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:43.983 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:29:44.027 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:44.023+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:44.123 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:44.123 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:44.123 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:44.123 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:44.124 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:44.120+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:44.133 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:29:44.225 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:44.221+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:44.312 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:44.312 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:44.312 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:44.312 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:44.313 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:44.322 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:29:44.325 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:44.322+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:44.426 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:44.423+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:44.465 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:44.465 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:44.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:44.465 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:29:44.465 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:44.475 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:29:44.528 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:44.524+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:44.619 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:44.619 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:44.619 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:44.619 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:44.628 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:29:44.629 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:44.625+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:44.729 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:44.726+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:44.767 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:44.767 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:44.767 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:44.767 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:44.767 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:44.776 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:29:44.861 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:44.857+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:44.932 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:44.928+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:45.013 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:45.013 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:45.013 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:45.014 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:45.014 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.014 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:29:45.014 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:45.014 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 169.602716536, 2023-12-06T18:29:45.014 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:29:45.014 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.014 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.014 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.014 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.014 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.014 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:29:45.014 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:29:45.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:29:45.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:29:45.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:29:45.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:29:45.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:29:45.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:29:45.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:29:45.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:29:45.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:29:45.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:29:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:29:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:29:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:29:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:29:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:29:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:29:45.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:29:45.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:29:45.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:29:45.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:29:45.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:29:45.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:29:45.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:29:45.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:29:45.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:29:45.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:29:45.020 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.020 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.021 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.021 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.021 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.021 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:45.021 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:45.021 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 167.49414329699999, 2023-12-06T18:29:45.021 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:29:45.021 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.021 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.021 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.021 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.021 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.022 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:29:45.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.022 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:45.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:45.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:29:45.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:29:45.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:29:45.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:29:45.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:29:45.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:29:45.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:29:45.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:29:45.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:29:45.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:29:45.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:29:45.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:29:45.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:29:45.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:29:45.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:29:45.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:29:45.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.026 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:29:45.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:29:45.026 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.026 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.026 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.026 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:29:45.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:29:45.026 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.026 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.026 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.026 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:29:45.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:29:45.026 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.026 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.027 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.027 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:45.027 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:45.027 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 167.49339806500001, 2023-12-06T18:29:45.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:29:45.027 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.027 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.027 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.027 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.027 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.027 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:29:45.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.028 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:45.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:45.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:29:45.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:29:45.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:29:45.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:29:45.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:29:45.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:29:45.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:29:45.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:29:45.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:29:45.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:29:45.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:29:45.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:29:45.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:29:45.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:29:45.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:29:45.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:29:45.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:29:45.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:29:45.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:29:45.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:29:45.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:29:45.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:29:45.032 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.032 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.032 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.032 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:45.032 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:45.032 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 167.49271431, 2023-12-06T18:29:45.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:29:45.033 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.033 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.033 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.033 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.033 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:29:45.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.033 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:45.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.034 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:45.034 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.034 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.034 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:29:45.034 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:29:45.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.034 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.034 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:29:45.034 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:29:45.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.034 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:29:45.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:29:45.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:29:45.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:29:45.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:29:45.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:29:45.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:29:45.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:29:45.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:29:45.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:29:45.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:29:45.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:29:45.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:29:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:29:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:29:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:29:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:29:45.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:29:45.038 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.038 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.038 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.038 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:45.038 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:45.038 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 167.487141055, 2023-12-06T18:29:45.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:29:45.038 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.038 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.038 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.038 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.038 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.038 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:29:45.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.039 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:45.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:45.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:29:45.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:29:45.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:29:45.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:29:45.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:29:45.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:29:45.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.041 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:29:45.041 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:29:45.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.041 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.041 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:29:45.041 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:29:45.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.041 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.041 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:29:45.041 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:29:45.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.041 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.041 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:29:45.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:29:45.042 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.042 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.042 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.042 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:29:45.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:29:45.042 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.042 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.042 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.042 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:29:45.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:29:45.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:29:45.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:29:45.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:29:45.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:29:45.043 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.043 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.043 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.044 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:45.044 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:45.044 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 167.48646741499999, 2023-12-06T18:29:45.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:29:45.044 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.044 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.044 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.044 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.044 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.044 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:29:45.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.044 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:45.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:45.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:29:45.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:29:45.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:29:45.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:29:45.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.046 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.046 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:29:45.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:29:45.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.046 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.046 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:29:45.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:29:45.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.046 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.046 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:29:45.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:29:45.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:29:45.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:29:45.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:29:45.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:29:45.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:29:45.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:29:45.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:29:45.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:29:45.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:29:45.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:29:45.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:29:45.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:29:45.049 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.049 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.049 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.049 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:45.049 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:45.049 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 167.48580521599999, 2023-12-06T18:29:45.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:29:45.049 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.049 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.049 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.049 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.049 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.049 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:29:45.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.050 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:45.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:45.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:29:45.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:29:45.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:29:45.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:29:45.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:29:45.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:29:45.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:29:45.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:29:45.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:29:45.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:29:45.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:29:45.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:29:45.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:29:45.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:29:45.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:29:45.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:29:45.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:29:45.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:29:45.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.054 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:29:45.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:29:45.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.054 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:29:45.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:29:45.054 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.054 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.055 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.055 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:45.055 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:45.055 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 167.41309578100001, 2023-12-06T18:29:45.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:29:45.055 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.055 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.055 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.055 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.055 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.055 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:29:45.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.055 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:45.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:45.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:29:45.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:29:45.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:29:45.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:29:45.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:29:45.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:29:45.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:29:45.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:29:45.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:29:45.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:29:45.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:29:45.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:29:45.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:29:45.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:29:45.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:29:45.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:29:45.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:29:45.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:29:45.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:29:45.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:29:45.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.060 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:29:45.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:29:45.060 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.060 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.060 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.060 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:45.060 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:45.060 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 167.41241407199999, 2023-12-06T18:29:45.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:29:45.060 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.061 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.061 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.061 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.061 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.061 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:29:45.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.061 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.061 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.061 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:45.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.061 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.061 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:45.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:29:45.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:29:45.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:29:45.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:29:45.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:29:45.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:29:45.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:29:45.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:29:45.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:29:45.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:29:45.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:29:45.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:29:45.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:29:45.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:29:45.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:29:45.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:29:45.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:29:45.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:29:45.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:29:45.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:29:45.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:29:45.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:29:45.065 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.065 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.065 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.066 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.066 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:45.066 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:45.066 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 166.33360998399999, 2023-12-06T18:29:45.066 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:29:45.066 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.066 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.066 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.066 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.066 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.066 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:29:45.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.066 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.067 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.067 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.067 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:45.067 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.067 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.067 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.067 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.067 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:45.067 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.067 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.067 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.067 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.067 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:29:45.067 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:29:45.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:29:45.068 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:29:45.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:29:45.068 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:29:45.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:29:45.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:29:45.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:29:45.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:29:45.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:29:45.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:29:45.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:29:45.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:29:45.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.070 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.070 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:29:45.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:29:45.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.070 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.070 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:29:45.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:29:45.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.070 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.071 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:29:45.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:29:45.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.071 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.071 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:29:45.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:29:45.071 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.071 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.071 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.071 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:45.071 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:45.071 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 166.33219341, 2023-12-06T18:29:45.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:29:45.072 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.072 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.072 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.072 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.072 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.072 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:29:45.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.072 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.072 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.072 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.072 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:45.072 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.072 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.072 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:45.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:29:45.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:29:45.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:29:45.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:29:45.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:29:45.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:29:45.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.074 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:29:45.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:29:45.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.074 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:29:45.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:29:45.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:29:45.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:29:45.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.075 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:29:45.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:29:45.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.075 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:29:45.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:29:45.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.075 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:29:45.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:29:45.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.076 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:29:45.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:29:45.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.076 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:29:45.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:29:45.076 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.076 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.076 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.077 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.077 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:45.077 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:45.077 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 166.331108285, 2023-12-06T18:29:45.077 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:29:45.077 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.077 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.077 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.077 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.077 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.077 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:29:45.077 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.077 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.077 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.077 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.078 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:45.078 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.078 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.078 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:45.078 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.078 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.078 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:29:45.078 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:29:45.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:29:45.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:29:45.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:29:45.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:29:45.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:29:45.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:29:45.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:29:45.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:29:45.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:29:45.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:29:45.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:29:45.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:29:45.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.081 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:29:45.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:29:45.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.081 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:29:45.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:29:45.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.081 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:29:45.082 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:29:45.082 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.082 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.082 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.082 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:29:45.082 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:29:45.082 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.082 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.082 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.082 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.082 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.082 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:45.082 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:45.083 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 166.29595918300001, 2023-12-06T18:29:45.083 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:29:45.083 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.083 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.083 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.083 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.083 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.083 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:29:45.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.083 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.083 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.083 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.083 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:45.083 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:45.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:29:45.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:29:45.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:29:45.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:29:45.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:29:45.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:29:45.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:29:45.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:29:45.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:29:45.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:29:45.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:29:45.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:29:45.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:29:45.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:29:45.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.087 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:29:45.087 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:29:45.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.087 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:29:45.087 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:29:45.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.087 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:29:45.087 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:29:45.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.088 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.088 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:29:45.088 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:29:45.088 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.088 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.088 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.088 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.088 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.088 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:45.088 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:45.088 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 159.74622865800001, 2023-12-06T18:29:45.088 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:29:45.088 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.088 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.088 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.088 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.089 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.089 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:29:45.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.089 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:45.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:45.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:29:45.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:29:45.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:29:45.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:29:45.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:29:45.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:29:45.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:29:45.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:29:45.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:29:45.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:29:45.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:29:45.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:29:45.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:29:45.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:29:45.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:29:45.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:29:45.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:29:45.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:29:45.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.093 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.093 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:29:45.093 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:29:45.093 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.093 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.093 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.093 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:29:45.093 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:29:45.093 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.093 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.093 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.093 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.093 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.093 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:45.094 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:45.094 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 159.74542305, 2023-12-06T18:29:45.094 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:29:45.094 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.094 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.094 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.094 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.094 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.094 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:29:45.094 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.094 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.094 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.094 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.094 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:45.095 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.095 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.095 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.095 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:45.095 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.095 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.095 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.095 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:29:45.095 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:29:45.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.095 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.095 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.095 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:29:45.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:29:45.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.096 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.096 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:29:45.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:29:45.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.096 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.096 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:29:45.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:29:45.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.096 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.096 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:29:45.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:29:45.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.097 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.097 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:29:45.097 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:29:45.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.097 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.097 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:29:45.097 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:29:45.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.097 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.097 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:29:45.097 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:29:45.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.098 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.098 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.098 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:29:45.098 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:29:45.098 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.098 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.098 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.098 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:29:45.098 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:29:45.098 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.098 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.098 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.098 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:29:45.098 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:29:45.099 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.099 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.099 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.099 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.099 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:45.099 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:45.099 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 159.74473224100001, 2023-12-06T18:29:45.099 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:29:45.099 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.099 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.099 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.099 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.100 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.100 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:29:45.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.100 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.100 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:45.100 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.100 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:45.100 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:29:45.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:29:45.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:29:45.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:29:45.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:29:45.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:29:45.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:29:45.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:29:45.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:29:45.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:29:45.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:29:45.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:29:45.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.103 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.103 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.103 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:29:45.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:29:45.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.103 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.103 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.103 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:29:45.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:29:45.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.103 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.103 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.103 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:29:45.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:29:45.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.104 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.104 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:29:45.104 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:29:45.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.104 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.104 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:29:45.104 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:29:45.104 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.104 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.104 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.104 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:45.105 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:45.105 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 159.740434733, 2023-12-06T18:29:45.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:29:45.105 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.105 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.105 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.105 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.105 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.105 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:29:45.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.105 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.105 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.105 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:45.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.106 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.106 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:45.106 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.106 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.106 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:29:45.106 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:29:45.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.106 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.106 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:29:45.106 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:29:45.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.107 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:29:45.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:29:45.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.107 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:29:45.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:29:45.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.107 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:29:45.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:29:45.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.108 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.108 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:29:45.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:29:45.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.108 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.108 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:29:45.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:29:45.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.108 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.108 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:29:45.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:29:45.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.109 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.109 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.109 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:29:45.109 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:29:45.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.109 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.109 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.109 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:29:45.109 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:29:45.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.109 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.109 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.109 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:29:45.109 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:29:45.109 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.110 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.110 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.110 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.110 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.110 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:45.110 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:45.110 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 159.73976705800001, 2023-12-06T18:29:45.110 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:29:45.110 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.110 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.110 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.110 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.110 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.110 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:29:45.110 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.111 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:45.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:45.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:29:45.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:29:45.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:29:45.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:29:45.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:29:45.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:29:45.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:29:45.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:29:45.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:29:45.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:29:45.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:29:45.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:29:45.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:29:45.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:29:45.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.114 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:29:45.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:29:45.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.114 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:29:45.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:29:45.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:29:45.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:29:45.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:29:45.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:29:45.115 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.115 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.115 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.115 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:45.115 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:45.115 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 159.73911163, 2023-12-06T18:29:45.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:29:45.116 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.116 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.116 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.116 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.116 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.116 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:29:45.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.116 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:45.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:45.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:29:45.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:29:45.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:29:45.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:29:45.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:29:45.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:29:45.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:29:45.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:29:45.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:29:45.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:29:45.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:29:45.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:29:45.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:29:45.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:29:45.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:29:45.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:29:45.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:29:45.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:29:45.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:29:45.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:29:45.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:29:45.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:29:45.120 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.120 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.120 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.121 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:45.121 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:45.121 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 159.69845708299999, 2023-12-06T18:29:45.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:29:45.121 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.121 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:45.121 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:45.121 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:45.121 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:45.121 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:29:45.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.121 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:45.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:45.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:29:45.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:29:45.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:29:45.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:29:45.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:29:45.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:29:45.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:29:45.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:29:45.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.124 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.124 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.124 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:29:45.124 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:29:45.124 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.124 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.124 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.124 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:29:45.124 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:29:45.124 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.124 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.124 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:45.124 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:29:45.124 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:29:45.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.125 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:45.125 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:29:45.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:29:45.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.125 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:45.125 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:29:45.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:29:45.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.125 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:45.125 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:29:45.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:29:45.126 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.126 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.126 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.126 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:29:45.126 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:29:45.126 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.126 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.126 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.126 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.126 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.126 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:45.128 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:29:45.131 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:45.127+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:45.229 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:45.226+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:45.267 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:45.267 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:45.267 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:45.267 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:45.278 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:29:45.330 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:45.326+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:45.415 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:45.415 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:45.415 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:45.415 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:45.415 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:45.424 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:29:45.430 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:45.427+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:45.467 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:29:45.531 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:45.527+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:45.579 INFO:teuthology.orchestra.run.smithi152.stderr:no valid command found; 10 closest matches: 2023-12-06T18:29:45.580 INFO:teuthology.orchestra.run.smithi152.stderr:0 2023-12-06T18:29:45.580 INFO:teuthology.orchestra.run.smithi152.stderr:1 2023-12-06T18:29:45.580 INFO:teuthology.orchestra.run.smithi152.stderr:2 2023-12-06T18:29:45.580 INFO:teuthology.orchestra.run.smithi152.stderr:abort 2023-12-06T18:29:45.580 INFO:teuthology.orchestra.run.smithi152.stderr:assert 2023-12-06T18:29:45.580 INFO:teuthology.orchestra.run.smithi152.stderr:bluefs debug_inject_read_zeros 2023-12-06T18:29:45.580 INFO:teuthology.orchestra.run.smithi152.stderr:bluefs files list 2023-12-06T18:29:45.580 INFO:teuthology.orchestra.run.smithi152.stderr:bluefs stats 2023-12-06T18:29:45.580 INFO:teuthology.orchestra.run.smithi152.stderr:bluestore allocator dump block 2023-12-06T18:29:45.580 INFO:teuthology.orchestra.run.smithi152.stderr:bluestore allocator fragmentation block 2023-12-06T18:29:45.581 INFO:teuthology.orchestra.run.smithi152.stderr:admin_socket: invalid command 2023-12-06T18:29:45.582 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-12-06T18:29:45.582 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-7, ['dump_ops_in_flight'] 2023-12-06T18:29:45.605 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:45.606 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:45.606 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:45.606 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:45.606 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.606 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:29:45.606 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:45.606 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 143.78955411800001, 2023-12-06T18:29:45.607 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:29:45.607 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:45.607 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:29:45.607 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:45.607 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.607 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:45.607 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:45.607 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.607 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.607 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.607 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:45.608 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:45.608 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:45.608 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.608 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.608 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:45.608 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:29:45.608 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:29:45.608 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.608 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.608 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:45.608 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:29:45.608 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:29:45.608 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.609 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.609 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:45.609 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:29:45.609 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:29:45.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.609 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.609 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:45.609 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:29:45.609 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:29:45.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.609 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.609 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:45.609 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:29:45.610 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:29:45.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.610 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.610 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:45.610 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:29:45.610 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:29:45.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:45.610 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:45.610 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:45.610 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:29:45.611 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:29:45.611 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.611 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.611 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.611 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:45.611 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:45.611 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:45.615 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:29:45.632 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:45.629+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:45.733 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:45.730+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:45.753 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:45.753 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:45.753 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:45.753 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:45.762 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:29:45.834 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:45.831+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:45.904 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:45.904 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:45.904 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:45.905 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:45.905 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:45.914 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:29:45.958 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:45.954+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:46.036 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:46.032+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:46.053 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:46.053 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:46.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:46.053 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:46.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:46.054 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:46.054 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:46.054 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 24.955487805000001, 2023-12-06T18:29:46.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:29:46.054 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:46.054 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:29:46.054 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:46.054 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:29:46.054 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:46.054 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:29:46.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:46.054 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:46.054 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:46.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:46.055 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:46.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:46.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:46.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:46.055 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:46.055 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:46.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:46.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:46.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:46.055 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:46.055 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:29:46.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:29:46.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:46.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:46.055 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:46.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:29:46.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:29:46.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:46.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:46.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:46.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:29:46.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:29:46.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:46.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:46.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:46.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:29:46.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:29:46.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:46.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:46.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:46.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:29:46.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:29:46.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:46.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:46.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:46.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:29:46.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:29:46.057 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:46.057 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:46.057 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:46.057 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:46.057 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:46.057 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:46.063 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:29:46.137 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:46.133+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:46.192 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:46.193 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:46.193 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:46.193 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:46.203 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:29:46.237 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:46.234+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:46.338 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:46.335+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:46.373 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:46.374 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:46.374 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:46.374 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:46.374 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:46.384 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:29:46.439 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:46.436+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:46.518 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:46.518 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:46.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:46.518 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:46.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:46.518 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:46.518 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:46.518 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 85.024193921000006, 2023-12-06T18:29:46.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:29:46.519 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:46.519 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:29:46.519 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:29:46.519 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:29:46.519 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:46.519 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:29:46.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:46.519 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:46.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:46.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:46.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:46.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:46.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:46.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:46.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:46.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:46.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:46.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:46.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:46.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:46.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:29:46.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:29:46.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:46.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:46.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:46.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:29:46.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:29:46.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:46.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:46.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:46.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:29:46.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:29:46.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:46.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:46.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:46.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:29:46.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:29:46.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:46.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:46.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:46.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:29:46.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:29:46.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:46.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:46.522 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:46.522 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:29:46.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:29:46.522 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:46.522 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:46.522 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:46.522 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:46.522 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:46.522 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:46.526 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:29:46.573 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:46.567+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:46.641 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:46.638+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:46.656 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:46.656 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:46.656 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:46.656 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:46.665 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:29:46.742 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:46.738+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:46.836 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:46.837 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:46.843 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:46.839+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:46.845 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:29:46.944 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:46.941+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:47.016 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:47.016 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:47.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:47.016 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:29:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 170.399150392, 2023-12-06T18:29:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:29:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:29:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:29:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:29:47.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:29:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:29:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:29:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:29:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:29:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.019 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:29:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:29:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:29:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:29:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:29:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:29:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:29:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:29:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:29:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 169.376323383, 2023-12-06T18:29:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:29:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.021 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:29:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:29:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:29:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:29:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:29:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:29:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:29:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:29:47.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:29:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:29:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:29:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:29:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:29:47.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:29:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:29:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:29:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:29:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 169.36427674500001, 2023-12-06T18:29:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:29:47.025 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:47.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:29:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:29:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:29:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:29:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:29:47.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:29:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:29:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:29:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:29:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:29:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:29:47.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:29:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:29:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:29:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:29:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:29:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:29:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 169.34287613800001, 2023-12-06T18:29:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:29:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:29:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:29:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:29:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:29:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:29:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:29:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:29:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:29:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:29:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:29:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:29:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:29:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:29:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:29:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:29:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:29:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.033 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:29:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 169.34169681500001, 2023-12-06T18:29:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:29:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:29:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:29:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:29:47.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:29:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:29:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:29:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:29:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:29:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:29:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:29:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:29:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:29:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:29:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:29:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:29:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:29:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:29:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 169.341124834, 2023-12-06T18:29:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:29:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:29:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:29:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:29:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:29:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:29:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:29:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:29:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:29:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:29:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:29:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:29:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:29:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:29:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:29:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:29:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:29:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:29:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 168.89528936400001, 2023-12-06T18:29:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:29:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:29:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:29:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:29:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:29:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:29:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:29:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:29:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:29:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:29:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:29:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:29:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:29:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:29:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:29:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:29:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:29:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:29:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 168.89368950799999, 2023-12-06T18:29:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:29:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.046 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:29:47.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:29:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:29:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:29:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:29:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:29:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:29:47.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:29:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:29:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:29:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:29:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:29:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:29:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:29:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:29:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:29:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:29:47.050 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 168.51448389199999, 2023-12-06T18:29:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:29:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:29:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:29:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:29:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:29:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:29:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:29:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:29:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:29:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:29:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:29:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:29:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:29:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:29:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:29:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:29:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:29:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:29:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 168.27698171, 2023-12-06T18:29:47.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:29:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:29:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:29:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:29:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:29:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:29:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:29:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:29:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:29:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:29:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:29:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:29:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:29:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:29:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:29:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:29:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:29:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:29:47.058 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 168.27625371100001, 2023-12-06T18:29:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:29:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:29:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:29:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:29:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:29:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:29:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:29:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:29:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:29:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:29:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:29:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:29:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:29:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:29:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:29:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:29:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:29:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:29:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:47.062 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 168.271241301, 2023-12-06T18:29:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:29:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:29:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:29:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:29:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:29:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:29:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:29:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:29:47.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:29:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:29:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:29:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:29:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:29:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:29:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:29:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:29:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:29:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:29:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 168.26571422200001, 2023-12-06T18:29:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:29:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.066 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:29:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:29:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:29:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:29:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:29:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:29:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:29:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:29:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:29:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:29:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:29:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:29:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:29:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:29:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:29:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:29:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:29:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 168.26520338500001, 2023-12-06T18:29:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:29:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.070 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:29:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:29:47.071 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:29:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:29:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:29:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:29:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:29:47.072 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:29:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:29:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:29:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:29:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:29:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:29:47.073 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:29:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:29:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:29:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:29:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 168.264744124, 2023-12-06T18:29:47.074 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:29:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.075 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:29:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:29:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:29:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:29:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:29:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:29:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:29:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:29:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:29:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:29:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:29:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:29:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:29:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:29:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:29:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:29:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:29:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 161.68482831700001, 2023-12-06T18:29:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:29:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.079 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:29:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:29:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:29:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:29:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.080 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:29:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:29:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:29:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:29:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:29:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:29:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.081 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:29:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:29:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:29:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:29:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:29:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:29:47.082 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:29:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 161.678590512, 2023-12-06T18:29:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:29:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.083 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:29:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:29:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.084 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.085 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.085 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:29:47.085 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:29:47.085 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.085 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.085 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:29:47.085 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:29:47.085 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.085 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.085 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:29:47.085 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:29:47.085 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.085 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.085 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:29:47.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:29:47.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:29:47.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:29:47.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.086 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.086 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:29:47.086 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:29:47.086 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.086 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.087 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.087 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:29:47.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:29:47.087 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.087 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.087 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.087 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.087 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.087 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:29:47.087 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:47.087 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 161.672527433, 2023-12-06T18:29:47.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:29:47.087 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.087 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.087 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.088 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.088 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:47.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.088 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.088 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:47.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.088 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.088 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.088 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:29:47.088 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:29:47.088 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.089 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.089 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.089 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:29:47.089 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:29:47.089 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.089 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.089 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.089 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:29:47.089 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:29:47.089 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.089 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.089 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.089 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:29:47.089 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:29:47.089 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.090 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.090 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.090 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:29:47.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:29:47.090 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.090 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.090 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.090 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:29:47.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:29:47.090 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.090 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.090 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.090 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:29:47.090 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:29:47.090 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.091 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.091 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:29:47.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:29:47.091 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.091 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.091 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.091 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.091 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.091 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:29:47.091 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:47.091 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 161.535140348, 2023-12-06T18:29:47.091 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:29:47.091 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.092 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.092 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.092 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:47.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.092 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:47.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.092 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.092 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.092 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.092 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:29:47.092 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:29:47.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.093 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.093 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:29:47.093 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:29:47.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.093 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.093 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:29:47.093 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:29:47.093 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.093 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.093 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:29:47.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:29:47.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:29:47.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:29:47.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:29:47.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:29:47.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:29:47.095 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:29:47.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.095 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:29:47.095 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:29:47.095 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.095 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.095 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.095 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:29:47.095 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:47.095 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 161.528469621, 2023-12-06T18:29:47.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:29:47.096 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:47.096 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:47.096 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:47.096 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.096 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:47.096 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:47.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.096 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.096 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:47.096 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:47.097 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:47.097 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.097 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:47.097 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:29:47.097 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:29:47.097 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.097 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:47.097 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:29:47.097 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:29:47.097 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.097 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:47.097 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:29:47.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:29:47.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.098 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:47.098 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:29:47.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:29:47.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.098 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:47.098 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:29:47.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:29:47.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.098 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:47.099 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:29:47.099 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:29:47.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.099 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.099 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:47.099 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:29:47.099 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:29:47.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:47.099 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:47.099 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:47.099 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:29:47.099 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:29:47.099 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.099 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.099 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.100 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:47.100 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:47.100 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:47.100 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-12-06T18:29:47.101 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:29:47.102 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:47.098+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:47.203 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:47.199+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:47.274 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:47.274 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:47.274 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:47.275 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:47.283 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:29:47.328 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:47.324+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:47.406 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:47.402+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:47.415 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:47.416 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:47.416 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:47.416 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:47.416 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:47.425 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:29:47.506 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:47.503+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:47.597 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:47.597 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:47.597 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:47.597 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:29:47.597 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:47.603 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:29:47.607 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:47.603+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:47.708 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:47.704+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:47.741 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:47.741 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:47.741 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:47.741 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:47.751 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:29:47.837 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:47.833+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:47.910 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:47.906+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:47.928 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:47.928 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:47.928 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:47.928 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:47.928 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:47.937 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:29:48.022 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:48.018+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:48.112 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:48.109+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:48.118 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:48.118 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:48.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:48.119 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:29:48.119 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:48.129 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:29:48.214 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:48.210+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:48.307 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:48.307 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:48.307 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:48.307 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:48.313 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:48.309+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:48.316 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:29:48.414 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:48.411+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:48.492 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:48.492 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:48.492 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:48.493 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:48.493 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:48.502 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:29:48.516 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:48.512+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:48.617 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:48.613+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:48.718 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:48.714+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:48.739 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:48.739 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:48.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:48.739 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:48.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.739 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:29:48.739 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:48.739 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 173.326549403, 2023-12-06T18:29:48.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:29:48.739 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.740 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.740 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.740 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.740 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.740 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:29:48.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.740 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:48.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:48.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:29:48.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:29:48.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:29:48.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:29:48.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:29:48.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:29:48.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:29:48.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:29:48.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:29:48.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:29:48.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:29:48.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:29:48.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:29:48.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:29:48.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:29:48.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:29:48.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:29:48.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:29:48.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:29:48.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:29:48.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:29:48.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:29:48.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:29:48.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:29:48.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:29:48.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:29:48.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:29:48.745 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.745 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.745 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.745 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:48.745 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:48.745 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 171.21797616399999, 2023-12-06T18:29:48.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:29:48.745 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.746 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.746 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.746 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.746 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.746 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:29:48.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.746 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:48.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:48.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:29:48.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:29:48.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:29:48.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:29:48.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:29:48.748 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:29:48.748 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.748 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.748 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.748 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:29:48.748 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:29:48.748 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.748 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.748 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.748 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:29:48.748 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:29:48.748 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.748 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.748 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.748 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:29:48.749 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:29:48.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.749 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.749 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.749 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:29:48.749 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:29:48.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.749 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.749 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.749 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:29:48.749 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:29:48.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.749 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.749 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.749 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:29:48.749 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:29:48.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.750 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.750 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.750 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:29:48.750 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:29:48.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.750 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.750 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.750 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:29:48.750 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:29:48.750 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.750 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.750 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.750 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.751 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:48.751 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:48.751 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 171.21723093200001, 2023-12-06T18:29:48.751 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:29:48.751 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.751 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.751 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.751 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.751 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.751 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:29:48.751 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.751 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.751 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.751 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:48.752 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:48.752 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:29:48.752 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:29:48.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:29:48.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:29:48.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.753 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.753 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.753 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:29:48.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:29:48.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.753 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.753 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.753 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:29:48.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:29:48.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.753 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.753 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.753 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:29:48.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:29:48.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.754 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.754 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.754 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:29:48.754 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:29:48.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.754 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.754 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.754 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:29:48.754 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:29:48.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.754 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.754 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.754 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:29:48.754 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:29:48.755 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.755 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.755 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.755 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:29:48.755 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:29:48.755 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.755 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.755 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.755 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:29:48.755 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:29:48.755 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.755 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.755 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.755 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:29:48.755 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:29:48.755 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.756 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.756 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.756 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.756 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:48.756 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:48.756 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 171.216547177, 2023-12-06T18:29:48.756 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:29:48.756 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.756 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.756 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.756 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.756 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.756 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:29:48.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.757 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:48.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:48.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:29:48.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:29:48.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.758 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.758 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.758 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:29:48.758 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:29:48.758 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.758 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.758 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.758 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:29:48.758 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:29:48.758 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.758 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.758 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.758 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:29:48.758 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:29:48.758 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:29:48.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:29:48.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:29:48.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:29:48.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:29:48.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:29:48.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.760 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.760 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:29:48.760 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:29:48.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.760 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.760 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.760 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:29:48.760 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:29:48.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.760 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.760 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.760 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:29:48.760 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:29:48.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.760 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.761 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.761 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:29:48.761 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:29:48.761 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.761 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.761 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.761 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.761 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:48.761 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:48.761 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 171.21097392199999, 2023-12-06T18:29:48.761 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:29:48.761 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.761 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.761 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.761 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.762 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.762 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:29:48.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.762 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.762 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.762 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:48.762 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.762 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.762 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:48.762 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.763 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:29:48.763 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:29:48.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.763 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:29:48.763 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:29:48.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.763 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:29:48.763 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:29:48.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:29:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:29:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:29:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:29:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:29:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:29:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:29:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:29:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:29:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:29:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:29:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:29:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:29:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:29:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:29:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:29:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 171.21030028199999, 2023-12-06T18:29:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:29:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:29:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:29:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:29:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:29:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:29:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:29:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:29:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:29:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:29:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:29:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:29:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:29:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:29:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:29:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:29:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:29:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:29:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:29:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:29:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:29:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:29:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:29:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:29:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 171.20963808299999, 2023-12-06T18:29:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:29:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:29:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:48.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:48.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:29:48.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:29:48.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:29:48.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:29:48.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:29:48.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:29:48.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:29:48.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:29:48.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:29:48.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:29:48.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:29:48.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:29:48.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:29:48.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:29:48.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:29:48.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:29:48.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.776 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:29:48.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:29:48.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.776 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:29:48.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:29:48.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:29:48.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:29:48.777 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.777 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.777 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.777 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:48.777 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:48.777 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 171.13692864800001, 2023-12-06T18:29:48.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:29:48.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.777 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.777 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.777 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.778 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.778 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:29:48.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.778 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:48.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:48.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:29:48.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:29:48.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:29:48.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:29:48.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:29:48.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:29:48.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:29:48.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:29:48.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:29:48.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:29:48.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:29:48.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:29:48.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:29:48.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:29:48.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:29:48.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:29:48.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:29:48.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:29:48.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.782 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.782 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:29:48.782 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:29:48.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.782 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.782 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:29:48.782 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:29:48.782 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.782 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.782 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.782 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:48.782 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 171.13624693899999, 2023-12-06T18:29:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:29:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:29:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:29:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:29:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:29:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:29:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.785 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:29:48.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:29:48.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.785 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:29:48.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:29:48.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.785 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:29:48.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:29:48.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:29:48.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:29:48.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:29:48.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:29:48.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:29:48.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:29:48.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:29:48.787 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:29:48.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.787 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.787 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:29:48.787 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:29:48.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.787 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.787 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:29:48.787 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:29:48.787 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.787 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.787 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.788 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:48.788 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:48.788 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 170.05744285099999, 2023-12-06T18:29:48.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:29:48.788 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.788 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.788 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.788 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.788 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.788 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:29:48.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.788 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.788 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:48.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:48.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:29:48.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:29:48.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:29:48.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:29:48.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.790 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.790 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:29:48.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:29:48.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.790 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.790 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:29:48.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:29:48.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.790 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.790 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:29:48.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:29:48.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:29:48.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:29:48.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:29:48.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:29:48.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:29:48.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:29:48.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.792 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.792 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:29:48.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:29:48.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.792 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.792 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:29:48.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:29:48.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.792 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.792 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:29:48.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:29:48.792 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.792 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.793 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.793 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:48.793 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:48.793 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 170.056026277, 2023-12-06T18:29:48.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:29:48.793 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.793 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.793 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.793 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.793 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.793 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:29:48.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.793 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:48.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:48.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:29:48.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:29:48.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:29:48.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:29:48.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:29:48.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:29:48.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:29:48.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:29:48.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:29:48.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:29:48.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:29:48.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:29:48.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:29:48.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:29:48.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:29:48.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:29:48.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:29:48.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:29:48.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:29:48.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:29:48.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:29:48.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:29:48.798 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.798 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.798 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.798 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:48.798 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:48.798 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 170.054941152, 2023-12-06T18:29:48.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:29:48.798 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.798 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.798 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.798 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.798 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.799 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:29:48.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.799 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:48.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:48.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:29:48.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:29:48.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:29:48.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:29:48.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:29:48.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:29:48.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:29:48.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:29:48.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:29:48.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:29:48.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:29:48.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:29:48.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.802 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:29:48.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:29:48.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.802 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:29:48.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:29:48.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:29:48.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:29:48.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:29:48.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:29:48.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:29:48.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:29:48.803 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.803 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.803 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.804 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:48.804 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:48.804 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 170.01979205000001, 2023-12-06T18:29:48.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:29:48.804 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.804 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.804 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.804 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.804 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.804 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:29:48.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.804 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:48.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:48.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:29:48.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:29:48.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:29:48.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:29:48.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:29:48.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:29:48.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:29:48.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:29:48.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:29:48.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:29:48.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:29:48.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:29:48.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:29:48.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:29:48.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.808 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:29:48.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:29:48.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.808 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.808 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:29:48.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:29:48.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.808 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.808 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:29:48.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:29:48.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.808 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:29:48.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:29:48.809 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.809 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.809 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.809 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:48.809 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:48.809 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 163.47006152500001, 2023-12-06T18:29:48.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:29:48.809 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.809 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.809 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.809 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.810 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.810 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:29:48.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.810 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:48.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:48.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:29:48.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:29:48.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:29:48.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:29:48.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:29:48.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:29:48.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:29:48.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:29:48.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:29:48.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:29:48.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:29:48.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:29:48.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:29:48.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:29:48.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:29:48.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:29:48.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:29:48.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:29:48.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.814 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:29:48.814 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:29:48.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.814 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.814 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:29:48.814 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:29:48.814 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.814 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.814 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.814 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:48.814 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:48.815 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 163.469255917, 2023-12-06T18:29:48.815 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:29:48.815 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.815 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.815 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.815 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.815 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.815 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:29:48.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.815 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.815 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.815 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.815 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:48.815 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.815 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:48.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:29:48.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:29:48.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:29:48.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:29:48.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:29:48.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:29:48.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:29:48.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:29:48.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:29:48.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:29:48.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:29:48.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:29:48.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:29:48.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:29:48.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:29:48.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:29:48.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.819 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:29:48.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:29:48.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.819 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.819 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.819 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:29:48.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:29:48.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.819 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.819 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.819 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:29:48.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:29:48.819 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.819 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.819 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.820 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.820 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:48.820 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:48.820 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 163.46856510800001, 2023-12-06T18:29:48.820 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:29:48.820 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.820 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.820 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.820 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.820 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.820 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:29:48.820 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.820 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.820 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.820 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.821 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:48.821 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.821 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.821 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.821 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.821 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:48.821 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.821 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.821 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.821 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.821 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:29:48.821 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:29:48.821 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.821 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.821 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.821 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:29:48.822 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:29:48.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.822 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.822 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.822 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:29:48.822 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:29:48.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.822 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.822 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.822 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:29:48.822 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:29:48.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.822 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.822 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.822 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:29:48.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:29:48.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:29:48.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:29:48.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:29:48.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:29:48.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:29:48.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:29:48.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.824 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:29:48.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:29:48.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.824 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:29:48.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:29:48.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.824 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:29:48.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:29:48.824 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.825 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.825 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.825 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.825 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:48.825 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:48.825 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 163.4642676, 2023-12-06T18:29:48.825 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:29:48.825 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.825 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.825 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.825 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.825 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.825 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:29:48.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.825 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.826 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.826 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.826 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:48.826 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.826 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.826 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.826 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.826 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:48.826 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.826 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.826 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.826 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.826 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:29:48.826 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:29:48.826 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.827 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.827 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.827 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:29:48.827 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:29:48.827 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.827 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.827 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.827 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:29:48.827 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:29:48.827 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.827 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.827 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.827 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:29:48.827 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:29:48.827 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.827 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.828 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.828 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:29:48.828 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:29:48.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.828 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.828 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:29:48.828 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:29:48.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.828 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.828 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:29:48.828 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:29:48.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.828 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:29:48.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:29:48.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.829 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.829 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:29:48.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:29:48.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.829 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.829 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:29:48.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:29:48.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.829 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.829 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:29:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:29:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 163.46359992500001, 2023-12-06T18:29:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:29:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.830 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:29:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.831 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:29:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:29:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:29:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:29:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:29:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:29:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:29:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:29:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:29:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:29:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:29:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:29:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:29:48.833 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:29:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:29:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:29:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:29:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:29:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:29:48.834 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:29:48.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.835 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.835 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.835 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:29:48.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:29:48.835 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.835 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.835 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.835 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.835 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:48.835 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:48.835 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 163.462944497, 2023-12-06T18:29:48.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:29:48.835 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.835 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.836 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.836 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.836 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.836 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:29:48.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.836 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.836 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.836 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.836 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:48.836 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.836 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.836 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.836 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:48.836 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.837 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.837 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:29:48.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:29:48.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.837 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.837 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:29:48.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:29:48.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.837 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.837 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:29:48.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:29:48.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:29:48.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:29:48.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:29:48.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:29:48.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:29:48.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:29:48.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.839 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:29:48.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:29:48.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.839 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:29:48.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:29:48.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.839 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:29:48.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:29:48.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.840 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:29:48.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:29:48.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.840 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:29:48.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:29:48.840 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.840 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.840 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.840 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:48.840 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:48.841 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 163.42228994999999, 2023-12-06T18:29:48.841 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:29:48.841 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:48.841 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:48.841 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:48.841 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:48.841 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:48.841 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:29:48.841 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.841 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:48.841 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.841 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:48.841 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:48.841 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.841 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.841 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.842 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:48.842 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:48.842 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:48.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.842 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.842 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:48.842 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:29:48.842 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:29:48.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.842 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.842 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:48.842 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:29:48.842 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:29:48.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.842 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:48.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:29:48.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:29:48.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:48.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:29:48.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:29:48.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:48.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:29:48.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:29:48.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:48.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:29:48.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:29:48.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:48.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:29:48.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:29:48.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:48.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:29:48.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:29:48.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:48.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:29:48.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:29:48.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.845 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:48.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:29:48.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:29:48.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:48.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:48.845 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:48.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:29:48.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:29:48.845 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.845 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.845 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.845 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:48.846 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:48.846 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:48.847 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:29:48.848 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:48.844+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:48.948 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:48.944+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:49.025 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:49.025 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:49.025 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:49.025 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:49.035 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:29:49.049 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:49.045+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:49.150 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:49.147+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:49.212 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:49.212 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:49.213 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:49.213 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:49.213 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:49.222 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:29:49.267 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:49.263+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:49.352 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:49.348+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:49.364 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:49.365 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:49.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:49.365 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:49.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.365 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:29:49.365 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:49.366 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 147.548604442, 2023-12-06T18:29:49.366 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:29:49.366 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:49.366 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:29:49.366 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:49.366 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.367 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:49.367 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:49.367 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:49.367 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:49.367 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.367 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:49.367 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:49.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:49.368 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:49.368 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:49.368 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:29:49.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:29:49.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:49.369 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.369 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:49.369 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:29:49.369 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:29:49.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:49.369 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.369 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:49.369 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:29:49.370 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:29:49.370 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:49.370 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.370 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:49.370 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:29:49.370 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:29:49.370 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:49.370 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.371 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:49.371 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:29:49.371 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:29:49.371 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:49.371 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.371 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:49.371 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:29:49.371 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:29:49.371 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:49.372 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.372 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:49.372 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:29:49.372 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:29:49.372 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:49.372 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:49.372 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:49.373 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:49.373 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:49.373 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:49.374 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:29:49.459 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:49.455+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:49.554 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:49.551+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:49.571 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:49.571 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:49.571 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:49.571 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:49.580 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:29:49.655 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:49.652+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:49.756 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:49.753+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:49.770 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:49.770 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:49.771 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:49.771 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:49.771 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:49.780 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:29:49.857 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:49.853+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:49.920 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:49.920 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:49.921 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:49.921 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:49.921 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.921 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:49.921 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:49.921 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 28.822972088, 2023-12-06T18:29:49.921 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:29:49.921 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:49.921 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:29:49.921 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:49.921 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:29:49.921 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:49.921 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:29:49.921 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:49.922 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:49.922 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.922 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:49.922 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:49.922 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:49.922 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:49.922 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.922 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:49.922 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:49.922 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:49.922 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:49.922 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.922 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:49.922 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:29:49.923 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:29:49.923 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:49.923 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.923 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:49.923 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:29:49.923 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:29:49.923 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:49.923 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.923 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:49.923 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:29:49.923 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:29:49.923 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:49.923 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.923 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:49.923 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:29:49.924 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:29:49.924 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:49.924 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.924 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:49.924 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:29:49.924 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:29:49.924 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:49.924 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:49.924 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:49.924 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:29:49.924 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:29:49.924 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:49.924 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:49.924 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:49.924 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:49.925 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:49.925 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:49.930 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:29:49.975 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:49.971+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:50.059 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:50.055+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:50.059 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:50.059 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:50.060 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:50.060 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:50.069 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:29:50.160 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:50.157+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:50.239 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:50.240 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:50.249 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:29:50.293 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:50.289+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:50.364 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:50.360+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:50.381 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:50.381 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:50.381 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:50.382 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:50.382 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.382 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:50.382 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:50.382 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 88.888166736000002, 2023-12-06T18:29:50.382 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:29:50.382 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:50.382 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:29:50.382 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:29:50.382 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:29:50.382 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:50.382 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:29:50.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.383 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:50.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.383 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:50.383 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:50.383 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:50.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.383 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:50.383 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:50.383 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:50.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.383 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:50.383 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:29:50.383 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:29:50.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.384 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.384 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:50.384 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:29:50.384 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:29:50.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.384 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.384 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:50.384 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:29:50.384 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:29:50.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.384 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.384 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:50.384 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:29:50.384 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:29:50.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.384 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.385 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:50.385 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:29:50.385 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:29:50.385 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.385 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.385 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:50.385 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:29:50.385 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:29:50.385 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:50.385 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:50.385 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:50.385 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:50.385 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:50.385 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:50.390 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:29:50.464 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:50.461+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:50.560 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:50.560 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:50.561 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:50.561 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:50.565 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:50.562+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:50.571 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:29:50.583 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:29:50.669 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:50.662+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:50.725 INFO:teuthology.orchestra.run.smithi152.stderr:no valid command found; 10 closest matches: 2023-12-06T18:29:50.725 INFO:teuthology.orchestra.run.smithi152.stderr:0 2023-12-06T18:29:50.725 INFO:teuthology.orchestra.run.smithi152.stderr:1 2023-12-06T18:29:50.726 INFO:teuthology.orchestra.run.smithi152.stderr:2 2023-12-06T18:29:50.726 INFO:teuthology.orchestra.run.smithi152.stderr:abort 2023-12-06T18:29:50.726 INFO:teuthology.orchestra.run.smithi152.stderr:assert 2023-12-06T18:29:50.726 INFO:teuthology.orchestra.run.smithi152.stderr:bluefs debug_inject_read_zeros 2023-12-06T18:29:50.726 INFO:teuthology.orchestra.run.smithi152.stderr:bluefs files list 2023-12-06T18:29:50.726 INFO:teuthology.orchestra.run.smithi152.stderr:bluefs stats 2023-12-06T18:29:50.726 INFO:teuthology.orchestra.run.smithi152.stderr:bluestore allocator dump block 2023-12-06T18:29:50.726 INFO:teuthology.orchestra.run.smithi152.stderr:bluestore allocator fragmentation block 2023-12-06T18:29:50.726 INFO:teuthology.orchestra.run.smithi152.stderr:admin_socket: invalid command 2023-12-06T18:29:50.727 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-12-06T18:29:50.728 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-7, ['dump_ops_in_flight'] 2023-12-06T18:29:50.741 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:07.933 101796== Warning: unimplemented fcntl command: 1036 2023-12-06T18:29:50.742 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:50.742 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:50.742 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:50.742 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:50.742 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:50.752 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:29:50.793 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:50.790+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:50.868 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:50.865+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:50.964 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:50.965 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:50.965 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:50.965 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:50.965 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.965 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:29:50.965 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:50.965 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 174.347396914, 2023-12-06T18:29:50.965 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:29:50.965 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:50.965 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:50.965 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:50.965 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.965 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:50.965 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:50.966 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:50.966 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.966 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.966 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:50.966 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:50.966 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:50.966 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.966 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.966 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:50.966 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:29:50.966 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:29:50.966 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.966 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.966 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:50.967 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:29:50.967 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:29:50.967 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.967 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.967 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:50.967 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:29:50.967 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:29:50.967 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.967 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.967 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:50.967 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:29:50.967 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:29:50.967 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.967 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.967 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:50.967 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:29:50.968 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:29:50.968 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.968 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.968 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:50.968 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:29:50.968 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:29:50.968 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.968 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.968 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:50.968 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:29:50.968 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:29:50.968 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.968 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.968 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:50.968 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:29:50.968 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:29:50.969 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:50.969 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:50.969 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:50.969 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.969 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.969 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:29:50.969 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:50.969 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 173.324569905, 2023-12-06T18:29:50.969 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:29:50.969 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:50.969 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:50.969 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:50.969 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.969 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:50.969 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:50.970 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:50.970 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.970 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.970 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:50.970 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:50.970 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:50.970 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.970 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.970 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:50.970 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:29:50.970 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:29:50.970 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.970 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.970 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:50.970 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:29:50.971 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:29:50.971 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.971 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:50.971 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:29:50.971 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:29:50.971 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.971 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:50.971 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:29:50.971 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:29:50.971 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.971 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:50.971 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:29:50.972 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:29:50.972 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.972 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.972 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:50.972 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:29:50.972 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:29:50.972 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.972 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.972 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:50.972 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:29:50.972 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:29:50.972 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.972 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.972 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:50.972 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:29:50.972 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:29:50.973 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:50.973 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:50.973 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:50.973 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.973 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.973 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:29:50.973 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:50.973 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 173.31252326699999, 2023-12-06T18:29:50.973 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:29:50.973 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:50.973 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:50.973 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:50.973 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.973 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:50.973 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:50.974 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:50.974 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.974 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.974 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:50.974 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:50.974 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:50.974 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.974 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.974 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:50.974 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:29:50.974 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:29:50.974 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.974 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.974 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:50.974 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:29:50.974 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:29:50.975 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.975 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:50.975 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:29:50.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:29:50.975 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.975 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:50.975 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:29:50.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:29:50.975 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.975 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:50.975 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:29:50.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:29:50.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.976 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:50.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:29:50.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:29:50.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.976 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:50.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:29:50.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:29:50.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.976 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:50.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:29:50.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:29:50.977 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:50.977 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:50.977 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:50.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.977 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.977 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:29:50.977 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:50.977 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 173.29112266000001, 2023-12-06T18:29:50.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:29:50.977 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:50.977 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:50.977 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:50.977 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:50.977 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:50.978 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:50.978 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.978 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:50.978 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:50.978 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:50.978 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.978 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:50.978 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:29:50.979 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:29:50.979 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.979 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.979 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:50.979 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:29:50.979 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:29:50.979 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.979 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.979 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:50.979 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:29:50.979 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:29:50.979 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.979 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.979 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:50.979 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:29:50.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:29:50.980 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.980 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:50.980 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:29:50.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:29:50.980 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.980 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:50.980 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:29:50.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:29:50.980 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.980 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:50.981 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:29:50.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:29:50.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:50.981 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:29:50.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:29:50.981 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:50.981 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:50.981 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:50.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.981 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:29:50.981 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:50.981 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 173.28994333700001, 2023-12-06T18:29:50.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:29:50.982 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:50.982 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:50.982 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:50.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:50.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:50.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:50.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:50.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:50.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:50.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:50.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:29:50.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:29:50.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:50.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:29:50.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:29:50.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:50.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:29:50.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:29:50.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:50.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:29:50.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:29:50.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:50.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:29:50.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:29:50.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:50.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:29:50.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:29:50.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:50.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:29:50.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:29:50.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:50.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:29:50.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:29:50.985 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:50.985 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:50.985 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:50.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.985 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:29:50.985 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:50.985 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 173.289371356, 2023-12-06T18:29:50.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:29:50.986 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:50.986 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:50.986 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:50.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:50.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:50.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:50.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:50.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:50.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:50.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:50.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:29:50.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:29:50.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:50.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:29:50.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:29:50.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:50.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:29:50.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:29:50.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:50.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:29:50.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:29:50.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:50.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:29:50.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:29:50.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:50.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:29:50.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:29:50.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:50.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:29:50.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:29:50.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:50.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:29:50.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:29:50.989 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:50.989 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:50.989 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:50.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.989 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:29:50.989 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:50.990 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 172.84353588600001, 2023-12-06T18:29:50.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:29:50.990 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:50.990 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:50.991 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:50.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:50.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:50.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:50.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:50.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:50.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:50.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:50.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:29:50.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:29:50.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:50.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:29:50.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:29:50.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:50.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:29:50.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:29:50.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:50.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:29:50.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:29:50.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:50.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:29:50.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:29:50.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:50.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:29:50.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:29:50.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:50.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:29:50.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:29:50.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:50.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:29:50.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:29:50.994 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:50.994 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:50.995 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:50.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.995 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:29:50.995 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:50.996 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 172.84193603, 2023-12-06T18:29:50.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:29:50.996 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:50.996 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:50.996 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:50.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:50.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:50.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:50.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:50.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:50.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:50.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:50.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:29:50.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:29:50.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:50.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:29:50.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:29:50.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:50.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:29:50.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:29:50.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:50.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:29:50.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:29:50.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:50.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:29:50.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:29:50.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:50.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:29:50.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:29:50.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:50.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:29:50.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:29:50.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:50.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:50.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:50.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:29:50.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:29:50.999 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:50.999 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:50.999 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.000 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:29:51.000 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:51.000 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 172.46273041399999, 2023-12-06T18:29:51.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:29:51.000 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:51.000 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:51.000 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:51.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:51.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:51.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:51.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:51.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:51.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:29:51.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:29:51.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:51.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:29:51.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:29:51.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:51.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:29:51.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:29:51.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:51.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:29:51.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:29:51.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:51.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:29:51.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:29:51.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:51.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:29:51.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:29:51.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:51.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:29:51.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:29:51.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:51.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:29:51.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:29:51.004 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.004 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:51.004 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.004 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:29:51.005 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:51.005 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 172.22522823200001, 2023-12-06T18:29:51.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:29:51.005 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:51.005 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:51.005 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:51.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:51.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:51.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:51.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:51.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:51.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:29:51.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:29:51.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:51.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:29:51.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:29:51.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:51.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:29:51.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:29:51.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:51.008 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:29:51.008 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:29:51.008 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.008 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.008 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:51.008 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:29:51.008 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:29:51.008 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.008 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.008 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:51.008 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:29:51.008 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:29:51.008 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:51.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:29:51.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:29:51.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:51.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:29:51.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:29:51.010 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.010 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:51.010 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.010 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:29:51.010 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:51.010 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 172.22450023299999, 2023-12-06T18:29:51.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:29:51.010 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:51.010 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:51.010 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:51.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:51.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:51.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:51.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:51.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:51.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:29:51.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:29:51.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:51.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:29:51.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:29:51.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:51.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:29:51.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:29:51.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:51.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:29:51.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:29:51.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.013 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:51.013 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:29:51.013 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:29:51.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.013 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:51.013 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:29:51.013 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:29:51.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.013 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:51.014 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:29:51.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:29:51.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.014 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:51.014 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:29:51.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:29:51.014 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.014 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:51.014 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.014 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:29:51.014 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:51.015 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 172.21948782300001, 2023-12-06T18:29:51.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:29:51.015 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:51.015 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:51.015 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:51.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:51.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:51.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:51.016 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:51.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:51.016 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:29:51.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:29:51.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:51.016 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:29:51.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:29:51.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:51.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:29:51.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:29:51.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:51.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:29:51.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:29:51.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:51.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:29:51.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:29:51.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:51.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:29:51.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:29:51.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:51.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:29:51.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:29:51.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:51.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:29:51.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:29:51.018 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.018 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:51.019 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.019 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:29:51.019 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:51.019 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 172.21396074399999, 2023-12-06T18:29:51.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:29:51.019 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:51.019 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:51.019 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:51.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:51.019 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:51.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:51.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:51.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:51.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:29:51.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:29:51.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:51.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:29:51.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:29:51.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:51.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:29:51.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:29:51.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.021 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:51.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:29:51.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:29:51.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.021 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:51.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:29:51.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:29:51.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:51.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:29:51.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:29:51.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:51.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:29:51.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:29:51.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:51.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:29:51.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:29:51.023 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.023 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:51.023 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.024 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.024 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:29:51.024 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:51.024 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 172.21344990700001, 2023-12-06T18:29:51.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:29:51.024 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:51.025 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:51.025 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:51.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.025 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:51.025 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:51.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.025 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:51.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:51.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:51.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:29:51.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:29:51.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:51.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:29:51.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:29:51.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:51.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:29:51.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:29:51.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:51.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:29:51.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:29:51.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.029 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:51.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:29:51.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:29:51.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:51.030 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:29:51.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:29:51.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:51.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:29:51.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:29:51.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:51.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:29:51.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:29:51.031 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.031 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:51.031 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.031 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:29:51.032 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:51.032 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 172.21299064600001, 2023-12-06T18:29:51.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:29:51.032 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:51.032 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:51.032 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:51.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:51.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:51.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:51.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:51.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:51.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:29:51.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:29:51.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:51.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:29:51.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:29:51.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.034 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:51.034 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:29:51.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:29:51.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.034 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:51.034 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:29:51.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:29:51.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:51.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:29:51.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:29:51.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:51.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:29:51.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:29:51.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:51.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:29:51.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:29:51.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:51.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:29:51.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:29:51.037 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.037 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:51.037 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.037 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:29:51.037 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:51.037 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 165.63307483899999, 2023-12-06T18:29:51.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:29:51.037 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:51.037 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:51.037 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:51.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:51.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:51.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:51.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:51.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:51.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:29:51.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:29:51.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:51.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:29:51.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:29:51.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:51.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:29:51.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:29:51.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:51.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:29:51.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:29:51.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:51.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:29:51.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:29:51.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:51.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:29:51.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:29:51.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:51.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:29:51.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:29:51.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:51.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:29:51.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:29:51.041 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.041 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:51.041 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.042 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:29:51.042 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:51.042 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 165.626837034, 2023-12-06T18:29:51.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:29:51.042 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:51.042 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:51.042 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:51.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:51.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:51.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:51.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:51.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:51.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:29:51.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:29:51.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:51.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:29:51.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:29:51.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:51.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:29:51.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:29:51.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:51.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:29:51.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:29:51.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:51.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:29:51.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:29:51.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:51.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:29:51.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:29:51.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:51.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:29:51.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:29:51.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:51.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:29:51.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:29:51.046 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.046 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:51.046 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.046 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:29:51.046 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:51.046 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 165.620773955, 2023-12-06T18:29:51.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:29:51.046 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:51.047 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:51.047 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:51.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:51.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:51.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:51.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:51.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:51.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:29:51.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:29:51.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:51.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:29:51.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:29:51.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:51.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:29:51.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:29:51.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:51.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:29:51.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:29:51.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:51.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:29:51.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:29:51.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:51.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:29:51.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:29:51.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:51.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:29:51.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:29:51.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:51.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:29:51.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:29:51.051 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.051 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:51.051 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.051 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:29:51.051 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:51.051 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 165.48338687, 2023-12-06T18:29:51.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:29:51.051 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:51.052 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:51.052 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:51.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:51.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:51.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:51.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:51.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:51.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:29:51.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:29:51.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:51.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:29:51.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:29:51.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:51.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:29:51.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:29:51.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:51.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:29:51.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:29:51.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:51.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:29:51.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:29:51.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:51.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:29:51.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:29:51.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:51.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:29:51.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:29:51.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:51.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:29:51.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:29:51.055 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.055 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:51.055 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.056 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:29:51.056 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:51.056 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 165.476716143, 2023-12-06T18:29:51.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:29:51.056 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:51.056 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:51.056 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:51.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:51.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:51.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:51.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:51.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:51.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:51.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:29:51.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:29:51.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:51.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:29:51.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:29:51.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:51.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:29:51.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:29:51.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:51.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:29:51.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:29:51.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:51.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:29:51.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:29:51.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:51.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:29:51.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:29:51.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:51.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:29:51.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:29:51.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:51.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:51.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:51.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:29:51.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:29:51.060 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.060 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:51.060 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.060 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:51.060 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:51.060 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:51.062 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:29:51.062 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:08.171 101796== Warning: unimplemented fcntl command: 1036 2023-12-06T18:29:51.062 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:51.058+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:51.162 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:51.159+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:51.208 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:08.394 106652== Warning: unimplemented fcntl command: 1036 2023-12-06T18:29:51.232 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:51.232 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:51.232 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:51.232 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:51.242 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:29:51.284 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:51.280+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:51.364 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:51.360+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:51.413 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:51.413 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:51.413 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:51.413 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:51.414 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:51.423 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:29:51.455 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:08.640 106652== Warning: unimplemented fcntl command: 1036 2023-12-06T18:29:51.465 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:51.461+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:51.554 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:51.554 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:51.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:51.554 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:29:51.554 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:51.563 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:29:51.565 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:51.562+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:51.667 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:51.663+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:51.701 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:51.701 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:51.701 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:51.701 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:51.711 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:29:51.768 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:51.764+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:51.849 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:51.849 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:51.849 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:51.849 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:51.850 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:51.856 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:29:51.901 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:51.897+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:51.970 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:51.966+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:52.013 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:52.013 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:52.013 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:52.013 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:29:52.013 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:52.022 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:29:52.070 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:52.067+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:52.171 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:52.168+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:52.199 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:52.199 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:52.199 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:52.199 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:52.209 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:29:52.295 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:52.291+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:52.373 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:52.369+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:52.391 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:52.391 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:52.391 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:52.391 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:52.391 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:52.400 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:29:52.485 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:52.481+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:52.575 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:52.571+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:52.636 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:52.636 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:52.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:52.636 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:52.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.637 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:29:52.637 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:52.637 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 177.224949565, 2023-12-06T18:29:52.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:29:52.637 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.637 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.637 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.637 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.637 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.637 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:29:52.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.637 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:52.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:52.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:29:52.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:29:52.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:29:52.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:29:52.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:29:52.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:29:52.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:29:52.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:29:52.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:29:52.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:29:52.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:29:52.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:29:52.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:29:52.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:29:52.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:29:52.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:29:52.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:29:52.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:29:52.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:29:52.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:29:52.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:29:52.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.642 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.642 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.642 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:29:52.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:29:52.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.642 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.642 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.642 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:29:52.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:29:52.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.642 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.642 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.642 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:29:52.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:29:52.643 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.643 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.643 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.643 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.643 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.643 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:52.643 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:52.643 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 175.11637632599999, 2023-12-06T18:29:52.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:29:52.643 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.643 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.643 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.643 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.643 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.644 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:29:52.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.644 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:52.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:52.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.645 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.645 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:29:52.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:29:52.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.645 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.645 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:29:52.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:29:52.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.645 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.645 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:29:52.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:29:52.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:29:52.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:29:52.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:29:52.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:29:52.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:29:52.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:29:52.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.647 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:29:52.647 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:29:52.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.647 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:29:52.647 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:29:52.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.647 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:29:52.647 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:29:52.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.648 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:29:52.648 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:29:52.648 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.648 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.648 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.648 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:29:52.648 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:29:52.648 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.648 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.648 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.648 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.648 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.648 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:52.648 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:52.648 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 175.11563109400001, 2023-12-06T18:29:52.649 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:29:52.649 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.649 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.649 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.649 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.649 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.649 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:29:52.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.649 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.649 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.649 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.649 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:52.649 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.649 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.650 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.650 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:52.650 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.650 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.650 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.650 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:29:52.650 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:29:52.650 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.650 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.650 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:29:52.650 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:29:52.650 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.651 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.651 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:29:52.651 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:29:52.651 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.651 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.651 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.651 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:29:52.651 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:29:52.651 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.651 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.651 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.651 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:29:52.651 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:29:52.651 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.651 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.652 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:29:52.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:29:52.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.652 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:29:52.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:29:52.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.652 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:29:52.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:29:52.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.653 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:29:52.653 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:29:52.653 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.653 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.653 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.653 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:29:52.653 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:29:52.653 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.653 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.653 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.653 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:29:52.653 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:29:52.653 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.653 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.653 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.654 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.654 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.654 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:52.654 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:52.654 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 175.114947339, 2023-12-06T18:29:52.654 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:29:52.654 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.654 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.654 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.654 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.654 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.654 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:29:52.654 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.654 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.654 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.655 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.655 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:52.655 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.655 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.655 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.655 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.655 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:52.655 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.655 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.655 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.655 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.655 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:29:52.655 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:29:52.655 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.655 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.655 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.656 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:29:52.656 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:29:52.656 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.656 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.656 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.656 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:29:52.656 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:29:52.656 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.656 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.656 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.656 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:29:52.656 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:29:52.656 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.656 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.657 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.657 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:29:52.657 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:29:52.657 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.657 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.657 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.657 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:29:52.657 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:29:52.657 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.657 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.657 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.657 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:29:52.657 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:29:52.657 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.657 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.658 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:29:52.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:29:52.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.658 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:29:52.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:29:52.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.658 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:29:52.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:29:52.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.659 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:29:52.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:29:52.659 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.659 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.659 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.659 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:52.659 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:52.659 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 175.109374084, 2023-12-06T18:29:52.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:29:52.659 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.659 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.659 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.660 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.660 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.660 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:29:52.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.660 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.660 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.660 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:52.660 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.660 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.660 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:52.660 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.661 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.661 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:29:52.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:29:52.661 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.661 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.661 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:29:52.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:29:52.661 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.661 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.661 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:29:52.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:29:52.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.662 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.662 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.662 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:29:52.662 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:29:52.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.662 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.662 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.662 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:29:52.662 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:29:52.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.662 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.662 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.662 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:29:52.663 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:29:52.663 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.663 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.663 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.663 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:29:52.663 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:29:52.663 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.663 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.663 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.663 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:29:52.663 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:29:52.663 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.663 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.663 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.663 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:29:52.663 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:29:52.664 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.664 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.664 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.664 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:29:52.664 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:29:52.664 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.664 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.664 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.664 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:29:52.664 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:29:52.664 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.664 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.664 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.664 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.664 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.665 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:52.665 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:52.665 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 175.10870044399999, 2023-12-06T18:29:52.665 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:29:52.665 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.665 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.665 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.665 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.665 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.665 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:29:52.665 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.665 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.665 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.665 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.665 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:52.665 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.666 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.666 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.666 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.666 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:52.666 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.666 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.666 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.666 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.666 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:29:52.666 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:29:52.666 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.666 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.666 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.666 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:29:52.666 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:29:52.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.667 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.667 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.667 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:29:52.667 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:29:52.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.667 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.667 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.667 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:29:52.667 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:29:52.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.667 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.667 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.667 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:29:52.667 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:29:52.668 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.668 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.668 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:29:52.668 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:29:52.668 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.668 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.668 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:29:52.668 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:29:52.668 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.668 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.668 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:29:52.668 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:29:52.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.669 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.669 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:29:52.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:29:52.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.669 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.669 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:29:52.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:29:52.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.669 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.669 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:29:52.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:29:52.670 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.670 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.670 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.670 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:52.670 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:52.670 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 175.10803824499999, 2023-12-06T18:29:52.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:29:52.670 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.670 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.670 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.670 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.671 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.671 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:29:52.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.671 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:52.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:52.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:29:52.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:29:52.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:29:52.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:29:52.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:29:52.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:29:52.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.673 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:29:52.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:29:52.673 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.673 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.673 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.673 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:29:52.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:29:52.673 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.673 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.673 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.673 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:29:52.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:29:52.673 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.673 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.673 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:29:52.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:29:52.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:29:52.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:29:52.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:29:52.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:29:52.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:29:52.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:29:52.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:29:52.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:29:52.675 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.675 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.675 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.675 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:52.675 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:52.676 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 175.03532881000001, 2023-12-06T18:29:52.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:29:52.676 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.676 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.676 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.676 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.676 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.676 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:29:52.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.676 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.676 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.676 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.676 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:52.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:52.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:29:52.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:29:52.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:29:52.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:29:52.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:29:52.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:29:52.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:29:52.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:29:52.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:29:52.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:29:52.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:29:52.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:29:52.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:29:52.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:29:52.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:29:52.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:29:52.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:29:52.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:29:52.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:29:52.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:29:52.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:29:52.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:29:52.680 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.680 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.680 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.681 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:52.681 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:52.681 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 175.03464710099999, 2023-12-06T18:29:52.681 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:29:52.681 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.681 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.681 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.681 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.681 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.681 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:29:52.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.681 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.682 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.682 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:52.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.682 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.682 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:52.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.682 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.682 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:29:52.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:29:52.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.683 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.683 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.683 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:29:52.683 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:29:52.683 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.683 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.683 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.683 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:29:52.683 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:29:52.683 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.683 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.683 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.683 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:29:52.683 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:29:52.683 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.684 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.684 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.684 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:29:52.684 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:29:52.684 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.684 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.684 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.684 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:29:52.684 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:29:52.684 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.684 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.684 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.684 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:29:52.684 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:29:52.684 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:29:52.685 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:29:52.685 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:29:52.685 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:29:52.685 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:29:52.685 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:29:52.685 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.686 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.686 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:29:52.686 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:29:52.686 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.686 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.686 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.686 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.686 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.686 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:52.686 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:52.686 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 173.95584301299999, 2023-12-06T18:29:52.686 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:29:52.686 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.686 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.687 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.687 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.687 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.687 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:29:52.687 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.687 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.687 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.687 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.687 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:52.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.687 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.687 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.687 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.687 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:52.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.688 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.688 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.688 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:29:52.688 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:29:52.688 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.688 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.688 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:29:52.688 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:29:52.688 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.688 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.688 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:29:52.688 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:29:52.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.689 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.689 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:29:52.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:29:52.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.689 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.689 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:29:52.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:29:52.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.689 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.689 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:29:52.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:29:52.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:29:52.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:29:52.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:29:52.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:29:52.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:29:52.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:29:52.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:29:52.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:29:52.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:29:52.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:29:52.691 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.691 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.691 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.692 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:52.692 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:52.692 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 173.954426439, 2023-12-06T18:29:52.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:29:52.692 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.692 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.692 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.692 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.692 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.692 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:29:52.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.692 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:52.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:52.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:29:52.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:29:52.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:29:52.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:29:52.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:29:52.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:29:52.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:29:52.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:29:52.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:29:52.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:29:52.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:29:52.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:29:52.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:29:52.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:29:52.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:29:52.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:29:52.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:29:52.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:29:52.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:29:52.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:29:52.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:29:52.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:29:52.697 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.697 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.697 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.697 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:52.697 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:52.697 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 173.953341314, 2023-12-06T18:29:52.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:29:52.697 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.697 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.697 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.697 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.697 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.697 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:29:52.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.698 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:52.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:52.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:29:52.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:29:52.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:29:52.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:29:52.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:29:52.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:29:52.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:29:52.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:29:52.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:29:52.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:29:52.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:29:52.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:29:52.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:29:52.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:29:52.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:29:52.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:29:52.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:29:52.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:29:52.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:29:52.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:29:52.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:29:52.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:29:52.702 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.702 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.702 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.702 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:52.702 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:52.702 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 173.91819221200001, 2023-12-06T18:29:52.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:29:52.703 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.703 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.703 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.703 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.703 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.703 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:29:52.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.703 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:52.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:52.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:29:52.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:29:52.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:29:52.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:29:52.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:29:52.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:29:52.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.705 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.705 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:29:52.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:29:52.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.705 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.705 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:29:52.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:29:52.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.705 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.705 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:29:52.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:29:52.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:29:52.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:29:52.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:29:52.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:29:52.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:29:52.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:29:52.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:29:52.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:29:52.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:29:52.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:29:52.707 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.707 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.707 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.707 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:52.708 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:52.708 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 167.36846168700001, 2023-12-06T18:29:52.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:29:52.708 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.708 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.708 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.708 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.708 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:29:52.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.708 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:52.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:52.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:29:52.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:29:52.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:29:52.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:29:52.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.710 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:29:52.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:29:52.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.710 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:29:52.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:29:52.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.710 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:29:52.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:29:52.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:29:52.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:29:52.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:29:52.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:29:52.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:29:52.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:29:52.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:29:52.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:29:52.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:29:52.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:29:52.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:29:52.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:29:52.712 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.713 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.713 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.713 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:52.713 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:52.713 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 167.367656079, 2023-12-06T18:29:52.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:29:52.713 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.713 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.713 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.713 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.713 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.713 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:29:52.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.714 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:52.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:52.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:29:52.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:29:52.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:29:52.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:29:52.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:29:52.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:29:52.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:29:52.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:29:52.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:29:52.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:29:52.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:29:52.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:29:52.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:29:52.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:29:52.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:29:52.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:29:52.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:29:52.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:29:52.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:29:52.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:29:52.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:29:52.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:29:52.718 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.718 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.718 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.718 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:52.718 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:52.718 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 167.36696527000001, 2023-12-06T18:29:52.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:29:52.718 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.718 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.718 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.719 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.719 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.719 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:29:52.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.719 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:52.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:52.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:29:52.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:29:52.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:29:52.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:29:52.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:29:52.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:29:52.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:29:52.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:29:52.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:29:52.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:29:52.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:29:52.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:29:52.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:29:52.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:29:52.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:29:52.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:29:52.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:29:52.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:29:52.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:29:52.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:29:52.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:29:52.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:29:52.723 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.723 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.723 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.723 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:52.723 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:52.723 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 167.362667762, 2023-12-06T18:29:52.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:29:52.724 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.724 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.724 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.724 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.724 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.724 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:29:52.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.724 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:52.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:52.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:29:52.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:29:52.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:29:52.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:29:52.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.726 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:29:52.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:29:52.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.726 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:29:52.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:29:52.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.726 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:29:52.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:29:52.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:29:52.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:29:52.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:29:52.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:29:52.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:29:52.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:29:52.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:29:52.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:29:52.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:29:52.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:29:52.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:29:52.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:29:52.728 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.728 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.728 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.729 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:52.729 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:52.729 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 167.36200008700001, 2023-12-06T18:29:52.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:29:52.729 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.729 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.729 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.729 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.729 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.729 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:29:52.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.729 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:52.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:52.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:29:52.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:29:52.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:29:52.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:29:52.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:29:52.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:29:52.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:29:52.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:29:52.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:29:52.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:29:52.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:29:52.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:29:52.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:29:52.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:29:52.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:29:52.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:29:52.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:29:52.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:29:52.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:29:52.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:29:52.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:29:52.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:29:52.734 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.734 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.734 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.734 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:52.734 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:52.734 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 167.361344659, 2023-12-06T18:29:52.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:29:52.734 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.734 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.734 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.734 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.734 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.734 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:29:52.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.735 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:52.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:52.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:29:52.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:29:52.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:29:52.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:29:52.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:29:52.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:29:52.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:29:52.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:29:52.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:29:52.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:29:52.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:29:52.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:29:52.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:29:52.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:29:52.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:29:52.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:29:52.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:29:52.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:29:52.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:29:52.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:29:52.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:29:52.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:29:52.739 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.739 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.739 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.739 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:52.739 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:52.739 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 167.32069011199999, 2023-12-06T18:29:52.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:29:52.740 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:52.740 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:52.740 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:52.740 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:52.740 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:52.740 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:29:52.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.740 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:52.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:52.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:52.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:52.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:52.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:52.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:52.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:29:52.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:29:52.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:52.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:29:52.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:29:52.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:52.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:29:52.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:29:52.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:52.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:29:52.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:29:52.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:52.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:29:52.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:29:52.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:52.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:29:52.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:29:52.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:52.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:29:52.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:29:52.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:52.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:29:52.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:29:52.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:52.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:29:52.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:29:52.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:52.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:29:52.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:29:52.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:52.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:52.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:52.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:29:52.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:29:52.744 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.744 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.744 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.744 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:52.744 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:52.745 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:52.746 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:29:52.746 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:52.742+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:52.847 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:52.844+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:52.925 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:52.926 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:52.926 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:52.926 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:52.934 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:29:52.979 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:52.975+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:53.050 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:53.046+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:53.071 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:53.071 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:53.071 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:53.071 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:53.071 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:53.081 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:29:53.167 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:53.163+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:53.251 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:53.248+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:53.264 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:53.264 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:53.264 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:53.264 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:53.264 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.265 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:29:53.265 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:53.265 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 151.44783919400001, 2023-12-06T18:29:53.265 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:29:53.265 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:53.265 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:29:53.265 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:53.265 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.266 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:53.266 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:53.266 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:53.266 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:53.266 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.266 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:53.267 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:53.267 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:53.267 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:53.267 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.267 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:53.267 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:29:53.267 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:29:53.267 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:53.267 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.268 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:53.268 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:29:53.268 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:29:53.268 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:53.268 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.268 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:53.268 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:29:53.268 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:29:53.268 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:53.268 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.268 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:53.268 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:29:53.268 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:29:53.268 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:53.268 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.269 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:53.269 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:29:53.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:29:53.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:53.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.269 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:53.269 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:29:53.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:29:53.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:53.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.269 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:53.269 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:29:53.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:29:53.269 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:53.269 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:53.269 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:53.270 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:53.270 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:53.270 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:53.274 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:29:53.353 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:53.349+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:53.454 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:53.454 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:53.454 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:53.454 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:53.455 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:53.450+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:53.463 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:29:53.555 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:53.551+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:53.601 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:53.601 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:53.601 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:53.601 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:53.601 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:53.611 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:29:53.696 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:53.692+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:53.756 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:53.753+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:53.797 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:53.798 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:53.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:53.798 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:53.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.798 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:53.798 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:53.798 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 32.700163465999999, 2023-12-06T18:29:53.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:29:53.798 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:53.798 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:29:53.798 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:53.798 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:29:53.799 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:53.799 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:29:53.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:53.799 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:53.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:53.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:53.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:53.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:53.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:53.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:53.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:53.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:53.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:53.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:29:53.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:29:53.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:53.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:53.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:29:53.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:29:53.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:53.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:53.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:29:53.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:29:53.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:53.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:53.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:29:53.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:29:53.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:53.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:53.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:29:53.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:29:53.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:53.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:53.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:53.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:29:53.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:29:53.801 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:53.802 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:53.802 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:53.802 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:53.802 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:53.802 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:53.809 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:29:53.857 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:53.854+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:53.958 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:53.955+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:53.962 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:53.962 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:53.962 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:53.962 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:53.972 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:29:54.099 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:54.096+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:54.104 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:54.104 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:54.104 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:54.104 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:54.104 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:54.114 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:29:54.161 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:54.157+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:54.261 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:54.258+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:54.297 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:54.298 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:54.298 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:54.298 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:54.298 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.298 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:54.298 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:54.298 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 92.804155984000005, 2023-12-06T18:29:54.298 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:29:54.298 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.298 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:29:54.298 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:29:54.299 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:29:54.299 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:54.299 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:29:54.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.299 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.299 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.299 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:54.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.299 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.299 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:54.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:29:54.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:29:54.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:29:54.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:29:54.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.301 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:29:54.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:29:54.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.301 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.301 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:29:54.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:29:54.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.301 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.301 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:29:54.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:29:54.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.302 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.302 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:29:54.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:29:54.302 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.302 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.302 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.302 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.302 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.302 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:54.306 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:29:54.391 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:54.387+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:54.464 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:54.460+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:54.482 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:54.483 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:54.483 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:54.483 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:54.492 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:29:54.578 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:54.574+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:54.664 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:54.664 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:54.664 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:54.664 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:54.664 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:54.665 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:54.662+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:54.673 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:29:54.766 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:54.763+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:54.867 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:54.863+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:54.900 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:54.900 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:54.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:54.901 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:54.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.901 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:29:54.901 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:54.901 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 178.26918840600001, 2023-12-06T18:29:54.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:29:54.901 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.901 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.901 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:54.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:54.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:29:54.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:29:54.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:29:54.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:29:54.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:29:54.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:29:54.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:29:54.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:29:54.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:29:54.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:29:54.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:29:54.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:29:54.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:29:54.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:29:54.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:29:54.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:29:54.904 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.905 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.905 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.905 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:29:54.905 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:54.905 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 177.24636139699999, 2023-12-06T18:29:54.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:29:54.905 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.905 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.905 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:54.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:54.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:29:54.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:29:54.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:29:54.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:29:54.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:29:54.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:29:54.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:29:54.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:29:54.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:29:54.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:29:54.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:29:54.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:29:54.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:29:54.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:29:54.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:29:54.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:29:54.911 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.911 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.911 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.912 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:29:54.912 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:54.912 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 177.234314759, 2023-12-06T18:29:54.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:29:54.912 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.912 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.912 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:54.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:54.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:29:54.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:29:54.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:29:54.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:29:54.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:29:54.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:29:54.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:29:54.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:29:54.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:29:54.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:29:54.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:29:54.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:29:54.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:29:54.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:29:54.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:29:54.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:29:54.917 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.917 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.917 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.917 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:29:54.917 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:54.917 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 177.212914152, 2023-12-06T18:29:54.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:29:54.917 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.917 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.917 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:54.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:54.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:29:54.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:29:54.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:29:54.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:29:54.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:29:54.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:29:54.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:29:54.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:29:54.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:29:54.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:29:54.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:29:54.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:29:54.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:29:54.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:29:54.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:29:54.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:29:54.921 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.921 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.921 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.921 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:29:54.921 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:54.921 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 177.21173482899999, 2023-12-06T18:29:54.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:29:54.922 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.922 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.922 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:54.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:54.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:29:54.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:29:54.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:29:54.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:29:54.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:29:54.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:29:54.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:29:54.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:29:54.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:29:54.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:29:54.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:29:54.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:29:54.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:29:54.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:29:54.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:29:54.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:29:54.925 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.925 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.926 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.926 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:29:54.926 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:54.926 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 177.21116284799999, 2023-12-06T18:29:54.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:29:54.926 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.927 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.927 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:54.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:54.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:29:54.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:29:54.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:29:54.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:29:54.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:29:54.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:29:54.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:29:54.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:29:54.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:29:54.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:29:54.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:29:54.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:29:54.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:29:54.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:29:54.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:29:54.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:29:54.930 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.930 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.930 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.930 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:29:54.930 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:54.930 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 176.76532737799999, 2023-12-06T18:29:54.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:29:54.931 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.931 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.931 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:54.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:54.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:29:54.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:29:54.932 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.932 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:29:54.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:29:54.932 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.932 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:29:54.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:29:54.932 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:29:54.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:29:54.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:29:54.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:29:54.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:29:54.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:29:54.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:29:54.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:29:54.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:29:54.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:29:54.934 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.934 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.934 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.935 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:29:54.935 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:54.935 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 176.76372752200001, 2023-12-06T18:29:54.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:29:54.935 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.935 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.935 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.935 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.935 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:54.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.935 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.935 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:54.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.936 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.936 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.936 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.936 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:29:54.936 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:29:54.936 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.936 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.936 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.936 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:29:54.936 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:29:54.936 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.936 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.936 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.936 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:29:54.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:29:54.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:29:54.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:29:54.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:29:54.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:29:54.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:29:54.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:29:54.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:29:54.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:29:54.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:29:54.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:29:54.938 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.938 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.938 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.939 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:29:54.939 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:54.939 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 176.384521906, 2023-12-06T18:29:54.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:29:54.939 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.939 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.939 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.939 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.939 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.939 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:54.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.939 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.939 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.939 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:54.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:29:54.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:29:54.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:29:54.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:29:54.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:29:54.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:29:54.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:29:54.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:29:54.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:29:54.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:29:54.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:29:54.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:29:54.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:29:54.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:29:54.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:29:54.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:29:54.942 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.942 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.942 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.943 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:29:54.943 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:54.943 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 176.14701972399999, 2023-12-06T18:29:54.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:29:54.943 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.943 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.943 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.943 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:54.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.943 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:54.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.944 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.944 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.944 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.944 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:29:54.944 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:29:54.944 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.944 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.944 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.944 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:29:54.944 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:29:54.944 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.944 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.944 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.944 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:29:54.945 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:29:54.945 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.945 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.945 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.945 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:29:54.945 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:29:54.945 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.945 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.945 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.945 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:29:54.945 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:29:54.945 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.945 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.945 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.945 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:29:54.946 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:29:54.946 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.946 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.946 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.946 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:29:54.946 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:29:54.946 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.946 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.946 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.946 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:29:54.946 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:29:54.946 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.946 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.946 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.947 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.947 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.947 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:29:54.947 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:54.947 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 176.146291725, 2023-12-06T18:29:54.947 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:29:54.947 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.947 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.947 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.947 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.947 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.947 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:54.947 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.947 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.947 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.948 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.948 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:54.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.948 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.948 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:29:54.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:29:54.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.948 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.948 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:29:54.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:29:54.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.949 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.949 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:29:54.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:29:54.949 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.949 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.949 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.949 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:29:54.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:29:54.949 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.949 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.949 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.949 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:29:54.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:29:54.949 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.949 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.950 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.950 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:29:54.950 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:29:54.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.950 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.950 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.950 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:29:54.950 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:29:54.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.950 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.950 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.950 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:29:54.950 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:29:54.950 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.950 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.951 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.951 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:29:54.951 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:54.951 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 176.14127931499999, 2023-12-06T18:29:54.951 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:29:54.951 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.951 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.951 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.951 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.951 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:54.951 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.952 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.952 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.952 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.952 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:54.952 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.952 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.952 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.952 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.952 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:29:54.952 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:29:54.952 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.952 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.952 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.952 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:29:54.952 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:29:54.953 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.953 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.953 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.953 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:29:54.953 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:29:54.953 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.953 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.953 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.953 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:29:54.953 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:29:54.953 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.953 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.953 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.953 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:29:54.953 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:29:54.953 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.954 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.954 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.954 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:29:54.954 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:29:54.954 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.954 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.954 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.954 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:29:54.954 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:29:54.954 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.954 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.954 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.954 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:29:54.954 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:29:54.955 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.955 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.955 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.955 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.955 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.955 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:29:54.955 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:54.955 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 176.135752236, 2023-12-06T18:29:54.955 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:29:54.955 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.955 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.955 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.955 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.955 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.955 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:54.955 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.956 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.956 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.956 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.956 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:54.956 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.956 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.956 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.956 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.956 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:29:54.956 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:29:54.956 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.956 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.956 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.956 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:29:54.956 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:29:54.957 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.957 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.957 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.957 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:29:54.957 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:29:54.957 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.957 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.957 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.957 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:29:54.957 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:29:54.957 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.957 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.957 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.957 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:29:54.957 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:29:54.957 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.958 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.958 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.958 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:29:54.958 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:29:54.958 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.958 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.958 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.958 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:29:54.958 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:29:54.958 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.958 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.958 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.958 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:29:54.959 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:29:54.959 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.959 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.959 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.959 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.959 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.959 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:29:54.959 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:54.959 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 176.13524139899999, 2023-12-06T18:29:54.959 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:29:54.959 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.959 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.959 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.959 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.959 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:54.960 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.960 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.960 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.960 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:54.960 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.960 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.960 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.960 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:29:54.960 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:29:54.960 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.960 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.960 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:29:54.961 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:29:54.961 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.961 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.961 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.961 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:29:54.961 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:29:54.961 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.961 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.961 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.961 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:29:54.961 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:29:54.961 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.961 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.961 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.961 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:29:54.962 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:29:54.962 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.962 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.962 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.962 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:29:54.962 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:29:54.962 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.962 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.962 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.962 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:29:54.962 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:29:54.962 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.962 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.962 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.962 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:29:54.963 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:29:54.963 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.963 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.963 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.963 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.963 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.963 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:29:54.963 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:54.963 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 176.13478213799999, 2023-12-06T18:29:54.963 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:29:54.963 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.963 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.963 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.963 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.963 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.964 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:54.964 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.964 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.964 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.964 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.964 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:54.964 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.964 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.964 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.964 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.964 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:29:54.964 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:29:54.964 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.964 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.964 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.964 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:29:54.964 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:29:54.965 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.965 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.965 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.965 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:29:54.965 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:29:54.965 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.965 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.965 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.965 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:29:54.965 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:29:54.965 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.965 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.965 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.965 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:29:54.966 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:29:54.966 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.966 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.966 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.966 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:29:54.966 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:29:54.966 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.966 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.966 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.966 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:29:54.966 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:29:54.966 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.966 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.966 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.966 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:29:54.966 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:29:54.967 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.967 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.967 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.967 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.967 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.967 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:29:54.967 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:54.967 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 169.554866331, 2023-12-06T18:29:54.967 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:29:54.967 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.967 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.967 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.967 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.967 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.967 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:54.967 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.968 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.968 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.968 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.968 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:54.968 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.968 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.968 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.968 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.968 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:29:54.968 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:29:54.968 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.968 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.968 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.968 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:29:54.968 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:29:54.968 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.969 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.969 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.969 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:29:54.969 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:29:54.969 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.969 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.969 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.969 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:29:54.969 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:29:54.969 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.969 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.969 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.969 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:29:54.969 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:29:54.969 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.969 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.970 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.970 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:29:54.970 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:29:54.970 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.970 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.970 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.970 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:29:54.970 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:29:54.970 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.970 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.970 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.970 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:29:54.970 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:29:54.970 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.970 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.971 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.971 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.971 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:29:54.971 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:54.971 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 169.54862852599999, 2023-12-06T18:29:54.971 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:29:54.971 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.971 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.971 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.971 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.971 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:54.971 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.971 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.972 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.972 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:54.972 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.972 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.972 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.972 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.972 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:29:54.972 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:29:54.972 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.974 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.974 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.974 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:29:54.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:29:54.975 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.975 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.975 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:29:54.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:29:54.975 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.975 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:29:54.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:29:54.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.976 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:29:54.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:29:54.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.977 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:29:54.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:29:54.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.977 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.977 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:29:54.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:29:54.978 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.978 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.978 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:29:54.978 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:29:54.978 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.978 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.979 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.979 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.979 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.979 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:29:54.979 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:54.979 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 169.54256544699999, 2023-12-06T18:29:54.979 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:29:54.980 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.980 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.980 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.980 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.980 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:54.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.980 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.981 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:54.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.981 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:29:54.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:29:54.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:29:54.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:29:54.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:29:54.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:29:54.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:29:54.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:29:54.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:29:54.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:29:54.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:29:54.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:29:54.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:29:54.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:29:54.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:29:54.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:29:54.984 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.984 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.984 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.984 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:29:54.984 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:54.984 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 169.40517836199999, 2023-12-06T18:29:54.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:29:54.985 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.985 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.985 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:54.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:54.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:29:54.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:29:54.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:29:54.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:29:54.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:29:54.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:29:54.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:29:54.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:29:54.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:29:54.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:29:54.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:29:54.987 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:29:54.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.987 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.987 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:29:54.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:29:54.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:29:54.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:29:54.988 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.988 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.988 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.988 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:29:54.988 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:54.988 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 169.39850763499999, 2023-12-06T18:29:54.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:29:54.989 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:54.989 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:54.989 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:54.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:54.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:54.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:54.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:54.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:54.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:54.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:29:54.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:29:54.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:54.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:29:54.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:29:54.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:54.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:29:54.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:29:54.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:54.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:29:54.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:29:54.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:54.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:29:54.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:29:54.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:54.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:29:54.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:29:54.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:54.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:29:54.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:29:54.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:54.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:54.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:54.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:29:54.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:29:54.992 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.992 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.992 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.992 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:54.992 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:54.992 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:54.994 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:29:55.002 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:54.994+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:55.095 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:55.092+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:55.165 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:55.166 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:55.166 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:55.166 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:55.174 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:29:55.196 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:55.192+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:55.297 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:55.294+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:55.343 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:55.344 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:55.344 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:55.344 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:55.344 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:55.352 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:29:55.398 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:55.395+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:55.481 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:55.482 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:55.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:55.482 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:29:55.482 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:55.490 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:29:55.535 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:55.531+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:55.600 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:55.596+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:55.632 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:55.633 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:55.633 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:55.633 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:55.642 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:29:55.727 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:55.723+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:55.728 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:29:55.802 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:55.798+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:55.819 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:55.819 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:55.819 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:55.819 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:55.819 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:55.828 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:29:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:no valid command found; 10 closest matches: 2023-12-06T18:29:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:0 2023-12-06T18:29:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:1 2023-12-06T18:29:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:2 2023-12-06T18:29:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:abort 2023-12-06T18:29:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:assert 2023-12-06T18:29:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:bluefs debug_inject_read_zeros 2023-12-06T18:29:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:bluefs files list 2023-12-06T18:29:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:bluefs stats 2023-12-06T18:29:55.840 INFO:teuthology.orchestra.run.smithi152.stderr:bluestore allocator dump block 2023-12-06T18:29:55.841 INFO:teuthology.orchestra.run.smithi152.stderr:bluestore allocator fragmentation block 2023-12-06T18:29:55.841 INFO:teuthology.orchestra.run.smithi152.stderr:admin_socket: invalid command 2023-12-06T18:29:55.842 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-12-06T18:29:55.843 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-7, ['dump_ops_in_flight'] 2023-12-06T18:29:55.913 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:55.909+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:56.004 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:56.000+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:56.005 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:56.005 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:56.005 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:56.006 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:29:56.006 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:56.013 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:29:56.105 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:56.101+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:56.193 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:56.193 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:56.193 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:56.193 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:56.202 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:29:56.244 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:56.240+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:56.306 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:56.303+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:56.378 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:56.378 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:56.379 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:56.379 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:56.379 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:56.387 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:29:56.432 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:56.428+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:56.509 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:56.505+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:56.587 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:56.587 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:56.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:56.587 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:56.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.587 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:29:56.587 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:56.587 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 181.17660379099999, 2023-12-06T18:29:56.588 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:29:56.588 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.588 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.588 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.588 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.588 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.588 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:29:56.588 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.588 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.588 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.588 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.588 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:56.588 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.589 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.589 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.589 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:29:56.589 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.589 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.589 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.589 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:29:56.589 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:29:56.589 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.589 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.589 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:29:56.589 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:29:56.590 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.590 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.590 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.590 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:29:56.590 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:29:56.590 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.590 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.590 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.590 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:29:56.590 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:29:56.590 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.590 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.590 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.590 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:29:56.590 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:29:56.591 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.591 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.591 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:29:56.591 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:29:56.591 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:29:56.591 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.591 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.592 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.592 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:29:56.592 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:29:56.592 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.592 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.592 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.592 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:29:56.592 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:29:56.593 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.593 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.593 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:29:56.593 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:29:56.593 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.593 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.594 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.594 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:29:56.594 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:29:56.594 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.594 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.594 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.594 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:29:56.595 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:29:56.595 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.595 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.595 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.595 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:29:56.595 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:29:56.595 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.595 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.596 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.596 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:29:56.596 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:29:56.596 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.596 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.596 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.596 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.596 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.596 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:56.597 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:56.597 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 179.06803055200001, 2023-12-06T18:29:56.597 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:29:56.597 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.597 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.597 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.597 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.597 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.597 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:29:56.597 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.598 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.598 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:56.598 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.598 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:29:56.598 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:29:56.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:29:56.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.599 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:29:56.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:29:56.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.599 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:29:56.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:29:56.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.599 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.600 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.600 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:29:56.600 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:29:56.600 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.600 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.600 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.600 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:29:56.600 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:29:56.600 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.600 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.600 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.600 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:29:56.600 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:29:56.600 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.600 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.600 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.601 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:29:56.601 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:29:56.601 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.601 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.601 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.601 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:29:56.601 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:29:56.601 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.601 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.601 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.601 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:29:56.601 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:29:56.601 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.602 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.602 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.602 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:29:56.602 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:29:56.602 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.602 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.602 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.602 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:29:56.602 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:29:56.602 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.602 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.602 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.602 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.602 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.603 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:56.603 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:56.603 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 179.06728532, 2023-12-06T18:29:56.603 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:29:56.603 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.603 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.603 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.603 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.603 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.603 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:29:56.603 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.603 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.603 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.604 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.604 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:56.604 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.604 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.604 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.604 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.604 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:29:56.604 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.604 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.604 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.604 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.604 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:29:56.604 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:29:56.604 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.604 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.605 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.605 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:29:56.605 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:29:56.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.605 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.605 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.605 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:29:56.605 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:29:56.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.605 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.605 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.605 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:29:56.605 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:29:56.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.605 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.605 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.606 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:29:56.606 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:29:56.606 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.606 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.606 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.606 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:29:56.606 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:29:56.606 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.606 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.606 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.606 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:29:56.606 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:29:56.606 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.606 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.607 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.607 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:29:56.607 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:29:56.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.609 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.609 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.609 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:29:56.610 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:29:56.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.610 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.610 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.610 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:29:56.610 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:29:56.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.610 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.610 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.610 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:29:56.610 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:29:56.610 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.610 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.611 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.611 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.611 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.611 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:56.611 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:56.611 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 179.06660156500001, 2023-12-06T18:29:56.611 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:29:56.611 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.611 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.611 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.611 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.611 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.611 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:29:56.611 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.611 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.612 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.612 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.612 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:56.612 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.612 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.612 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.612 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.612 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:29:56.612 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.612 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.612 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.612 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.613 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:29:56.613 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:29:56.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.613 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.613 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.613 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:29:56.613 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:29:56.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.613 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.613 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.613 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:29:56.613 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:29:56.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.613 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.613 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:29:56.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:29:56.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.614 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:29:56.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:29:56.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.614 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:29:56.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:29:56.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.615 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:29:56.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:29:56.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.615 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:29:56.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:29:56.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.615 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:29:56.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:29:56.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:29:56.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:29:56.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:29:56.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:29:56.616 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.616 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.616 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.616 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:56.616 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:56.617 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 179.06102831000001, 2023-12-06T18:29:56.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:29:56.617 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.617 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.617 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.617 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.617 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.617 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:29:56.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.617 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:56.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:29:56.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:29:56.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:29:56.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:29:56.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:29:56.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.619 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.619 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:29:56.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:29:56.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.619 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.619 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:29:56.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:29:56.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.620 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.620 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.620 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:29:56.620 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:29:56.620 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.620 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.620 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.620 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:29:56.620 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:29:56.620 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.621 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.621 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.621 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:29:56.621 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:29:56.621 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.621 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.621 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.621 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:29:56.621 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:29:56.621 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.621 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.622 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.622 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:29:56.622 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:29:56.622 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.622 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.622 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.622 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:29:56.622 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:29:56.622 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.622 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.622 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.622 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:29:56.622 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:29:56.623 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.623 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.623 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.623 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.623 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.623 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:56.623 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:56.623 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 179.06035467000001, 2023-12-06T18:29:56.623 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:29:56.623 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.623 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.623 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.623 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.624 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.624 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:29:56.624 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.624 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.624 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.624 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.624 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:56.624 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.624 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.624 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.624 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:29:56.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:29:56.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:29:56.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.626 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:29:56.626 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:29:56.626 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.626 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.626 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.626 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:29:56.626 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:29:56.626 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.626 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.626 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.626 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:29:56.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:29:56.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.627 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:29:56.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:29:56.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.627 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:29:56.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:29:56.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:29:56.628 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:29:56.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.628 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.628 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:29:56.628 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:29:56.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.628 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.628 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:29:56.628 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:29:56.629 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.629 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.629 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.629 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:29:56.629 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:29:56.629 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.629 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.629 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.629 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:29:56.629 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:29:56.629 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.629 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.630 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.630 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.630 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:56.630 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:56.630 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 179.05969247100001, 2023-12-06T18:29:56.630 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:29:56.630 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.631 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.631 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.631 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.631 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.631 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:29:56.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.631 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.631 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.631 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.631 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:56.632 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.632 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.632 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:29:56.632 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.632 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.632 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:29:56.632 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:29:56.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.633 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.633 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:29:56.633 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:29:56.633 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.633 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.633 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.633 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:29:56.633 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:29:56.633 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.633 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.633 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.633 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:29:56.633 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:29:56.633 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.633 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.634 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.634 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:29:56.634 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:29:56.634 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.634 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.634 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.634 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:29:56.634 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:29:56.634 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.634 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.634 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.634 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:29:56.634 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:29:56.634 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.635 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.635 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.635 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:29:56.635 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:29:56.635 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.635 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.635 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.635 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:29:56.635 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:29:56.635 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:29:56.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:29:56.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:29:56.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:29:56.636 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.636 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.636 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.637 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:56.637 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:56.637 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 178.986983036, 2023-12-06T18:29:56.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:29:56.637 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.637 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.637 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.637 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.637 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.637 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:29:56.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.637 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:56.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:29:56.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:29:56.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:29:56.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:29:56.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:29:56.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:29:56.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:29:56.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:29:56.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:29:56.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:29:56.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:29:56.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:29:56.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:29:56.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:29:56.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:29:56.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:29:56.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:29:56.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:29:56.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:29:56.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.642 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.642 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.642 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:29:56.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:29:56.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.642 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.642 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.642 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:29:56.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:29:56.642 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.642 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.642 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.643 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.643 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.643 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:29:56.643 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:56.643 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 178.98630132700001, 2023-12-06T18:29:56.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:29:56.643 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.643 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.643 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.644 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.644 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.644 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:29:56.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.644 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.645 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:56.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.645 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.645 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:29:56.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:29:56.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:29:56.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.647 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:29:56.647 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:29:56.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.648 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:29:56.648 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:29:56.648 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.648 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.648 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.648 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:29:56.648 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:29:56.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.649 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.649 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.649 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:29:56.649 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:29:56.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.650 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.650 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:29:56.650 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:29:56.650 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.650 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.651 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:29:56.651 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:29:56.651 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.651 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.651 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.651 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:29:56.651 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:29:56.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.652 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:29:56.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:29:56.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.653 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.653 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.653 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:29:56.653 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:29:56.653 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.653 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.653 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.654 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:29:56.654 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:29:56.654 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.654 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.654 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.654 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.654 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.655 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:56.655 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:56.655 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 177.90749723900001, 2023-12-06T18:29:56.655 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:29:56.655 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.655 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.656 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.656 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.656 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.656 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:29:56.656 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.656 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.656 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.657 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.657 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:56.657 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.657 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.657 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.657 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:29:56.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.658 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:29:56.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:29:56.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.659 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:29:56.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:29:56.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.660 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:29:56.660 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:29:56.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.660 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:29:56.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:29:56.661 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.661 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.661 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:29:56.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:29:56.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.662 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.662 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.662 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:29:56.662 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:29:56.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.663 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.663 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.663 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:29:56.663 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:29:56.663 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.663 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.663 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.664 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:29:56.664 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:29:56.664 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.664 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.664 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.664 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:29:56.665 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:29:56.665 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.665 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.665 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.665 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:29:56.665 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:29:56.665 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.666 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.666 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.666 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:29:56.666 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:29:56.666 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.666 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.666 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.667 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.667 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:56.667 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:56.667 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 177.90608066499999, 2023-12-06T18:29:56.667 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:29:56.667 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.668 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.668 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.668 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.668 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.668 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:29:56.668 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.669 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.669 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.669 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:56.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.670 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.670 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:29:56.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.670 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.670 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:29:56.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:29:56.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:29:56.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:29:56.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:29:56.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:29:56.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.673 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.673 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:29:56.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:29:56.673 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.673 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.673 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:29:56.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:29:56.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:29:56.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:29:56.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:29:56.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:29:56.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.676 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.676 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.676 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:29:56.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:29:56.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.676 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:29:56.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:29:56.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:29:56.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:29:56.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:29:56.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:29:56.679 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.679 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.679 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.679 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:56.680 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:56.680 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 177.90499553999999, 2023-12-06T18:29:56.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:29:56.680 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.680 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.680 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.680 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.681 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.681 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:29:56.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.681 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:56.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.682 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.682 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:29:56.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.683 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.683 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.683 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.683 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:29:56.683 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:29:56.683 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.683 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.684 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.684 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:29:56.684 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:29:56.684 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.684 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.684 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.684 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:29:56.684 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:29:56.685 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:29:56.685 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:29:56.685 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.686 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:29:56.686 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:29:56.686 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.686 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.686 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.686 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:29:56.686 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:29:56.686 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.686 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.687 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.687 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:29:56.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:29:56.687 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.687 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.687 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.687 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:29:56.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:29:56.687 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.688 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.688 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:29:56.688 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:29:56.688 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.688 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.689 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:29:56.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:29:56.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.689 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.689 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:29:56.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:29:56.689 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.689 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.689 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.689 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:29:56.690 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:56.690 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 177.869846438, 2023-12-06T18:29:56.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:29:56.690 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.690 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.690 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.690 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.690 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.690 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:29:56.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.690 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:56.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:29:56.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:29:56.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:29:56.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:29:56.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:29:56.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:29:56.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:29:56.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:29:56.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:29:56.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:29:56.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:29:56.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:29:56.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:29:56.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:29:56.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:29:56.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:29:56.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:29:56.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:29:56.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:29:56.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:29:56.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:29:56.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:29:56.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:29:56.695 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.695 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.695 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.695 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:56.696 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:56.696 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 171.320115913, 2023-12-06T18:29:56.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:29:56.696 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.696 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.696 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.696 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.696 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.696 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:29:56.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.696 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:56.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:29:56.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:29:56.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:29:56.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:29:56.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:29:56.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:29:56.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:29:56.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:29:56.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:29:56.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:29:56.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:29:56.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:29:56.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:29:56.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:29:56.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:29:56.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:29:56.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:29:56.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:29:56.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:29:56.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:29:56.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:29:56.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:29:56.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:29:56.702 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.702 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.702 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.703 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:56.703 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:56.703 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 171.31931030499999, 2023-12-06T18:29:56.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:29:56.703 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.703 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.704 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.704 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.704 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.704 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:29:56.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.704 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.705 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.705 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:56.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.705 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.705 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:29:56.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:29:56.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:29:56.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:29:56.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:29:56.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:29:56.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:29:56.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:29:56.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:29:56.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:29:56.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:29:56.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:29:56.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:29:56.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:29:56.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:29:56.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:29:56.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:29:56.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:29:56.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:29:56.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:29:56.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:29:56.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:29:56.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:29:56.709 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.709 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.709 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.710 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:56.710 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:56.710 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 171.318619496, 2023-12-06T18:29:56.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:29:56.710 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.710 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.710 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.710 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.710 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.710 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:29:56.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.710 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:56.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:29:56.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:29:56.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:29:56.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:29:56.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:29:56.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:29:56.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:29:56.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:29:56.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:29:56.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:29:56.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:29:56.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:29:56.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:29:56.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:29:56.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:29:56.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:29:56.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:29:56.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:29:56.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:29:56.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:29:56.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:29:56.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:29:56.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:29:56.715 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.715 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.715 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.715 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:56.715 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:56.715 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 171.31432198799999, 2023-12-06T18:29:56.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:29:56.715 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.715 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.715 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.715 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.716 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.716 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:29:56.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.716 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:56.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:29:56.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:29:56.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:29:56.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:29:56.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:29:56.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:29:56.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:29:56.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:29:56.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:29:56.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:29:56.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:29:56.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:29:56.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:29:56.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:29:56.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:29:56.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:29:56.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:29:56.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:29:56.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:29:56.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:29:56.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:29:56.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:29:56.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:29:56.720 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.720 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.720 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.720 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:56.720 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:56.720 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 171.313654313, 2023-12-06T18:29:56.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:29:56.721 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.721 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.721 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.721 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.721 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.721 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:29:56.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.721 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:56.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:29:56.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:29:56.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:29:56.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:29:56.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:29:56.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:29:56.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:29:56.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:29:56.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:29:56.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:29:56.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:29:56.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:29:56.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:29:56.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:29:56.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:29:56.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:29:56.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:29:56.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:29:56.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:29:56.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:29:56.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:29:56.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:29:56.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:29:56.725 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.725 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.725 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.726 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:56.726 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:56.726 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 171.31299888500001, 2023-12-06T18:29:56.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:29:56.726 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.726 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.726 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.726 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.726 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.726 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:29:56.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.726 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:56.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:29:56.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:29:56.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:29:56.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:29:56.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:29:56.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:29:56.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:29:56.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:29:56.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:29:56.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:29:56.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:29:56.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:29:56.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:29:56.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:29:56.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:29:56.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:29:56.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:29:56.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:29:56.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:29:56.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:29:56.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:29:56.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:29:56.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:29:56.731 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.731 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.731 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.731 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:29:56.731 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:56.731 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 171.27234433800001, 2023-12-06T18:29:56.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:29:56.731 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:56.731 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:56.731 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:56.732 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:29:56.732 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:29:56.732 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:29:56.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.732 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:56.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:56.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:56.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:56.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:29:56.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:56.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:56.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:29:56.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:29:56.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:56.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:29:56.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:29:56.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:56.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:29:56.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:29:56.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:56.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:29:56.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:29:56.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:56.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:29:56.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:29:56.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:56.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:29:56.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:29:56.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:29:56.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:29:56.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:29:56.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:29:56.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:29:56.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:29:56.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:29:56.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:29:56.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:29:56.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:29:56.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:29:56.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:29:56.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:56.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:56.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:56.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:29:56.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:29:56.736 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.736 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.736 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.737 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:56.737 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:56.737 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:56.738 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:29:56.739 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:56.735+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:56.839 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:56.835+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:56.916 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:56.916 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:56.916 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:56.916 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:56.926 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:29:56.940 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:56.937+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:57.042 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:57.038+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:57.062 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:57.062 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:57.062 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:57.062 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:57.063 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:57.073 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:29:57.143 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:57.139+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:57.213 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:57.213 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:57.213 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:57.213 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:57.213 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.213 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:29:57.213 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:57.213 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 155.396876663, 2023-12-06T18:29:57.214 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:29:57.214 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:57.214 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:29:57.214 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:57.214 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.214 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:57.214 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:57.214 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:57.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:57.214 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.214 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:57.214 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:29:57.214 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:57.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:57.215 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.215 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:57.215 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:29:57.215 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:29:57.215 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:57.215 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.215 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:57.215 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:29:57.215 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:29:57.215 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:57.215 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.215 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:57.215 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:29:57.215 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:29:57.215 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:57.216 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.216 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:57.216 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:29:57.216 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:29:57.216 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:57.216 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.216 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:57.216 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:29:57.216 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:29:57.216 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:57.216 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.216 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:29:57.216 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:29:57.216 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:29:57.216 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:57.217 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.217 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:57.217 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:29:57.217 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:29:57.217 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:57.217 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:57.217 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:57.217 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:57.217 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:57.217 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:57.223 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:29:57.243 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:29:57.240+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:29:57.345 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:57.341+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:57.401 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:57.401 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:57.401 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:57.401 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:57.411 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:29:57.446 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:57.442+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:57.546 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:57.543+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:57.588 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:57.588 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:57.588 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:57.588 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:57.588 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:57.597 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:29:57.681 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:57.677+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:57.747 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:57.744+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:57.776 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:57.776 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:57.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:57.777 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:29:57.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.777 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:57.777 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:57.777 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 36.678933682999997, 2023-12-06T18:29:57.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:29:57.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:29:57.777 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:29:57.777 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:29:57.777 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:29:57.777 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:57.777 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:29:57.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:57.777 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:29:57.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:29:57.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:57.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:57.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:57.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:29:57.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:29:57.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:29:57.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:57.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:29:57.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:29:57.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:29:57.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:57.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:29:57.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:29:57.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:29:57.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:57.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:29:57.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:29:57.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:29:57.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:57.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:29:57.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:29:57.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:29:57.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:57.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:29:57.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:29:57.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:29:57.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:29:57.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:29:57.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:29:57.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:29:57.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:29:57.780 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:57.780 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:57.780 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:57.780 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:29:57.780 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:29:57.780 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:57.785 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:29:57.849 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:57.845+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:57.930 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:57.931 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:57.931 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:57.931 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:57.940 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:29:57.950 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:57.946+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:58.051 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:58.048+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:58.067 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:58.067 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:58.067 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:58.067 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:58.067 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:58.076 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:29:58.152 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:58.148+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:58.207 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:58.207 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:58.207 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:58.207 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:58.208 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.208 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:29:58.208 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:58.208 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 96.714144374, 2023-12-06T18:29:58.208 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:29:58.208 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.208 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:29:58.208 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:29:58.208 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:29:58.208 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:29:58.208 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:29:58.208 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.208 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.208 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.209 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.209 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:58.209 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.209 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.209 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.209 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.209 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:29:58.209 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.209 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.209 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.209 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.209 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:29:58.209 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:29:58.209 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.209 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.210 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.210 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:29:58.210 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:29:58.210 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.210 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.210 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.210 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:29:58.210 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:29:58.210 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.210 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.210 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.210 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:29:58.210 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:29:58.210 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.211 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.211 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.211 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:29:58.211 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:29:58.211 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.211 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.211 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.211 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:29:58.211 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:29:58.211 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.211 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.211 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.211 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.211 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.211 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:58.216 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:29:58.258 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:58.254+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:58.354 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:58.351+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:58.386 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:58.386 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:58.386 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:58.386 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:58.395 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:29:58.455 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:58.452+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:58.522 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:58.522 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:58.522 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:58.522 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:58.522 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:58.530 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:29:58.571 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:58.568+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:58.657 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:58.653+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:58.746 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:58.746 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:58.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:58.746 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:29:58.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.746 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:29:58.746 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:58.746 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 182.12856398, 2023-12-06T18:29:58.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:29:58.747 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.747 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.747 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:58.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:29:58.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:29:58.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:29:58.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:29:58.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:29:58.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:29:58.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:29:58.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:29:58.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:29:58.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:29:58.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:29:58.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:29:58.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:29:58.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.750 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:29:58.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:29:58.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.750 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:29:58.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:29:58.750 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.750 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.750 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.751 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:29:58.751 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:58.751 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 181.105736971, 2023-12-06T18:29:58.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:29:58.751 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.751 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.751 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:58.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:29:58.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:29:58.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:29:58.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:29:58.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:29:58.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:29:58.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:29:58.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:29:58.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:29:58.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:29:58.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:29:58.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:29:58.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:29:58.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:29:58.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:29:58.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:29:58.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:29:58.754 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.755 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.755 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.755 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:29:58.755 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:58.755 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 181.09369033300001, 2023-12-06T18:29:58.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:29:58.755 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.755 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.755 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:58.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:29:58.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:29:58.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:29:58.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:29:58.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:29:58.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:29:58.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:29:58.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:29:58.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:29:58.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:29:58.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:29:58.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:29:58.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:29:58.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:29:58.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:29:58.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:29:58.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:29:58.759 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.759 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.759 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.759 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:29:58.759 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:58.759 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 181.07228972600001, 2023-12-06T18:29:58.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:29:58.759 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.759 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.759 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:58.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.760 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:29:58.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.760 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:29:58.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:29:58.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:29:58.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:29:58.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:29:58.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:29:58.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:29:58.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:29:58.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:29:58.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:29:58.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:29:58.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:29:58.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:29:58.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:29:58.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:29:58.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:29:58.763 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.763 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.763 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.763 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:29:58.763 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:58.763 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 181.07111040300001, 2023-12-06T18:29:58.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:29:58.763 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.763 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.763 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:58.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:29:58.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:29:58.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:29:58.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:29:58.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:29:58.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:29:58.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:29:58.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:29:58.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:29:58.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:29:58.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:29:58.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:29:58.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:29:58.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:29:58.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:29:58.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:29:58.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:29:58.767 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.767 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.767 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.767 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:29:58.767 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:58.767 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 181.070538422, 2023-12-06T18:29:58.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:29:58.767 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.767 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.768 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.768 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.768 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:58.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.768 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.768 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:29:58.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.768 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.768 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:29:58.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:29:58.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:29:58.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:29:58.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:29:58.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:29:58.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:29:58.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:29:58.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:29:58.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:29:58.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:29:58.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:29:58.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:29:58.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:29:58.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:29:58.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:29:58.771 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.771 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.771 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.771 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:29:58.771 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:58.771 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 180.62470295200001, 2023-12-06T18:29:58.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:29:58.771 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.771 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.772 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:58.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:29:58.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:29:58.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:29:58.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:29:58.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:29:58.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:29:58.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:29:58.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:29:58.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:29:58.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:29:58.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:29:58.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:29:58.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:29:58.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:29:58.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:29:58.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:29:58.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:29:58.775 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.775 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.775 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.775 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:29:58.775 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:58.775 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 180.62310309599999, 2023-12-06T18:29:58.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:29:58.775 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.775 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.776 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:58.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:29:58.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:29:58.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:29:58.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:29:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:29:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:29:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:29:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:29:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:29:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:29:58.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:29:58.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:29:58.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:29:58.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:29:58.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:29:58.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:29:58.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:29:58.779 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.779 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.779 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.779 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:29:58.779 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:58.779 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 180.24389747999999, 2023-12-06T18:29:58.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:29:58.779 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.779 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.779 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:58.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:29:58.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:29:58.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:29:58.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:29:58.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:29:58.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:29:58.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:29:58.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:29:58.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:29:58.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:29:58.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:29:58.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:29:58.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:29:58.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:29:58.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:29:58.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:29:58.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:29:58.783 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.783 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.783 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.783 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:29:58.783 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:58.783 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 180.006395298, 2023-12-06T18:29:58.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:29:58.783 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.783 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.784 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:58.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:29:58.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:29:58.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:29:58.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:29:58.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:29:58.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:29:58.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:29:58.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:29:58.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:29:58.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:29:58.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:29:58.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:29:58.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:29:58.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:29:58.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:29:58.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:29:58.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:29:58.787 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.787 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.787 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.787 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:29:58.787 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:58.787 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 180.00566729900001, 2023-12-06T18:29:58.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:29:58.787 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.788 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.788 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:58.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:29:58.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:29:58.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:29:58.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:29:58.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:29:58.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:29:58.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:29:58.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:29:58.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:29:58.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:29:58.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:29:58.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:29:58.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:29:58.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:29:58.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:29:58.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:29:58.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:29:58.791 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.791 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.791 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.791 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:29:58.791 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:58.791 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 180.000654889, 2023-12-06T18:29:58.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:29:58.792 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.792 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.792 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:58.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:29:58.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:29:58.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:29:58.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:29:58.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:29:58.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:29:58.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:29:58.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:29:58.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:29:58.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:29:58.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:29:58.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:29:58.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:29:58.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:29:58.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:29:58.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:29:58.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:29:58.795 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.795 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.795 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.795 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:29:58.795 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:58.795 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 179.99512781000001, 2023-12-06T18:29:58.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:29:58.795 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.796 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.796 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:58.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:29:58.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:29:58.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:29:58.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:29:58.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:29:58.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:29:58.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:29:58.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:29:58.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:29:58.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:29:58.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:29:58.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:29:58.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:29:58.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:29:58.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:29:58.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:29:58.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:29:58.799 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.799 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.799 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.799 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:29:58.799 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:58.799 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 179.99461697300001, 2023-12-06T18:29:58.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:29:58.800 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.800 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.800 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:58.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:29:58.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:29:58.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:29:58.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:29:58.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:29:58.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:29:58.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:29:58.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.802 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.802 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:29:58.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:29:58.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.802 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.802 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:29:58.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:29:58.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.802 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.802 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:29:58.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:29:58.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:29:58.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:29:58.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:29:58.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:29:58.803 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.803 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.803 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.804 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:29:58.804 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:58.804 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 179.994157712, 2023-12-06T18:29:58.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:29:58.804 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.804 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.804 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:58.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:29:58.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:29:58.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:29:58.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:29:58.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:29:58.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:29:58.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:29:58.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:29:58.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:29:58.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:29:58.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:29:58.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:29:58.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:29:58.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:29:58.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:29:58.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:29:58.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:29:58.807 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.807 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.807 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.808 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:29:58.808 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:58.808 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 173.41424190500001, 2023-12-06T18:29:58.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:29:58.808 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.808 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.808 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:58.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:29:58.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:29:58.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:29:58.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:29:58.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:29:58.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:29:58.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:29:58.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:29:58.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:29:58.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:29:58.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:29:58.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:29:58.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:29:58.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:29:58.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:29:58.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:29:58.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:29:58.811 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.811 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.811 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.812 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.812 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.812 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:29:58.812 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:58.812 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 173.4080041, 2023-12-06T18:29:58.812 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:29:58.812 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.812 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.812 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.812 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.812 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.812 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:58.812 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.812 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.812 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.813 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.813 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:29:58.813 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.813 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.813 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.813 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.813 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:29:58.813 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:29:58.813 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.813 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.813 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.813 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:29:58.813 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:29:58.813 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.813 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.814 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.814 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:29:58.814 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:29:58.814 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.814 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.814 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.814 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:29:58.814 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:29:58.814 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.814 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.814 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.814 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:29:58.814 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:29:58.814 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.815 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.815 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.815 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:29:58.815 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:29:58.815 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.815 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.815 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.815 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:29:58.815 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:29:58.815 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.815 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.815 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.815 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:29:58.815 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:29:58.815 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.816 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.816 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.816 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.816 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.816 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:29:58.816 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:58.816 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 173.401941021, 2023-12-06T18:29:58.816 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:29:58.816 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.816 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.816 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.816 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.816 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.816 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:58.816 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.817 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.817 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.817 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.817 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:29:58.817 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.817 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.817 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.817 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.817 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:29:58.817 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:29:58.817 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.817 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.817 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.817 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:29:58.817 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:29:58.818 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.818 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.818 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.818 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:29:58.818 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:29:58.818 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.818 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.818 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.818 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:29:58.818 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:29:58.818 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.818 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.818 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.818 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:29:58.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:29:58.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.819 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:29:58.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:29:58.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.819 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:29:58.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:29:58.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.819 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:29:58.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:29:58.819 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.820 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.820 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.820 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:29:58.820 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:58.820 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 173.264553936, 2023-12-06T18:29:58.820 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:29:58.820 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.820 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.820 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.820 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.820 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:58.820 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.821 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.821 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:29:58.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.821 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.821 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:29:58.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:29:58.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.821 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.821 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:29:58.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:29:58.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.822 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.822 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:29:58.822 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:29:58.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.822 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.822 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.822 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:29:58.822 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:29:58.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.822 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.822 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.822 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:29:58.822 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:29:58.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.823 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.823 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:29:58.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:29:58.823 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.823 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.823 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:29:58.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:29:58.823 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.823 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.823 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:29:58.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:29:58.823 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.823 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.823 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.824 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.824 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.824 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:29:58.824 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:58.824 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 173.257883209, 2023-12-06T18:29:58.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:29:58.824 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:29:58.824 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:29:58.824 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:29:58.824 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.824 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:29:58.824 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:58.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.824 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.824 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.824 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:29:58.824 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:29:58.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:29:58.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.825 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:29:58.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:29:58.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:29:58.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.825 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:29:58.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:29:58.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:29:58.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.825 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:29:58.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:29:58.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:29:58.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:29:58.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:29:58.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:29:58.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:29:58.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:29:58.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:29:58.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:29:58.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:29:58.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:29:58.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:29:58.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:29:58.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:29:58.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:29:58.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:29:58.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:29:58.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:29:58.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:29:58.827 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.827 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.827 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.827 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:29:58.827 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:29:58.827 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:58.829 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:29:58.829 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:29:58.825+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:29:58.930 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:29:58.926+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:29:58.999 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:58.999 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:58.999 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:29:58.999 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:59.009 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:29:59.030 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:59.027+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:59.132 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:59.128+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:59.138 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:59.138 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:29:59.138 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:29:59.138 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:29:59.138 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:59.147 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:29:59.232 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:59.228+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:59.276 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:29:59.276 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:29:59.276 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:29:59.276 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:29:59.276 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:29:59.285 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:29:59.333 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:59.330+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:59.435 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:59.431+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:59.437 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:59.437 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:59.438 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:59.438 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:59.447 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:29:59.536 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:29:59.532+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:29:59.624 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:59.624 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:59.624 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:29:59.624 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:29:59.624 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:59.634 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:29:59.637 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:59.634+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:59.737 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:29:59.734+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:29:59.810 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:59.810 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:29:59.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:29:59.811 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:29:59.811 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:59.819 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:29:59.864 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:29:59.860+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:29:59.940 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:29:59.936+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:29:59.957 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:29:59.957 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:29:59.957 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:29:59.957 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:29:59.966 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:30:00.040 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:00.037+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:00.142 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:00.138+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:00.146 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:00.146 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:00.147 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:00.147 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:00.147 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:00.156 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:30:00.242 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:00.239+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:00.343 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:00.340+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:00.393 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:00.393 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:00.393 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:00.393 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:00.393 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.393 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:30:00.393 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:00.393 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 184.98023191600001, 2023-12-06T18:30:00.393 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:30:00.394 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.394 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.394 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.394 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.394 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.394 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:30:00.394 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.394 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.394 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.394 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.394 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:00.394 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.394 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.394 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.394 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.395 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:00.395 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.395 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.395 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.395 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.395 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:30:00.395 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:30:00.395 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.395 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.395 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.395 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:30:00.395 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:30:00.395 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.395 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.395 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.396 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:30:00.396 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:30:00.396 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.396 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.396 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.396 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:30:00.396 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:30:00.396 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.396 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.396 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.396 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:30:00.396 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:30:00.396 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.396 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.396 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:30:00.397 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:30:00.397 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:30:00.397 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.397 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.397 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.397 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:30:00.397 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:30:00.397 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.397 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.397 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.397 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:30:00.397 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:30:00.397 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.398 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.398 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.398 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:30:00.398 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:30:00.398 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.398 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.398 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.398 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:30:00.398 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:30:00.398 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.399 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.399 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.399 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:30:00.399 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:30:00.399 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.399 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.399 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.399 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:30:00.399 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:30:00.399 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.399 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.399 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.399 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:30:00.399 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:30:00.400 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.400 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.400 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.400 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.400 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.400 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:00.400 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:00.400 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 182.871658677, 2023-12-06T18:30:00.400 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:30:00.400 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.400 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.400 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.400 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.400 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.400 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:30:00.401 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.401 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.401 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.401 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.401 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:00.401 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.401 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.401 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.401 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.401 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:00.401 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.401 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.401 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.401 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.401 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:30:00.401 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:30:00.402 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.402 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.402 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.402 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:30:00.402 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:30:00.402 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.402 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.402 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.402 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:30:00.402 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:30:00.402 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.402 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.402 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.402 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:30:00.402 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:30:00.403 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.403 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.403 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:30:00.403 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:30:00.403 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.403 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.403 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:30:00.403 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:30:00.403 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.403 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.403 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:30:00.403 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:30:00.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.404 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.404 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:30:00.404 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:30:00.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.404 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.404 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:30:00.404 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:30:00.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.404 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.404 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:30:00.404 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:30:00.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.405 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.405 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.405 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:30:00.405 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:30:00.405 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.405 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.405 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.405 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.405 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.405 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:00.405 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:00.405 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 182.87091344500001, 2023-12-06T18:30:00.405 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:30:00.405 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.405 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.405 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.406 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.406 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.406 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:30:00.406 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.406 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.406 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.406 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.406 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:00.406 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.406 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.406 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.406 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:00.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.407 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.407 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:30:00.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:30:00.407 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.407 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:30:00.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:30:00.407 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.407 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.408 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:30:00.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:30:00.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.408 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.408 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:30:00.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:30:00.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.408 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.408 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:30:00.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:30:00.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.409 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.409 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:30:00.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:30:00.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.409 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.409 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.409 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:30:00.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:30:00.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.409 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.409 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.409 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:30:00.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:30:00.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.410 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.410 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.410 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:30:00.410 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:30:00.410 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.410 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.410 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.410 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:30:00.410 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:30:00.410 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.410 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.410 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.410 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:30:00.410 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:30:00.410 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.411 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.411 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.411 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.411 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.411 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:00.411 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:00.411 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 182.87022969, 2023-12-06T18:30:00.411 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:30:00.411 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.411 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.411 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.411 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.411 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.411 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:30:00.411 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.412 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:00.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.412 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:00.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.412 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:30:00.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:30:00.413 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.413 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.413 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.413 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:30:00.413 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:30:00.413 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.413 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.413 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.413 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:30:00.413 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:30:00.413 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.413 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.413 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.413 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:30:00.413 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:30:00.413 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.414 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.414 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.414 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:30:00.414 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:30:00.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.414 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.414 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.414 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:30:00.414 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:30:00.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.414 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.414 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.414 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:30:00.414 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:30:00.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.415 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.415 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.415 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:30:00.415 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:30:00.415 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.415 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.415 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.415 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:30:00.415 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:30:00.415 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.415 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.415 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.415 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:30:00.415 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:30:00.415 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.416 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.416 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.416 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:30:00.416 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:30:00.416 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.416 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.416 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.416 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.416 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.416 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:00.416 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:00.416 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 182.864656435, 2023-12-06T18:30:00.416 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:30:00.416 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.416 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.416 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.417 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.417 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.417 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:30:00.417 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.417 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.417 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.417 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.417 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:00.417 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.417 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.417 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.417 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.417 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:00.417 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.417 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.417 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.418 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.418 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:30:00.418 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:30:00.418 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.418 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.418 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.418 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:30:00.418 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:30:00.418 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.418 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.418 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.418 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:30:00.418 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:30:00.418 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.418 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.419 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.419 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:30:00.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:30:00.419 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.419 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.419 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.419 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:30:00.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:30:00.419 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.419 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.419 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.419 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:30:00.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:30:00.419 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:30:00.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:30:00.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:30:00.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:30:00.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:30:00.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:30:00.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.421 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.421 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.421 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:30:00.421 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:30:00.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.421 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.421 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.421 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:30:00.421 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:30:00.421 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.421 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.421 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.421 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.421 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:00.421 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:00.422 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 182.863982795, 2023-12-06T18:30:00.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:30:00.422 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.422 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.422 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.422 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.422 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.422 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:30:00.422 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.422 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.422 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.422 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.422 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:00.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.422 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.423 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.423 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:00.423 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.423 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.423 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:30:00.423 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:30:00.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.423 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.423 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:30:00.423 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:30:00.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.424 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.424 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:30:00.424 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:30:00.424 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.424 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.424 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:30:00.424 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:30:00.424 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.424 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.424 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:30:00.424 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:30:00.424 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:30:00.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:30:00.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:30:00.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:30:00.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:30:00.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:30:00.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:30:00.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:30:00.426 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.426 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.426 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.426 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:30:00.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:30:00.426 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.426 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.426 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.426 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:30:00.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:30:00.426 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.426 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.426 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.426 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.427 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.427 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:00.427 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:00.427 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 182.86332059599999, 2023-12-06T18:30:00.427 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:30:00.427 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.427 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.427 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.427 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.427 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.427 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:30:00.427 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.427 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.427 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.427 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.427 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:00.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.428 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.428 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.428 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:00.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.428 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.428 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.428 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:30:00.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:30:00.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.428 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.428 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.428 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:30:00.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:30:00.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.429 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.429 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.429 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:30:00.429 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:30:00.429 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.429 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.429 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.429 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:30:00.429 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:30:00.429 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.429 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.429 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.429 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:30:00.429 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:30:00.429 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.429 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.430 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.430 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:30:00.430 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:30:00.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.430 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.430 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.430 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:30:00.430 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:30:00.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.430 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.430 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.430 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:30:00.430 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:30:00.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.431 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.431 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.431 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:30:00.431 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:30:00.431 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.431 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.431 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.431 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:30:00.431 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:30:00.431 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.431 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.431 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.431 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:30:00.431 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:30:00.431 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.432 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.432 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.432 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.432 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.432 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:00.432 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:00.432 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 182.79061116099999, 2023-12-06T18:30:00.432 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:30:00.432 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.432 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.432 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.432 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.432 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.432 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:30:00.432 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.433 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.433 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.433 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.433 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:00.433 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.433 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.433 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.433 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.433 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:00.433 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.433 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.433 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.433 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.433 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:30:00.433 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:30:00.434 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.434 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.434 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.434 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:30:00.434 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:30:00.434 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.434 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.434 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.434 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:30:00.434 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:30:00.434 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.434 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.434 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.434 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:30:00.434 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:30:00.435 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.435 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.435 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.435 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:30:00.435 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:30:00.435 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.435 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.435 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.435 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:30:00.435 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:30:00.435 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.435 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.435 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.435 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:30:00.435 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:30:00.435 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.436 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.436 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:30:00.436 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:30:00.436 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.436 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.436 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:30:00.436 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:30:00.436 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.436 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.436 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:30:00.436 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:30:00.436 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.437 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.437 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:30:00.437 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:30:00.437 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.437 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.437 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.437 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.437 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.437 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:00.437 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:00.437 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 182.789929452, 2023-12-06T18:30:00.437 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:30:00.437 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.437 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.437 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.438 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.438 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.438 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:30:00.438 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.438 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.438 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.438 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.438 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:00.438 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.438 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.438 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.438 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.438 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:00.438 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.438 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.439 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.439 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.439 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:30:00.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:30:00.439 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.439 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.439 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.439 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:30:00.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:30:00.439 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.439 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.439 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.439 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:30:00.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:30:00.440 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.440 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.440 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.440 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:30:00.440 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:30:00.440 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.440 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.440 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.440 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:30:00.440 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:30:00.440 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.440 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.440 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.440 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:30:00.440 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:30:00.440 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:30:00.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:30:00.441 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:30:00.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:30:00.441 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:30:00.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:30:00.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.442 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.442 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.442 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:30:00.442 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:30:00.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.442 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.442 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.442 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:30:00.442 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:30:00.442 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.442 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.442 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.443 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.443 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:00.443 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:00.443 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 181.711125364, 2023-12-06T18:30:00.443 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:30:00.443 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.443 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.443 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.443 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.443 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.443 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:30:00.443 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.443 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.443 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.443 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.444 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:00.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.444 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.444 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:00.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.444 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.444 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:30:00.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:30:00.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.444 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.445 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:30:00.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:30:00.445 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.445 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.445 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.445 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:30:00.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:30:00.445 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.445 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.445 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.445 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:30:00.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:30:00.445 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.445 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.445 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.446 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:30:00.446 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:30:00.446 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.446 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.446 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:30:00.446 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:30:00.446 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.446 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.446 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:30:00.446 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:30:00.446 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.447 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.447 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:30:00.447 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:30:00.447 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.447 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.447 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.447 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:30:00.447 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:30:00.447 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.447 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.447 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.447 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:30:00.447 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:30:00.447 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.447 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.447 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.448 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:30:00.448 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:30:00.448 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.448 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.448 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.448 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.448 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.448 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:00.448 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:00.448 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 181.70970879000001, 2023-12-06T18:30:00.448 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:30:00.448 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.448 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.449 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.449 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.449 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.449 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:30:00.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.449 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:00.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:00.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.450 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.450 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:30:00.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:30:00.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.450 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.450 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:30:00.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:30:00.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.450 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.450 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:30:00.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:30:00.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.451 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.451 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:30:00.451 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:30:00.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.451 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.451 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:30:00.451 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:30:00.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.451 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.451 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:30:00.451 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:30:00.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:30:00.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:30:00.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:30:00.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:30:00.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:30:00.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:30:00.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.453 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.453 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:30:00.453 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:30:00.453 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.453 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.453 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.453 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:30:00.453 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:30:00.453 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.453 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.453 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.453 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.453 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.453 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:00.453 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:00.453 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 181.708623665, 2023-12-06T18:30:00.454 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:30:00.454 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.454 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.454 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.454 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.454 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.454 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:30:00.454 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.454 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.454 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.454 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.454 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:00.454 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.454 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.454 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:00.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:30:00.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:30:00.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:30:00.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:30:00.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:30:00.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:30:00.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.456 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:30:00.456 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:30:00.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.456 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:30:00.456 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:30:00.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.456 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:30:00.456 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:30:00.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:30:00.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:30:00.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:30:00.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:30:00.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:30:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:30:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:30:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:30:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:30:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:30:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 181.67347456300001, 2023-12-06T18:30:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:30:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:30:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:30:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:30:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:30:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:30:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:30:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:30:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:30:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:30:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:30:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:30:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:30:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:30:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:30:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:30:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:30:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:30:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:30:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:30:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:30:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:30:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:30:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:30:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 175.12374403800001, 2023-12-06T18:30:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:30:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:30:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:30:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:30:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:30:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:30:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:30:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:30:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:30:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:30:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:30:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:30:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:30:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:30:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:30:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:30:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:30:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:30:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:30:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:30:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:30:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:30:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:30:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:30:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 175.12293843, 2023-12-06T18:30:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:30:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:30:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:30:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:30:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:30:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:30:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:30:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:30:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:30:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:30:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:30:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:30:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:30:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:30:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:30:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:30:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:30:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:30:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:30:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:30:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:30:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:30:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:30:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:30:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 175.12224762100001, 2023-12-06T18:30:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:30:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:30:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:30:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:30:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:30:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:30:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:30:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:30:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:30:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:30:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:30:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:30:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:30:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:30:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:30:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:30:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:30:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:30:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:30:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:30:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:30:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:30:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:30:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:30:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 175.11795011300001, 2023-12-06T18:30:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:30:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:30:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:30:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:30:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:30:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:30:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:30:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:30:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:30:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:30:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:30:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:30:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:30:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:30:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:30:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:30:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:30:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:30:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:30:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:30:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:30:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:30:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:30:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:30:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 175.11728243799999, 2023-12-06T18:30:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:30:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:30:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:30:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:30:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:30:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:30:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:30:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:30:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:30:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:30:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:30:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:30:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:30:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:30:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:30:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:30:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:30:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:30:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:30:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:30:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:30:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:30:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:30:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:30:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 175.11662701, 2023-12-06T18:30:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:30:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:30:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:30:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:30:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:30:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:30:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:30:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:30:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:30:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:30:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:30:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:30:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:30:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:30:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:30:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:30:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:30:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:30:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:30:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:30:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:30:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:30:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:30:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:30:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 175.075972463, 2023-12-06T18:30:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:30:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:30:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:30:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:30:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:30:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:30:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:30:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:30:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:30:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:30:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:30:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:30:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:30:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:30:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:30:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:30:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:30:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:30:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:30:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:30:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:30:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:30:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:30:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:30:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:00.505 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:00.507 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:30:00.507 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:00.503+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:00.607 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:00.604+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:00.684 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:00.685 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:00.685 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:00.685 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:00.695 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:30:00.708 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:00.705+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:00.810 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:00.806+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:00.844 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:30:00.884 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:00.884 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:00.884 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:00.884 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:00.884 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:00.893 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:30:00.935 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:00.931+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:00.960 INFO:teuthology.orchestra.run.smithi152.stderr:no valid command found; 10 closest matches: 2023-12-06T18:30:00.960 INFO:teuthology.orchestra.run.smithi152.stderr:0 2023-12-06T18:30:00.960 INFO:teuthology.orchestra.run.smithi152.stderr:1 2023-12-06T18:30:00.960 INFO:teuthology.orchestra.run.smithi152.stderr:2 2023-12-06T18:30:00.960 INFO:teuthology.orchestra.run.smithi152.stderr:abort 2023-12-06T18:30:00.960 INFO:teuthology.orchestra.run.smithi152.stderr:assert 2023-12-06T18:30:00.961 INFO:teuthology.orchestra.run.smithi152.stderr:bluefs debug_inject_read_zeros 2023-12-06T18:30:00.961 INFO:teuthology.orchestra.run.smithi152.stderr:bluefs files list 2023-12-06T18:30:00.961 INFO:teuthology.orchestra.run.smithi152.stderr:bluefs stats 2023-12-06T18:30:00.961 INFO:teuthology.orchestra.run.smithi152.stderr:bluestore allocator dump block 2023-12-06T18:30:00.961 INFO:teuthology.orchestra.run.smithi152.stderr:bluestore allocator fragmentation block 2023-12-06T18:30:00.961 INFO:teuthology.orchestra.run.smithi152.stderr:admin_socket: invalid command 2023-12-06T18:30:00.962 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-12-06T18:30:00.962 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-7, ['dump_ops_in_flight'] 2023-12-06T18:30:01.012 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:01.009+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:01.081 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:30:01.083 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:01.083 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:01.083 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:01.084 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:01.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.084 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:30:01.084 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:01.084 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 159.267153261, 2023-12-06T18:30:01.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:30:01.085 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:01.085 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:30:01.085 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:01.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:01.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:01.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:01.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:01.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:01.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:01.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:01.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:01.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.087 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:01.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:30:01.087 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:30:01.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:01.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.087 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:01.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:30:01.088 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:30:01.088 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:01.088 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.088 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:01.088 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:30:01.088 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:30:01.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:01.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:01.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:30:01.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:30:01.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:01.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:30:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:30:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:30:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:30:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:30:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:30:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:01.092 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:01.092 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:01.092 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:01.092 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:01.092 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:01.093 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:30:01.094 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:30:01.095 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:30:01.113 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:01.109+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:01.215 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:01.211+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:01.234 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:01.234 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:01.234 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:01.234 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:01.244 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:30:01.316 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:01.312+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:01.389 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:01.389 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:01.389 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:01.389 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:01.389 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:01.398 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:30:01.439 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":30,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:30:01.439 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:01.436+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:01.452 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2023-12-06T18:30:01.453 INFO:tasks.ceph.mon.a:Restarting daemon 2023-12-06T18:30:01.453 DEBUG:teuthology.orchestra.run.smithi061:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2023-12-06T18:30:01.495 INFO:tasks.ceph.mon.a:Started 2023-12-06T18:30:01.495 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2023-12-06T18:30:01.495 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:30:01.518 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:01.515+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:01.550 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:01.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 40.452021299000002, 2023-12-06T18:30:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:30:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:30:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:01.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:30:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:30:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:01.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:30:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:30:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:30:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:30:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:30:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:30:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:30:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:30:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:30:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:30:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:01.554 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:01.555 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:01.555 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:01.555 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:01.555 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:01.564 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:30:01.619 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:01.615+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:01.710 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:01.710 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:01.710 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:01.710 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:01.719 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:30:01.721 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:01.717+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:01.822 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:01.818+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:01.848 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:01.848 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:01.848 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:01.849 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:01.849 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:01.859 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:30:01.925 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:01.920+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:02.010 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:02.010 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:02.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:02.010 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:02.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.010 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:02.010 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:02.010 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 100.516288989, 2023-12-06T18:30:02.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:30:02.010 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.011 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:02.011 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:02.011 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:02.011 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:02.011 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:30:02.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.011 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:02.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:02.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:30:02.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:30:02.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:30:02.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:30:02.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:30:02.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:30:02.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.013 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.013 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:30:02.013 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:30:02.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.013 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.013 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:30:02.013 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:30:02.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.013 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.013 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:30:02.013 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:30:02.014 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.014 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.014 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.014 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.014 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.014 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:02.019 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:30:02.024 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:02.021+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:02.105 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is unset 2023-12-06T18:30:02.164 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:02.160+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:02.191 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:02.191 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:02.192 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:02.192 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:02.200 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:30:02.228 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":30,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:30:02.245 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:02.241+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:02.327 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:02.324+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:02.332 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:02.332 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:02.333 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:02.333 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:02.333 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:02.343 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:30:02.428 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:02.425+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:02.529 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:02.526+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:02.552 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:02.552 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:02.552 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:02.552 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:02.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.552 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:30:02.552 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:02.552 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 185.934502111, 2023-12-06T18:30:02.552 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:30:02.552 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.552 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.552 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:02.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:02.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:30:02.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:30:02.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.554 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.554 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:30:02.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:30:02.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.554 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.554 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:30:02.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:30:02.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.554 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.555 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:30:02.555 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:30:02.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.555 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.555 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.555 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:30:02.555 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:30:02.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.555 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.555 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.555 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:30:02.555 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:30:02.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.556 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.556 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.556 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:30:02.556 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:30:02.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.556 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.556 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.556 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:30:02.556 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:30:02.556 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.556 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.556 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.556 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.556 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:30:02.556 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:02.557 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 184.911675102, 2023-12-06T18:30:02.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:30:02.557 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.557 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.557 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.557 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:02.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.557 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:02.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.558 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.558 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.558 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:30:02.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:30:02.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.558 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.558 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.558 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:30:02.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:30:02.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.558 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.558 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.558 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:30:02.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:30:02.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.559 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.559 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.559 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:30:02.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:30:02.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.559 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.559 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.559 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:30:02.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:30:02.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.559 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.559 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.559 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:30:02.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:30:02.560 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.560 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.560 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.560 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:30:02.560 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:30:02.560 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.560 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.560 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.560 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:30:02.560 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:30:02.560 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.560 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.561 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.561 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.561 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.561 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:30:02.561 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:02.561 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 184.89962846399999, 2023-12-06T18:30:02.561 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:30:02.561 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.561 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.561 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.561 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.561 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.561 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:02.561 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.561 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.562 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.562 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.562 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:02.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.562 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.562 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.562 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:30:02.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:30:02.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.562 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.562 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.562 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:30:02.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:30:02.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.563 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.563 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.563 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:30:02.563 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:30:02.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.563 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.563 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.563 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:30:02.563 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:30:02.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.563 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.564 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.564 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:30:02.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:30:02.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.564 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.564 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:30:02.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:30:02.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.564 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.564 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:30:02.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:30:02.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.565 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:30:02.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:30:02.565 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.565 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.565 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.565 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:30:02.565 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:02.565 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 184.87822785700001, 2023-12-06T18:30:02.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:30:02.565 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.565 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.565 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:02.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:02.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:30:02.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:30:02.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:30:02.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:30:02.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:30:02.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:30:02.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:30:02.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:30:02.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.568 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.568 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:30:02.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:30:02.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.568 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.568 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:30:02.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:30:02.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.568 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.568 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:30:02.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:30:02.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:30:02.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:30:02.569 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.569 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.569 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.569 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:30:02.569 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:02.569 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 184.87704853400001, 2023-12-06T18:30:02.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:30:02.569 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.569 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.570 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:02.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:02.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:30:02.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:30:02.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.571 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:30:02.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:30:02.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.571 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:30:02.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:30:02.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.571 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:30:02.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:30:02.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.572 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.572 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:30:02.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:30:02.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.572 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.572 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:30:02.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:30:02.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.572 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.572 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:30:02.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:30:02.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.573 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.573 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:30:02.573 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:30:02.573 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.573 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.573 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.573 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.573 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:30:02.573 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:02.573 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 184.876476553, 2023-12-06T18:30:02.573 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:30:02.573 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.573 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.573 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:02.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:02.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:30:02.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:30:02.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.575 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.575 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.575 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:30:02.575 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:30:02.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.575 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.575 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.575 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:30:02.575 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:30:02.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.575 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.575 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.576 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:30:02.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:30:02.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.576 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:30:02.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:30:02.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.576 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:30:02.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:30:02.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.576 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:30:02.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:30:02.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.577 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.577 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:30:02.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:30:02.577 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.577 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.577 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.577 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:30:02.577 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:02.578 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 184.43064108300001, 2023-12-06T18:30:02.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:30:02.578 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.578 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.578 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.578 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.578 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.578 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:02.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.578 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.578 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.578 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:02.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.579 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.579 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:30:02.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:30:02.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.579 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.579 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:30:02.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:30:02.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.579 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.579 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:30:02.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:30:02.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:30:02.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:30:02.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:30:02.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:30:02.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:30:02.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:30:02.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:30:02.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:30:02.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:30:02.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:30:02.581 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.581 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.581 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.581 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:30:02.582 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:02.582 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 184.429041227, 2023-12-06T18:30:02.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:30:02.582 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.582 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.582 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:02.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:02.583 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.583 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.583 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.583 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.583 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:30:02.583 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:30:02.583 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.583 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.583 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.583 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:30:02.583 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:30:02.583 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.583 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.584 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.584 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:30:02.584 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:30:02.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.584 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.584 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.584 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:30:02.584 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:30:02.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.584 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.584 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.584 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:30:02.584 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:30:02.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:30:02.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:30:02.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:30:02.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:30:02.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:30:02.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:30:02.586 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.586 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.586 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.586 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.586 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:30:02.586 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:02.586 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 184.04983561099999, 2023-12-06T18:30:02.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:30:02.586 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.586 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.586 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.586 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.586 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.587 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:02.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.587 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.587 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.587 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:02.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.587 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.587 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.587 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:30:02.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:30:02.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.587 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.587 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.588 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:30:02.588 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:30:02.588 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.588 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.588 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.588 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:30:02.588 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:30:02.588 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.588 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.588 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.588 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:30:02.588 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:30:02.588 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.588 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.588 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.589 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:30:02.589 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:30:02.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.589 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.589 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.589 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:30:02.589 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:30:02.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.589 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.589 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.589 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:30:02.589 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:30:02.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.589 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.589 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.589 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:30:02.590 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:30:02.590 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.590 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.590 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.590 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.590 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:30:02.590 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:02.590 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 183.81233342900001, 2023-12-06T18:30:02.590 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:30:02.590 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.591 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.591 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.591 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.591 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.591 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:02.591 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.591 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.591 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.592 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.592 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:02.592 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.592 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.592 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.592 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.592 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:30:02.592 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:30:02.592 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.593 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.593 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.593 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:30:02.593 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:30:02.593 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.593 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.594 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.594 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:30:02.594 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:30:02.594 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.594 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.594 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.595 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:30:02.595 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:30:02.595 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.595 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.595 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.595 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:30:02.595 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:30:02.596 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.596 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.596 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.596 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:30:02.596 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:30:02.596 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.596 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.596 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.596 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:30:02.597 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:30:02.597 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.597 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.597 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.597 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:30:02.597 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:30:02.597 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.597 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.598 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.598 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.598 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.598 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:30:02.598 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:02.598 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 183.81160542999999, 2023-12-06T18:30:02.599 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:30:02.599 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.599 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.599 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.599 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.599 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.599 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:02.600 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.600 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.600 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.600 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.600 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:02.600 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.600 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.600 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.600 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.600 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:30:02.601 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:30:02.601 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.601 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.601 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.601 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:30:02.601 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:30:02.601 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.601 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.601 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.601 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:30:02.601 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:30:02.601 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.601 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.601 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.601 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:30:02.602 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:30:02.602 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.602 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.602 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.602 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:30:02.602 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:30:02.602 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.602 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.602 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.602 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:30:02.602 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:30:02.602 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.602 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.602 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.603 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:30:02.603 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:30:02.603 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.603 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.603 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.603 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:30:02.603 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:30:02.603 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.603 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.603 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.603 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.603 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.603 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:30:02.603 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:02.603 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 183.80659302000001, 2023-12-06T18:30:02.603 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:30:02.604 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.604 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.604 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.604 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.604 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.604 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:02.604 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.604 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.604 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.604 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.604 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:02.604 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.604 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.604 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.604 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.605 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:30:02.605 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:30:02.605 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.605 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.605 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.605 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:30:02.605 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:30:02.605 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.605 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.605 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.605 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:30:02.605 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:30:02.605 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.605 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.605 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.606 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:30:02.606 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:30:02.606 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.606 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.606 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.606 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:30:02.606 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:30:02.606 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.606 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.606 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.606 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:30:02.606 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:30:02.607 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.607 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.607 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.607 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:30:02.607 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:30:02.607 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.607 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.607 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.607 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:30:02.608 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:30:02.608 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.608 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.608 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.608 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.608 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.608 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:30:02.608 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:02.608 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 183.80106594099999, 2023-12-06T18:30:02.608 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:30:02.608 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.608 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.608 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.608 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.608 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.609 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:02.609 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.609 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.609 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.609 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.609 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:02.609 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.609 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.609 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.609 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.609 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:30:02.609 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:30:02.609 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.609 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.609 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.610 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:30:02.610 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:30:02.610 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.610 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.610 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.610 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:30:02.610 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:30:02.610 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.610 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.610 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.610 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:30:02.610 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:30:02.610 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.610 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.610 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.611 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:30:02.611 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:30:02.611 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.611 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.611 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.611 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:30:02.611 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:30:02.611 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.611 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.611 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.611 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:30:02.611 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:30:02.611 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.612 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.612 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.612 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:30:02.612 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:30:02.612 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.612 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.612 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.612 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.612 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.612 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:30:02.612 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:02.612 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 183.80055510400001, 2023-12-06T18:30:02.612 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:30:02.612 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.612 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.613 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.613 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.613 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.613 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:02.613 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.613 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.613 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.613 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:02.613 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.613 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.613 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.613 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:30:02.613 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:30:02.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.614 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.614 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.614 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:30:02.614 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:30:02.614 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.614 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.614 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.614 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:30:02.614 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:30:02.614 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.614 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.614 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.614 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:30:02.614 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:30:02.614 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.615 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.615 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.615 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:30:02.615 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:30:02.615 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.615 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.615 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.615 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:30:02.615 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:30:02.615 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.615 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.615 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.615 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:30:02.615 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:30:02.615 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.616 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.616 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.616 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:30:02.616 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:30:02.616 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.616 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.616 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.616 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.616 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.616 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:30:02.616 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:02.617 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 183.80009584300001, 2023-12-06T18:30:02.617 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:30:02.617 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.617 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.617 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.617 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.617 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.617 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:02.618 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.618 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.618 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.618 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.618 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:02.618 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.618 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.619 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.619 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.619 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:30:02.619 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:30:02.619 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.619 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.619 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.620 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:30:02.620 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:30:02.620 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.620 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.620 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.620 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:30:02.620 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:30:02.621 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.621 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.621 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.621 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:30:02.621 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:30:02.621 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.621 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.621 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.622 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:30:02.622 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:30:02.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.622 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.622 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.622 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:30:02.622 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:30:02.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:30:02.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:30:02.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:30:02.624 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:30:02.624 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.624 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.624 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.624 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.625 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:30:02.625 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:02.625 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 177.22018003599999, 2023-12-06T18:30:02.625 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:30:02.625 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.625 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.625 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.626 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.626 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:02.626 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.626 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.626 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:02.626 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.627 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.627 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:30:02.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:30:02.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.627 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.627 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:30:02.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:30:02.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.627 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:30:02.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:30:02.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:30:02.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:30:02.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:30:02.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:30:02.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:30:02.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:30:02.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.629 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.629 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:30:02.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:30:02.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.629 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.629 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:30:02.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:30:02.629 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.629 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.629 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.630 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:30:02.630 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:02.630 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 177.213942231, 2023-12-06T18:30:02.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:30:02.630 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.630 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.630 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:02.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:02.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:30:02.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:30:02.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.632 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.632 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:30:02.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:30:02.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.632 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.632 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:30:02.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:30:02.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.632 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.632 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:30:02.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:30:02.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.633 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.633 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:30:02.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:30:02.633 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.633 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.633 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:30:02.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:30:02.633 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.633 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.633 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:30:02.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:30:02.633 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:30:02.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:30:02.634 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.634 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.634 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.634 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:30:02.634 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:02.634 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 177.207879152, 2023-12-06T18:30:02.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:30:02.634 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.634 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.635 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:02.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:02.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:30:02.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:30:02.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:30:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:30:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:30:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:30:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:30:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:30:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.637 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.637 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:30:02.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:30:02.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.637 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.637 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.637 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:30:02.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:30:02.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.637 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.637 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.637 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:30:02.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:30:02.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.637 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.638 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:30:02.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:30:02.638 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.638 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.638 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.638 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:30:02.638 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:02.638 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 177.070492067, 2023-12-06T18:30:02.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:30:02.638 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.638 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.639 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:02.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:02.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:30:02.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:30:02.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:30:02.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:30:02.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:30:02.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:30:02.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:30:02.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:30:02.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:30:02.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:30:02.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:30:02.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:30:02.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.643 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:30:02.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:30:02.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.643 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:30:02.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:30:02.643 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.643 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.643 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.643 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:30:02.643 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:02.643 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 177.06382134, 2023-12-06T18:30:02.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:30:02.644 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:02.644 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:02.644 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:02.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:02.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:02.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:02.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:02.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:02.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:02.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:30:02.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:30:02.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:02.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:30:02.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:30:02.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:02.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:30:02.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:30:02.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:02.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:30:02.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:30:02.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:02.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:30:02.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:30:02.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:02.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:30:02.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:30:02.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:02.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:30:02.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:30:02.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:02.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:02.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:02.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:30:02.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:30:02.647 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.647 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.647 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.647 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:02.647 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:02.647 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:02.649 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:30:02.649 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:02.645+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:02.751 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:02.747+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:02.852 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:02.848+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:02.868 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:02.869 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:02.869 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:02.869 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:02.881 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:30:02.955 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:02.951+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:03.011 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:03.011 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:03.011 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:03.011 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:03.011 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:03.020 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:30:03.067 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:03.064+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:03.095 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:30:03.161 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:03.158+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:03.184 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:03.184 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:03.184 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:03.184 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:30:03.184 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:03.194 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:30:03.258 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:03.255+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:03.359 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:03.356+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:03.391 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:30:03.392 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:30:03.393 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:03.393 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:03.406 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:30:03.456 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:20.641 106652== Warning: unimplemented fcntl command: 1036 2023-12-06T18:30:03.460 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:03.456+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:03.486 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:30:03.486 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:30:03.522 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:20.707 106652== Warning: unimplemented fcntl command: 1036 2023-12-06T18:30:03.550 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:03.534+0000 12a58640 -1 osd.7 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-06T18:30:03.558 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:20.750 101796== Warning: unimplemented fcntl command: 1036 2023-12-06T18:30:03.584 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:03.580+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:03.612 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:03.612 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:03.613 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:30:03.613 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:30:03.624 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:20.816 101796== Warning: unimplemented fcntl command: 1036 2023-12-06T18:30:03.626 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:30:03.669 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:03.664+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:03.723 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:20.909 106652== Warning: unimplemented fcntl command: 1036 2023-12-06T18:30:03.730 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:30:03.730 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:30:03.765 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:03.760+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:03.822 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:30:03.822 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:30:03.827 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:03.827 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:03.830 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:21.023 101796== Warning: unimplemented fcntl command: 1036 2023-12-06T18:30:03.836 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:30:03.862 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:03.859+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:03.928 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:30:03.929 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:30:03.964 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:03.960+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:04.016 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:04.016 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:04.024 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:30:04.024 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:30:04.025 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:30:04.070 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:04.066+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:04.077 INFO:teuthology.orchestra.run.smithi061.stderr:Error ENXIO: problem getting command descriptions from osd.7 2023-12-06T18:30:04.082 DEBUG:teuthology.orchestra.run:got remote process result: 6 2023-12-06T18:30:04.082 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2023-12-06T18:30:04.123 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-12-06T18:30:04.167 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:04.163+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:04.190 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:04.190 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:04.204 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:30:04.289 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:04.285+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:04.368 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:04.364+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:04.386 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:04.386 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:04.395 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:30:04.480 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:04.476+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:04.570 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:04.566+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:04.573 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:04.573 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:04.583 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:30:04.671 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:04.667+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:04.765 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:04.765 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:04.772 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:04.768+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:04.775 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:30:04.873 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:04.869+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:04.953 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:04.953 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:04.963 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:30:05.004 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:05.000+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:05.075 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:05.071+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:05.159 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:05.160 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:05.170 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:30:05.214 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:05.210+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:05.231 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is unset 2023-12-06T18:30:05.244 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:30:05.291 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:05.287+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:05.333 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:05.333 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:05.347 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:30:05.376 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:05.373+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:05.478 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:05.474+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:05.525 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:05.525 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:05.534 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:30:05.579 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:05.575+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:05.669 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:05.669 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:05.680 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:30:05.680 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:05.676+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:05.782 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:05.778+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:05.808 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:05.808 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:05.818 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:30:05.882 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:05.878+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:05.943 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":30,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":6,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:30:05.964 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:30:05.988 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:05.988 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:05.989 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:05.985+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:05.998 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:30:06.083 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:30:06.084 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:06.080+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:06.113 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:06.113 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:06.114 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:06.114 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:06.123 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-12-06T18:30:06.184 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:06.181+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:06.191 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:06.191 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:06.201 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:30:06.285 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:06.281+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:06.342 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:06.342 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:06.355 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:30:06.402 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:06.397+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:06.486 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:06.482+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:06.497 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:06.497 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:06.507 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:30:06.586 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:06.583+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:06.633 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:06.633 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:06.642 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:30:06.687 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:06.683+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:06.773 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:06.774 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:06.782 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:30:06.788 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:06.784+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:06.889 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:06.885+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:06.910 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:06.910 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:06.919 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:30:06.990 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:06.987+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:07.047 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:07.047 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:07.047 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:07.048 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:07.056 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:30:07.101 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:07.097+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:07.114 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.0 to 1 (10000) 2023-12-06T18:30:07.127 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-12-06T18:30:07.212 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:07.208+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:07.215 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:07.215 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:07.215 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:07.215 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:07.215 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:07.225 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:30:07.250 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-12-06T18:30:07.294 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:07.291+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:07.394 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:07.391+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:07.430 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:07.430 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:07.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:07.430 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:30:07.430 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:07.439 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:30:07.520 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:07.514+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:07.577 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:07.577 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:07.586 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:30:07.596 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:07.593+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:07.697 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:07.694+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:07.739 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:07.739 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:07.752 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:30:07.836 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:07.832+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:07.899 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:07.895+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:07.936 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:07.936 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:07.946 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:30:07.999 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:07.996+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:08.101 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:08.097+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:08.126 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:08.126 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:08.136 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:30:08.220 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:08.216+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:08.264 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is set 2023-12-06T18:30:08.265 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.1 to 1 (10000) 2023-12-06T18:30:08.282 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-12-06T18:30:08.332 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:08.332 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:08.345 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:30:08.367 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:08.363+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:08.415 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:25.600 106652== Warning: unimplemented fcntl command: 1036 2023-12-06T18:30:08.416 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:08.412+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:08.422 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:25.607 106652== Warning: unimplemented fcntl command: 1036 2023-12-06T18:30:08.452 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:25.637 106652== Warning: unimplemented fcntl command: 1036 2023-12-06T18:30:08.462 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:25.654 101796== Warning: unimplemented fcntl command: 1036 2023-12-06T18:30:08.468 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:25.660 101796== Warning: unimplemented fcntl command: 1036 2023-12-06T18:30:08.485 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:25.678 101796== Warning: unimplemented fcntl command: 1036 2023-12-06T18:30:08.504 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:08.500+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:08.560 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:08.561 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:08.573 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:30:08.614 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:08.610+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:08.706 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:08.703+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:08.762 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:08.763 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:08.771 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:30:08.814 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:08.809+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:08.952 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:08.953 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:08.959 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:30:08.962 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:30:08.999 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:08.995+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:09.009 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:09.006+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:09.134 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:09.130+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:09.148 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:09.148 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:09.157 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:30:09.210 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:09.207+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:09.312 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:09.308+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:09.335 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:09.335 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:09.344 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:30:09.412 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:09.409+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:09.514 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:09.514 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:09.515 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:09.511+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:09.524 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:30:09.616 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:09.612+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:09.680 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":30,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":9,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:30:09.711 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:09.711 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:09.720 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:30:09.741 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:30:09.742 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:30:09.742 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:09.738+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:09.818 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:09.814+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:09.834 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:30:09.834 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:30:09.850 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:09.850 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:09.859 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:30:09.918 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:09.915+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:09.926 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:30:09.926 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:30:09.987 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:09.987 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:09.996 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:30:10.017 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:30:10.017 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:30:10.019 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:10.016+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:10.106 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:30:10.106 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:30:10.120 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:10.117+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:10.147 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:10.148 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:10.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:10.148 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:10.148 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.148 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:10.148 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:10.148 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 108.65447491, 2023-12-06T18:30:10.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:30:10.148 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.148 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:10.148 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:10.148 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:10.149 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:10.149 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:30:10.149 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.149 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.149 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.149 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.149 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:10.149 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.149 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.149 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.149 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.149 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:10.149 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.150 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.150 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:30:10.150 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:30:10.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.150 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.150 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:30:10.150 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:30:10.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.150 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.150 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:30:10.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:30:10.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.151 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.151 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.151 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:30:10.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:30:10.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.151 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.151 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.151 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:30:10.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:30:10.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.151 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.151 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.151 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:30:10.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:30:10.152 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.152 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.152 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.152 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.152 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.152 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:10.157 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:30:10.196 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:30:10.196 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:30:10.243 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:10.239+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:10.282 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-12-06T18:30:10.285 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:30:10.285 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:30:10.321 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:10.318+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:10.337 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:10.337 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:10.337 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:10.337 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:10.341 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:08.772 107953== Warning: unimplemented fcntl command: 1036 2023-12-06T18:30:10.346 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:30:10.422 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:10.418+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:10.449 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:30:10.449 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:30:10.523 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:10.519+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:10.535 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.2 to 1 (10000) 2023-12-06T18:30:10.537 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:10.537 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:10.537 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:10.537 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:10.537 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:10.547 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:30:10.549 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-12-06T18:30:10.594 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:09.025 107953== Warning: unimplemented fcntl command: 1036 2023-12-06T18:30:10.624 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:10.620+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:10.725 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:10.721+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:10.732 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:10.732 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:10.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:10.732 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:10.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.732 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:30:10.732 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:10.732 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 194.114782416, 2023-12-06T18:30:10.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:30:10.732 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.732 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.732 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:10.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:10.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:30:10.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:30:10.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:30:10.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:30:10.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:30:10.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:30:10.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:30:10.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:30:10.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:30:10.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:30:10.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:30:10.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:30:10.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:30:10.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:30:10.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:30:10.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:30:10.736 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.736 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.736 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.736 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:30:10.736 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:10.736 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 193.091955407, 2023-12-06T18:30:10.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:30:10.736 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.736 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.736 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:10.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:10.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:30:10.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:30:10.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:30:10.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:30:10.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:30:10.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:30:10.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:30:10.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:30:10.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:30:10.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:30:10.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:30:10.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:30:10.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:30:10.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:30:10.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:30:10.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:30:10.742 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.742 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.742 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.743 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:30:10.743 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:10.743 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 193.07990876900001, 2023-12-06T18:30:10.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:30:10.743 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.743 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.743 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:10.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:10.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:30:10.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:30:10.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:30:10.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:30:10.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:30:10.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:30:10.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:30:10.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:30:10.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:30:10.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:30:10.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:30:10.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:30:10.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:30:10.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:30:10.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:30:10.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:30:10.749 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.749 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.749 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.749 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:30:10.750 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:10.750 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 193.05850816200001, 2023-12-06T18:30:10.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:30:10.750 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.750 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.750 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:10.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:10.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:30:10.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:30:10.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:30:10.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:30:10.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:30:10.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:30:10.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:30:10.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:30:10.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:30:10.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:30:10.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:30:10.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:30:10.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:30:10.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:30:10.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:30:10.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:30:10.754 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.754 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.754 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.754 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:30:10.754 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:10.754 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 193.05732883900001, 2023-12-06T18:30:10.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:30:10.755 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.755 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.755 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:10.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:10.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:30:10.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:30:10.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:30:10.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:30:10.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:30:10.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:30:10.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:30:10.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:30:10.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:30:10.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:30:10.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:30:10.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:30:10.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:30:10.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:30:10.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:30:10.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:30:10.758 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.758 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.758 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.759 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:30:10.759 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:10.759 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 193.056756858, 2023-12-06T18:30:10.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:30:10.759 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.759 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.759 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:10.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.760 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:10.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.760 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:30:10.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:30:10.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.760 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:30:10.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:30:10.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:30:10.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:30:10.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:30:10.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:30:10.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:30:10.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:30:10.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:30:10.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:30:10.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:30:10.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:30:10.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:30:10.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:30:10.763 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.763 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.763 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.763 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:30:10.763 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:10.763 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 192.61092138800001, 2023-12-06T18:30:10.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:30:10.763 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.763 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.763 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:10.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:10.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:30:10.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:30:10.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:30:10.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:30:10.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:30:10.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:30:10.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:30:10.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:30:10.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:30:10.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:30:10.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:30:10.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:30:10.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:30:10.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:30:10.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:30:10.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:30:10.767 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.767 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.767 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.767 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:30:10.767 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:10.767 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 192.609321532, 2023-12-06T18:30:10.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:30:10.767 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.767 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.767 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:10.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.768 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.768 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:10.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.768 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.768 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:30:10.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:30:10.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:30:10.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:30:10.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:30:10.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:30:10.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:30:10.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:30:10.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:30:10.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:30:10.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:30:10.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:30:10.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:30:10.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:30:10.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:30:10.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:30:10.771 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.771 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.771 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.771 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:30:10.771 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:10.771 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 192.23011591599999, 2023-12-06T18:30:10.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:30:10.772 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.772 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.772 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:10.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:10.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:30:10.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:30:10.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:30:10.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:30:10.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:30:10.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:30:10.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:30:10.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:30:10.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:30:10.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:30:10.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:30:10.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:30:10.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:30:10.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:30:10.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:30:10.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:30:10.775 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.775 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.775 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.776 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:30:10.776 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:10.776 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 191.992613734, 2023-12-06T18:30:10.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:30:10.776 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.776 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.776 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:10.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:10.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:30:10.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:30:10.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:30:10.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:30:10.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:30:10.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:30:10.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:30:10.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:30:10.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:30:10.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:30:10.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:30:10.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:30:10.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:30:10.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:30:10.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:30:10.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:30:10.779 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.779 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.780 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.780 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:30:10.780 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:10.780 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 191.99188573500001, 2023-12-06T18:30:10.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:30:10.780 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.780 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.780 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:10.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:10.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:30:10.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:30:10.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:30:10.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:30:10.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:30:10.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:30:10.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:30:10.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:30:10.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:30:10.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:30:10.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:30:10.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:30:10.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:30:10.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:30:10.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:30:10.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:30:10.784 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.784 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.784 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.784 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:30:10.784 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:10.784 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 191.986873325, 2023-12-06T18:30:10.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:30:10.784 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.784 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.784 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:10.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:10.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:30:10.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:30:10.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:30:10.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:30:10.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:30:10.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:30:10.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:30:10.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:30:10.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:30:10.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:30:10.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:30:10.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:30:10.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:30:10.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:30:10.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:30:10.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:30:10.788 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.788 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.788 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.788 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:30:10.788 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:10.788 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 191.98134624599999, 2023-12-06T18:30:10.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:30:10.788 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.788 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.788 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:10.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:10.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:30:10.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:30:10.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:30:10.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:30:10.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:30:10.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:30:10.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:30:10.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:30:10.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:30:10.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:30:10.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:30:10.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:30:10.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:30:10.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:30:10.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:30:10.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:30:10.792 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.792 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.792 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.792 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:30:10.792 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:10.792 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 191.98083540900001, 2023-12-06T18:30:10.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:30:10.792 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.792 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.792 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:10.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:10.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:30:10.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:30:10.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:30:10.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:30:10.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:30:10.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:30:10.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:30:10.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:30:10.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:30:10.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:30:10.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:30:10.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:30:10.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:30:10.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:30:10.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:30:10.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:30:10.796 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.796 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.796 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.796 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:30:10.796 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:10.796 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 191.980376148, 2023-12-06T18:30:10.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:30:10.796 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.796 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.796 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:10.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:10.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:30:10.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:30:10.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:30:10.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:30:10.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:30:10.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:30:10.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:30:10.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:30:10.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:30:10.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:30:10.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:30:10.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:30:10.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:30:10.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:30:10.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:30:10.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:30:10.800 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.800 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.800 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.800 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:30:10.800 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:10.800 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 185.40046034100001, 2023-12-06T18:30:10.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:30:10.800 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.800 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.800 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:10.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:10.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:30:10.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:30:10.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:30:10.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:30:10.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.802 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.802 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:30:10.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:30:10.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.802 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.802 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:30:10.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:30:10.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.802 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.802 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:30:10.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:30:10.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:30:10.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:30:10.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:30:10.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:30:10.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:30:10.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:30:10.803 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.804 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.804 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.804 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:30:10.804 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:10.804 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 185.394222536, 2023-12-06T18:30:10.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:30:10.804 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.804 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.804 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:10.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:10.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:30:10.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:30:10.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:30:10.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:30:10.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:30:10.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:30:10.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:30:10.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:30:10.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:30:10.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:30:10.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:30:10.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:30:10.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:30:10.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:30:10.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:30:10.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:30:10.807 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.808 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.808 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.809 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:30:10.809 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:10.809 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 185.388159457, 2023-12-06T18:30:10.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:30:10.809 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.809 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.809 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:10.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:10.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:30:10.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:30:10.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:30:10.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:30:10.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:30:10.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:30:10.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:30:10.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:30:10.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:30:10.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:30:10.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.812 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.812 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:30:10.812 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:30:10.812 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.812 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.812 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.812 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:30:10.812 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:30:10.812 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.812 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.812 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.812 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:30:10.812 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:30:10.812 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.812 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.813 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.813 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.813 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.813 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:30:10.813 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:10.813 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 185.250772372, 2023-12-06T18:30:10.813 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:30:10.813 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.813 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.813 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.813 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.813 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.813 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:10.813 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.814 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.814 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.814 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.814 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:10.814 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.814 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.814 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.814 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.814 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:30:10.814 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:30:10.814 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.814 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.814 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.814 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:30:10.814 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:30:10.815 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.815 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.815 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.815 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:30:10.815 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:30:10.815 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.815 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.815 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.815 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:30:10.815 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:30:10.815 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.815 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.815 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.815 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:30:10.815 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:30:10.816 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.816 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.816 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.816 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:30:10.816 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:30:10.816 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.816 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.816 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.816 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:30:10.816 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:30:10.816 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.816 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.816 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.816 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:30:10.817 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:30:10.817 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.817 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.817 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.817 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.817 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.817 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:30:10.817 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:10.817 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 185.244101645, 2023-12-06T18:30:10.817 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:30:10.817 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:10.817 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:10.817 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:10.817 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.818 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:10.818 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:10.818 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.818 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.818 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.818 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:10.818 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:10.818 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:10.818 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.818 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.818 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:10.818 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:30:10.818 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:30:10.818 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:10.819 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:30:10.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:30:10.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:10.819 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:30:10.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:30:10.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:10.819 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:30:10.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:30:10.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.820 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:10.820 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:30:10.820 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:30:10.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.820 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:10.820 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:30:10.820 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:30:10.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.820 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:10.820 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:30:10.820 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:30:10.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:10.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:10.821 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:10.821 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:30:10.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:30:10.821 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.821 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.821 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.821 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:10.821 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:10.821 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:10.822 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:30:10.826 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:10.822+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:10.926 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:10.923+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:10.993 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:10.994 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:10.994 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:10.994 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:11.003 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:30:11.028 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:11.024+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:11.129 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:11.125+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:11.173 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:11.173 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:11.182 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:30:11.267 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:11.263+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:11.330 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:11.326+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:11.352 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:11.352 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:11.352 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:11.352 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:30:11.352 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:11.361 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:30:11.446 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:11.442+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:11.531 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:11.528+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:11.534 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:11.534 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:11.534 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:11.534 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:11.544 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:30:11.633 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:11.629+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:11.719 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:11.719 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:11.719 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:11.719 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:11.719 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:11.728 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:30:11.734 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:11.730+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:11.741 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.3 to 1 (10000) 2023-12-06T18:30:11.749 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is set 2023-12-06T18:30:11.756 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2023-12-06T18:30:11.847 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:11.844+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:11.861 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:11.861 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:11.862 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:11.862 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:30:11.862 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:11.870 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:30:11.935 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:11.932+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:12.012 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:12.012 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:12.012 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:12.012 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:12.022 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:30:12.036 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:12.032+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:12.137 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:12.133+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:12.201 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:12.201 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:12.201 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:12.201 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:12.201 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:12.210 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:30:12.237 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:12.234+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:12.338 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:12.335+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:12.390 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:12.390 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:12.391 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:12.391 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:30:12.391 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:12.398 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:30:12.443 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:12.439+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:12.462 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:30:12.541 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:12.537+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:12.571 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:12.571 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:12.571 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:12.571 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:12.584 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:30:12.642 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:12.638+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:12.699 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:30:12.743 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:12.744 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:12.744 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:12.744 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:12.744 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:12.750 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.4 to 1 (10000) 2023-12-06T18:30:12.756 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:30:12.766 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2023-12-06T18:30:12.790 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:30:12.791 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:30:12.791 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:12.787+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:12.843 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:12.840+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:12.899 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:30:12.900 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:30:12.944 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:12.941+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:12.975 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:12.976 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:12.988 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:30:13.040 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:30:13.040 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:30:13.046 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:13.042+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:13.147 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:13.143+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:13.162 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:13.162 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:13.167 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:30:13.168 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:30:13.173 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:30:13.257 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:30:13.258 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:30:13.258 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:13.254+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:13.349 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:13.345+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:13.349 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:30:13.349 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:30:13.355 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:13.355 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:13.367 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:30:13.438 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:30:13.439 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:30:13.449 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:13.446+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:13.524 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":30,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":13,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:30:13.540 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:30:13.540 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:30:13.551 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:13.547+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:13.567 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:13.567 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:13.576 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:30:13.651 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:13.648+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:13.752 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:13.749+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:13.760 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:13.760 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:13.771 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:30:13.771 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-12-06T18:30:13.854 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:13.850+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:13.955 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:13.951+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:13.959 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:13.959 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:13.968 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:30:14.056 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:14.052+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:14.077 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:12.508 107953== Warning: unimplemented fcntl command: 1036 2023-12-06T18:30:14.136 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:12.567 107953== Warning: unimplemented fcntl command: 1036 2023-12-06T18:30:14.160 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:14.156+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:14.172 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:14.173 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:14.182 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:30:14.259 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:14.255+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:14.313 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:14.314 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:14.322 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:30:14.359 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:14.356+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:14.376 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:12.807 107953== Warning: unimplemented fcntl command: 1036 2023-12-06T18:30:14.460 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:14.456+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:14.471 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.6 to 1 (10000) 2023-12-06T18:30:14.485 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2023-12-06T18:30:14.500 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:14.500 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:14.509 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:30:14.571 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:14.567+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:14.648 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:14.648 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:14.659 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:30:14.661 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:14.657+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:14.762 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:14.759+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:14.805 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:14.806 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:14.815 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:30:14.863 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:14.859+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:14.964 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:14.960+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:15.001 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:15.001 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:15.011 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:30:15.065 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:15.061+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:15.141 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:15.142 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:15.151 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:30:15.193 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:15.189+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:15.268 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:15.264+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:15.325 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:15.325 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:15.333 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:30:15.368 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:15.364+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:15.468 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:15.465+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:15.504 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:15.504 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:15.513 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:30:15.553 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:30:15.594 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is unset 2023-12-06T18:30:15.594 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:15.591+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:15.603 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.7 to 1 (10000) 2023-12-06T18:30:15.619 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2023-12-06T18:30:15.694 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:15.695 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:15.704 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:30:15.704 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:15.700+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:15.771 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:15.767+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:15.865 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:15.865 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:15.871 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:15.867+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:15.872 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:30:15.872 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:30:15.877 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:30:15.964 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:30:15.964 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:30:15.972 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:15.968+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:16.007 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:16.007 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:16.016 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:30:16.054 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:30:16.055 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:30:16.072 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:16.069+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:16.151 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:30:16.151 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:30:16.173 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:16.169+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:16.189 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:30:16.189 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:30:16.198 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:30:16.239 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:30:16.240 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:30:16.275 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:16.271+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:16.327 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:30:16.328 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:30:16.341 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:16.341 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:16.341 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:16.341 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:16.351 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:30:16.394 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:16.389+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:16.413 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:30:16.413 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:30:16.476 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:16.472+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:16.515 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:30:16.516 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:30:16.541 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:16.541 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:16.541 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:16.541 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:16.541 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:16.541 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:30:16.550 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:30:16.577 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:16.574+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:16.679 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:16.675+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:16.695 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.5 to 1 (10000) 2023-12-06T18:30:16.711 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-12-06T18:30:16.711 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:30:16.745 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:16.745 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:16.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:16.745 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:30:16.746 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:16.755 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:30:16.780 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:16.776+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:16.880 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:16.877+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:16.951 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:16.951 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:16.952 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:16.952 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:16.962 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:30:16.981 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:16.977+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:17.082 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:17.078+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:17.125 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:17.126 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:17.126 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:17.126 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:17.126 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:17.136 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:30:17.221 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:17.217+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:17.283 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:17.279+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:17.334 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":32,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"c","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:30:17.351 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2023-12-06T18:30:17.351 INFO:tasks.mon_recovery:stopping all monitors 2023-12-06T18:30:17.351 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-12-06T18:30:17.351 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:30:17.365 INFO:tasks.ceph.mon.b.smithi152.stderr:2023-12-06T18:30:17.352+0000 9f54640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 101794) UID: 0 2023-12-06T18:30:17.366 INFO:tasks.ceph.mon.b.smithi152.stderr:2023-12-06T18:30:17.353+0000 9f54640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2023-12-06T18:30:17.383 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:17.380+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:17.417 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:17.417 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:17.417 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:17.417 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:17.418 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.418 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:30:17.418 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:17.418 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 202.00439711499999, 2023-12-06T18:30:17.418 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:30:17.418 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.418 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.418 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.418 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.418 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.418 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:30:17.419 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.419 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.419 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.419 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.419 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:17.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.419 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.419 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.419 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.419 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:17.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.419 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.419 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.419 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.419 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:30:17.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:30:17.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:30:17.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:30:17.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:30:17.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:30:17.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:30:17.421 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:30:17.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.421 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.421 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.421 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:30:17.421 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:30:17.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.421 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.421 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:30:17.421 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:30:17.421 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:30:17.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.421 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.421 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.422 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:30:17.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:30:17.422 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.422 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.422 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.422 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:30:17.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:30:17.422 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.422 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.422 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.422 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:30:17.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:30:17.422 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.422 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.423 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.423 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:30:17.423 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:30:17.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.423 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.423 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:30:17.423 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:30:17.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.423 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.423 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:30:17.423 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:30:17.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.423 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.424 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:30:17.424 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:30:17.424 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.424 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.424 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.424 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.424 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:17.424 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:17.424 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 199.89582387600001, 2023-12-06T18:30:17.424 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:30:17.424 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.424 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.424 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.424 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.424 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.425 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:30:17.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.425 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:17.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:17.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:30:17.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:30:17.426 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.426 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.426 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.426 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:30:17.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:30:17.426 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.426 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.426 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.426 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:30:17.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:30:17.426 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.426 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.426 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.427 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:30:17.427 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:30:17.427 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.427 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.427 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.427 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:30:17.427 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:30:17.427 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.427 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.427 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.427 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:30:17.427 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:30:17.427 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.427 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.427 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.427 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:30:17.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:30:17.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.428 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.428 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.428 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:30:17.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:30:17.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.428 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.428 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.428 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:30:17.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:30:17.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.428 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.428 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.428 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:30:17.429 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:30:17.429 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.429 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.429 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.429 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:30:17.429 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:30:17.429 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.429 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.429 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.429 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.429 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.429 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:17.429 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:17.429 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 199.89507864399999, 2023-12-06T18:30:17.429 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:30:17.430 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.430 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.430 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.430 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.430 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.430 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:30:17.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.430 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.430 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.430 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.430 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:17.430 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.431 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.431 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.431 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.431 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:17.431 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.431 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.431 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.431 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.431 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:30:17.431 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:30:17.431 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.431 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.431 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.431 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:30:17.431 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:30:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:30:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:30:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:30:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:30:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:30:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:30:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.433 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.433 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:30:17.433 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:30:17.433 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.433 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.433 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.433 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:30:17.433 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:30:17.433 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.433 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.433 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.433 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:30:17.433 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:30:17.433 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.433 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.433 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.434 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:30:17.434 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:30:17.434 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.434 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.434 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.434 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:30:17.434 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:30:17.434 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.434 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.434 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.434 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:30:17.434 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:30:17.434 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.434 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.434 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.434 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.435 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.435 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:17.435 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:17.435 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 199.89439488900001, 2023-12-06T18:30:17.435 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:30:17.435 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.435 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.435 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.435 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.435 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.435 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:30:17.435 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.435 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.435 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.435 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.435 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:17.435 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.436 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.436 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.436 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:17.436 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.436 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.436 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.436 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:30:17.436 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:30:17.436 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.436 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.436 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:30:17.436 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:30:17.436 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.437 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.437 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:30:17.437 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:30:17.437 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.437 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.437 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.437 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:30:17.437 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:30:17.437 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.437 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.437 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.437 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:30:17.437 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:30:17.437 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.437 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.437 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.437 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:30:17.438 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:30:17.438 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.438 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.438 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.438 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:30:17.438 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:30:17.438 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.438 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.438 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.438 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:30:17.438 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:30:17.438 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.438 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.438 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.439 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:30:17.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:30:17.439 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.439 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.439 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.439 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:30:17.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:30:17.439 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.439 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.439 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.439 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:30:17.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:30:17.439 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.439 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.440 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.440 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.440 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.440 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:17.440 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:17.440 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 199.88882163400001, 2023-12-06T18:30:17.440 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:30:17.440 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.440 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.440 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.440 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.440 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.440 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:30:17.440 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.440 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:17.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.441 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:17.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.441 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:30:17.442 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:30:17.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.442 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.442 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.442 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:30:17.442 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:30:17.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.442 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.442 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.442 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:30:17.442 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:30:17.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.442 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.442 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.443 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:30:17.443 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:30:17.443 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.443 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.443 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.443 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:30:17.443 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:30:17.443 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.443 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.443 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.443 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:30:17.443 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:30:17.443 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.443 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.443 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.443 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:30:17.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:30:17.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.444 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.444 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:30:17.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:30:17.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.444 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.444 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:30:17.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:30:17.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.444 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.445 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:30:17.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:30:17.445 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.445 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.445 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.445 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:30:17.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:30:17.445 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.445 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.445 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.445 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.445 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.445 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:17.445 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:17.445 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 199.88814799400001, 2023-12-06T18:30:17.446 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:30:17.446 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.446 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.446 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.446 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.446 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.446 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:30:17.446 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.446 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.446 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.446 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:17.446 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.446 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.447 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.447 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.447 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:17.447 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.447 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.447 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.447 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.447 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:30:17.447 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:30:17.447 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.447 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.447 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.448 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:30:17.448 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:30:17.448 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.448 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.448 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.448 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:30:17.448 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:30:17.448 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.448 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.448 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.448 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:30:17.448 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:30:17.448 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.448 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:30:17.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:30:17.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:30:17.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:30:17.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:30:17.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:30:17.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.450 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.450 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:30:17.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:30:17.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.450 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.450 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:30:17.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:30:17.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.450 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.450 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:30:17.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:30:17.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.451 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.451 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:30:17.451 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:30:17.451 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.451 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.451 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.451 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:17.451 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:17.451 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 199.887485795, 2023-12-06T18:30:17.451 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:30:17.451 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.451 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.451 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.452 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.452 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.452 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:30:17.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.452 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:17.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:17.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.453 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.453 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.453 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:30:17.453 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:30:17.453 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.453 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.453 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.453 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:30:17.453 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:30:17.453 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.453 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.453 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.453 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:30:17.453 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:30:17.453 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.453 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.454 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.454 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:30:17.454 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:30:17.454 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.454 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.454 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.454 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:30:17.454 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:30:17.454 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.454 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.454 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.454 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:30:17.454 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:30:17.454 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.454 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:30:17.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:30:17.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:30:17.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:30:17.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:30:17.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:30:17.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.456 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:30:17.456 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:30:17.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.456 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:30:17.456 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:30:17.456 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.456 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.456 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.456 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:17.456 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:17.457 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 199.81477636, 2023-12-06T18:30:17.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:30:17.457 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.457 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.457 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.457 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.457 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.457 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:30:17.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.457 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:17.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.458 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.458 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.458 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:17.458 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.458 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.458 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.458 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.458 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:30:17.458 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:30:17.458 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.458 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.458 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.458 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:30:17.458 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:30:17.458 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.459 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.459 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.459 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:30:17.459 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:30:17.459 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.459 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.459 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.459 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:30:17.459 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:30:17.459 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.459 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.459 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.459 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:30:17.459 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:30:17.459 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.459 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:30:17.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:30:17.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:30:17.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:30:17.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:30:17.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:30:17.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:30:17.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:30:17.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:30:17.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:30:17.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:30:17.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:30:17.461 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.461 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.462 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.462 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:17.462 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:17.462 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 199.814094651, 2023-12-06T18:30:17.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:30:17.462 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.462 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.462 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.462 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.462 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.462 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:30:17.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.463 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:17.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:17.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:30:17.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:30:17.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.464 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.464 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:30:17.464 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:30:17.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.464 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.464 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.464 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:30:17.464 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:30:17.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.464 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.464 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.464 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:30:17.464 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:30:17.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:30:17.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:30:17.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:30:17.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:30:17.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:30:17.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:30:17.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:30:17.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:30:17.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:30:17.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:30:17.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:30:17.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:30:17.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:30:17.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:30:17.467 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.467 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.467 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.467 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:17.467 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:17.467 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 198.73529056300001, 2023-12-06T18:30:17.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:30:17.467 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.467 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.468 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.468 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.468 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.468 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:30:17.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.468 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:17.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:17.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:30:17.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:30:17.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:30:17.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:30:17.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:30:17.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:30:17.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:30:17.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:30:17.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:30:17.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:30:17.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:30:17.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:30:17.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:30:17.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:30:17.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:30:17.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:30:17.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:30:17.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:30:17.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:30:17.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:30:17.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:30:17.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:30:17.472 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.472 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.472 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.472 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:17.472 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:17.473 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 198.73387398899999, 2023-12-06T18:30:17.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:30:17.473 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.473 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.473 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.473 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.473 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.473 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:30:17.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.473 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:17.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:17.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:30:17.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:30:17.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:30:17.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:30:17.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:30:17.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:30:17.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:30:17.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:30:17.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:30:17.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:30:17.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:30:17.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:30:17.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:30:17.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:30:17.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:30:17.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:30:17.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:30:17.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:30:17.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:30:17.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:30:17.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:30:17.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:30:17.477 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.478 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.478 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.478 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:17.478 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:17.478 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 198.73278886400001, 2023-12-06T18:30:17.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:30:17.478 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.478 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.478 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.478 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.478 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.478 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:30:17.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.479 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:17.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:17.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:30:17.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:30:17.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:30:17.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:30:17.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:30:17.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:30:17.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:30:17.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:30:17.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:30:17.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:30:17.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:30:17.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:30:17.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:30:17.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:30:17.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:30:17.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:30:17.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:30:17.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:30:17.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:30:17.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:30:17.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:30:17.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:30:17.483 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.483 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.483 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.484 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.484 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:17.484 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:17.484 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 198.69763976199999, 2023-12-06T18:30:17.484 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:30:17.484 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.484 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.484 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.484 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.484 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.484 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:30:17.484 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.484 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.485 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.485 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:17.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.485 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.485 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:17.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.485 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.485 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:30:17.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:30:17.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:30:17.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:30:17.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:30:17.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:30:17.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:30:17.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:30:17.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:30:17.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:30:17.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:30:17.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:30:17.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:30:17.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:30:17.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:30:17.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:30:17.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:30:17.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:30:17.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:30:17.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:30:17.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:30:17.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:30:17.489 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.489 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.489 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.490 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:17.490 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:17.490 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 192.14790923699999, 2023-12-06T18:30:17.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:30:17.490 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.490 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.490 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.490 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.490 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.490 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:30:17.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.491 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:17.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:17.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:30:17.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:30:17.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:30:17.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:30:17.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:30:17.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:30:17.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:30:17.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:30:17.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:30:17.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:30:17.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:30:17.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:30:17.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:30:17.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:30:17.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:30:17.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:30:17.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:30:17.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:30:17.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:30:17.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:30:17.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:30:17.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:30:17.495 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.496 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.496 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.496 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:17.496 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:17.496 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 192.14710362899999, 2023-12-06T18:30:17.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:30:17.496 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.496 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.496 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.496 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.496 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.496 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:30:17.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.497 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:17.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:17.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:30:17.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:30:17.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:30:17.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:30:17.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:30:17.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:30:17.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:30:17.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:30:17.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:30:17.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:30:17.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:30:17.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:30:17.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:30:17.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:30:17.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:30:17.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:30:17.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:30:17.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:30:17.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:30:17.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:30:17.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:30:17.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:30:17.503 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.503 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.503 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.503 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:17.503 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:17.503 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 192.14641281999999, 2023-12-06T18:30:17.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:30:17.503 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.503 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.504 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.504 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.504 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.504 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:30:17.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.504 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:17.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:17.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:30:17.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:30:17.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:30:17.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:30:17.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:30:17.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:30:17.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:30:17.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:30:17.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:30:17.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:30:17.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:30:17.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:30:17.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:30:17.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:30:17.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:30:17.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:30:17.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:30:17.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:30:17.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:30:17.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:30:17.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:30:17.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:30:17.508 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.508 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.508 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.508 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:17.508 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:17.509 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 192.14211531199999, 2023-12-06T18:30:17.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:30:17.509 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.509 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.509 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.509 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.509 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.509 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:30:17.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.509 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:17.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.510 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.510 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:17.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.510 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.510 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:30:17.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:30:17.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.510 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.510 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:30:17.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:30:17.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:30:17.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:30:17.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:30:17.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:30:17.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:30:17.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:30:17.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:30:17.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:30:17.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:30:17.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:30:17.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:30:17.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:30:17.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:30:17.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:30:17.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:30:17.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:30:17.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:30:17.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:30:17.514 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.514 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.514 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.514 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:17.514 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:17.514 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 192.141447637, 2023-12-06T18:30:17.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:30:17.514 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.514 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.514 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.514 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.514 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.514 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:30:17.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.515 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:17.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:17.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:30:17.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:30:17.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:30:17.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:30:17.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:30:17.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:30:17.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:30:17.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:30:17.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:30:17.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:30:17.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:30:17.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:30:17.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:30:17.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:30:17.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:30:17.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:30:17.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:30:17.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:30:17.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:30:17.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:30:17.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:30:17.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:30:17.519 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.519 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.519 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.519 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:17.519 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:17.519 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 192.14079220900001, 2023-12-06T18:30:17.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:30:17.520 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.520 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.520 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.520 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.520 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.520 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:30:17.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.520 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:17.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:17.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:30:17.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:30:17.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:30:17.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:30:17.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.522 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:30:17.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:30:17.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.522 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.522 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:30:17.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:30:17.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.522 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.522 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:30:17.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:30:17.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:30:17.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:30:17.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:30:17.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:30:17.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:30:17.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:30:17.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:30:17.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:30:17.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:30:17.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:30:17.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:30:17.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:30:17.524 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.525 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.525 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.525 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:17.525 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:17.525 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 192.10013766200001, 2023-12-06T18:30:17.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:30:17.525 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:17.525 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:17.525 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:17.525 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:17.525 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:17.525 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:30:17.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.526 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:17.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:17.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:17.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:17.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:17.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:17.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:17.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:30:17.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:30:17.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:17.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:30:17.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:30:17.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:17.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:30:17.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:30:17.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:17.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:30:17.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:30:17.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:17.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:30:17.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:30:17.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:17.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:30:17.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:30:17.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:17.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:30:17.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:30:17.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:17.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:30:17.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:30:17.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:17.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:30:17.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:30:17.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:17.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:30:17.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:30:17.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:17.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:17.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:17.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:30:17.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:30:17.530 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.530 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.530 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.530 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:17.530 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:17.530 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:17.532 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:30:17.533 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:30:17.533 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:30:17.535 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2023-12-06T18:30:16.647210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226772,"kb_used_data":5212,"kb_used_omap":15,"kb_used_meta":217008,"kb_avail":754747948,"statfs":{"total":773094113280,"available":772861898752,"internally_reserved":0,"allocated":5337088,"data_stored":2714839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15557,"internal_metadata":222217019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":146,"apply_latency_ms":146,"commit_latency_ns":146000000,"apply_latency_ns":146000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.398444"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-06T18:30:08.687970+0000","last_change":"2023-12-06T18:29:09.051293+0000","last_active":"2023-12-06T18:30:08.687970+0000","last_peered":"2023-12-06T18:30:08.687970+0000","last_clean":"2023-12-06T18:30:08.687970+0000","last_became_active":"2023-12-06T18:29:09.045331+0000","last_became_peered":"2023-12-06T18:29:09.045331+0000","last_unstale":"2023-12-06T18:30:08.687970+0000","last_undegraded":"2023-12-06T18:30:08.687970+0000","last_fullsized":"2023-12-06T18:30:08.687970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T02:46:01.854508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":50,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-06T18:30:08.224980+0000","last_change":"2023-12-06T18:27:04.883840+0000","last_active":"2023-12-06T18:30:08.224980+0000","last_peered":"2023-12-06T18:30:08.224980+0000","last_clean":"2023-12-06T18:30:08.224980+0000","last_became_active":"2023-12-06T18:27:04.858001+0000","last_became_peered":"2023-12-06T18:27:04.858001+0000","last_unstale":"2023-12-06T18:30:08.224980+0000","last_undegraded":"2023-12-06T18:30:08.224980+0000","last_fullsized":"2023-12-06T18:30:08.224980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:44:02.916244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-12-06T18:30:07.284987+0000","last_change":"2023-12-06T18:27:04.336778+0000","last_active":"2023-12-06T18:30:07.284987+0000","last_peered":"2023-12-06T18:30:07.284987+0000","last_clean":"2023-12-06T18:30:07.284987+0000","last_became_active":"2023-12-06T18:27:04.321269+0000","last_became_peered":"2023-12-06T18:27:04.321269+0000","last_unstale":"2023-12-06T18:30:07.284987+0000","last_undegraded":"2023-12-06T18:30:07.284987+0000","last_fullsized":"2023-12-06T18:30:07.284987+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T22:25:15.703579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-06T18:30:08.676926+0000","last_change":"2023-12-06T18:28:47.898016+0000","last_active":"2023-12-06T18:30:08.676926+0000","last_peered":"2023-12-06T18:30:08.676926+0000","last_clean":"2023-12-06T18:30:08.676926+0000","last_became_active":"2023-12-06T18:28:47.891814+0000","last_became_peered":"2023-12-06T18:28:47.891814+0000","last_unstale":"2023-12-06T18:30:08.676926+0000","last_undegraded":"2023-12-06T18:30:08.676926+0000","last_fullsized":"2023-12-06T18:30:08.676926+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T00:49:06.937273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-12-06T18:30:07.277692+0000","last_change":"2023-12-06T18:27:04.408081+0000","last_active":"2023-12-06T18:30:07.277692+0000","last_peered":"2023-12-06T18:30:07.277692+0000","last_clean":"2023-12-06T18:30:07.277692+0000","last_became_active":"2023-12-06T18:27:04.399765+0000","last_became_peered":"2023-12-06T18:27:04.399765+0000","last_unstale":"2023-12-06T18:30:07.277692+0000","last_undegraded":"2023-12-06T18:30:07.277692+0000","last_fullsized":"2023-12-06T18:30:07.277692+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:40:33.570114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-06T18:30:08.762262+0000","last_change":"2023-12-06T18:30:08.762262+0000","last_active":"2023-12-06T18:30:08.762262+0000","last_peered":"2023-12-06T18:30:08.762262+0000","last_clean":"2023-12-06T18:30:08.762262+0000","last_became_active":"2023-12-06T18:30:08.733421+0000","last_became_peered":"2023-12-06T18:30:08.733421+0000","last_unstale":"2023-12-06T18:30:08.762262+0000","last_undegraded":"2023-12-06T18:30:08.762262+0000","last_fullsized":"2023-12-06T18:30:08.762262+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:43:31.796397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":65,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-12-06T18:30:07.275897+0000","last_change":"2023-12-06T18:28:47.876556+0000","last_active":"2023-12-06T18:30:07.275897+0000","last_peered":"2023-12-06T18:30:07.275897+0000","last_clean":"2023-12-06T18:30:07.275897+0000","last_became_active":"2023-12-06T18:28:47.865169+0000","last_became_peered":"2023-12-06T18:28:47.865169+0000","last_unstale":"2023-12-06T18:30:07.275897+0000","last_undegraded":"2023-12-06T18:30:07.275897+0000","last_fullsized":"2023-12-06T18:30:07.275897+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:16:40.095542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-12-06T18:30:07.281930+0000","last_change":"2023-12-06T18:27:04.609381+0000","last_active":"2023-12-06T18:30:07.281930+0000","last_peered":"2023-12-06T18:30:07.281930+0000","last_clean":"2023-12-06T18:30:07.281930+0000","last_became_active":"2023-12-06T18:27:04.598701+0000","last_became_peered":"2023-12-06T18:27:04.598701+0000","last_unstale":"2023-12-06T18:30:07.281930+0000","last_undegraded":"2023-12-06T18:30:07.281930+0000","last_fullsized":"2023-12-06T18:30:07.281930+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:29:58.962692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":189,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-12-06T18:30:14.034503+0000","last_change":"2023-12-06T18:28:48.207134+0000","last_active":"2023-12-06T18:30:14.034503+0000","last_peered":"2023-12-06T18:30:14.034503+0000","last_clean":"2023-12-06T18:30:14.034503+0000","last_became_active":"2023-12-06T18:28:47.875258+0000","last_became_peered":"2023-12-06T18:28:47.875258+0000","last_unstale":"2023-12-06T18:30:14.034503+0000","last_undegraded":"2023-12-06T18:30:14.034503+0000","last_fullsized":"2023-12-06T18:30:14.034503+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:14:32.667039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1204224,"data_stored":1193520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411370,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":598016,"data_stored":242922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":13.292,"5min":10.651,"15min":10.521000000000001},"min":{"1min":6.0229999999999997,"5min":5.4009999999999998,"15min":5.4009999999999998},"max":{"1min":108.57599999999999,"5min":108.57599999999999,"15min":108.57599999999999},"last":10.488},{"interface":"front","average":{"1min":14.936,"5min":12.337,"15min":12.619},"min":{"1min":5.2030000000000003,"5min":3.7200000000000002,"15min":3.7200000000000002},"max":{"1min":128.09399999999999,"5min":128.09399999999999,"15min":128.09399999999999},"last":9.1509999999999998}]},{"osd":1,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":12.457000000000001,"5min":12.502000000000001,"15min":12.871},"min":{"1min":6.2539999999999996,"5min":6.2539999999999996,"15min":6.2539999999999996},"max":{"1min":70.072000000000003,"5min":70.072000000000003,"15min":70.072000000000003},"last":7.9809999999999999},{"interface":"front","average":{"1min":12.776,"5min":14.673,"15min":15.680999999999999},"min":{"1min":6.444,"5min":6.444,"15min":6.444},"max":{"1min":27.934000000000001,"5min":32.920000000000002,"15min":32.920000000000002},"last":12.253}]},{"osd":2,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":15.362,"5min":14.803000000000001,"15min":14.759},"min":{"1min":8.5649999999999995,"5min":7.5149999999999997,"15min":7.5149999999999997},"max":{"1min":75.125,"5min":75.125,"15min":75.125},"last":24.446000000000002},{"interface":"front","average":{"1min":17.251000000000001,"5min":17.606000000000002,"15min":18.052},"min":{"1min":9.6590000000000007,"5min":8.8290000000000006,"15min":8.8290000000000006},"max":{"1min":69.403999999999996,"5min":69.403999999999996,"15min":69.403999999999996},"last":25.609000000000002}]},{"osd":4,"last update":"Wed Dec 6 18:30:04 2023","interfaces":[{"interface":"back","average":{"1min":10.486000000000001,"5min":11.387,"15min":11.696999999999999},"min":{"1min":5.8879999999999999,"5min":5.8879999999999999,"15min":5.8879999999999999},"max":{"1min":21.064,"5min":21.064,"15min":21.064},"last":27.518000000000001},{"interface":"front","average":{"1min":16.756,"5min":15.946,"15min":16.045999999999999},"min":{"1min":6.3099999999999996,"5min":6.3099999999999996,"15min":6.3099999999999996},"max":{"1min":71.718000000000004,"5min":71.718000000000004,"15min":71.718000000000004},"last":13.378}]},{"osd":5,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":18.776,"5min":18.231999999999999,"15min":17.882000000000001},"min":{"1min":10.853999999999999,"5min":8.3179999999999996,"15min":8.3179999999999996},"max":{"1min":75.751999999999995,"5min":75.751999999999995,"15min":75.751999999999995},"last":33.335999999999999},{"interface":"front","average":{"1min":16.117000000000001,"5min":17.902000000000001,"15min":18.187000000000001},"min":{"1min":7.9269999999999996,"5min":7.9269999999999996,"15min":7.9269999999999996},"max":{"1min":70.933000000000007,"5min":70.933000000000007,"15min":70.933000000000007},"last":34.899999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":13.388,"5min":15.525,"15min":16.145},"min":{"1min":8.2050000000000001,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":28.111999999999998,"5min":29.245000000000001,"15min":29.245000000000001},"last":14.470000000000001},{"interface":"front","average":{"1min":20.686,"5min":20.766999999999999,"15min":21.321000000000002},"min":{"1min":12.061,"5min":10.973000000000001,"15min":10.973000000000001},"max":{"1min":74.224000000000004,"5min":74.224000000000004,"15min":74.224000000000004},"last":36.704999999999998}]},{"osd":7,"last update":"Wed Dec 6 18:27:59 2023","interfaces":[{"interface":"back","average":{"1min":20.715,"5min":20.715,"15min":20.715},"min":{"1min":12.949999999999999,"5min":12.949999999999999,"15min":12.949999999999999},"max":{"1min":34.771000000000001,"5min":34.771000000000001,"15min":34.771000000000001},"last":122.93000000000001},{"interface":"front","average":{"1min":19.077999999999999,"5min":19.077999999999999,"15min":19.077999999999999},"min":{"1min":12.787000000000001,"5min":12.787000000000001,"15min":12.787000000000001},"max":{"1min":32.286000000000001,"5min":32.286000000000001,"15min":32.286000000000001},"last":124.473}]}]},{"osd":2,"up_from":16,"seq":68719476778,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":536576,"data_stored":236938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:30:04 2023","interfaces":[{"interface":"back","average":{"1min":8.673,"5min":10.326000000000001,"15min":11.119999999999999},"min":{"1min":4.9989999999999997,"5min":4.609,"15min":4.609},"max":{"1min":24.404,"5min":38.740000000000002,"15min":38.740000000000002},"last":14.67},{"interface":"front","average":{"1min":11.497,"5min":10.827999999999999,"15min":11.194000000000001},"min":{"1min":4.6550000000000002,"5min":4.6550000000000002,"15min":4.6550000000000002},"max":{"1min":33.387,"5min":33.387,"15min":33.387},"last":9.7319999999999993}]},{"osd":1,"last update":"Wed Dec 6 18:30:04 2023","interfaces":[{"interface":"back","average":{"1min":13.208,"5min":11.56,"15min":11.606999999999999},"min":{"1min":5.9139999999999997,"5min":5.9139999999999997,"15min":5.9139999999999997},"max":{"1min":42.920999999999999,"5min":42.920999999999999,"15min":42.920999999999999},"last":9.1440000000000001},{"interface":"front","average":{"1min":13.545,"5min":13.532999999999999,"15min":13.851000000000001},"min":{"1min":7.4930000000000003,"5min":6.6260000000000003,"15min":6.6260000000000003},"max":{"1min":34.607999999999997,"5min":34.607999999999997,"15min":34.607999999999997},"last":15.356}]},{"osd":3,"last update":"Wed Dec 6 18:30:04 2023","interfaces":[{"interface":"back","average":{"1min":17.064,"5min":17.663,"15min":19.145},"min":{"1min":9.5180000000000007,"5min":6.1260000000000003,"15min":6.1260000000000003},"max":{"1min":49.609000000000002,"5min":112.129,"15min":112.129},"last":12.630000000000001},{"interface":"front","average":{"1min":17.344000000000001,"5min":21.335999999999999,"15min":23.93},"min":{"1min":8.4559999999999995,"5min":7.5510000000000002,"15min":7.5510000000000002},"max":{"1min":61.243000000000002,"5min":181.601,"15min":181.601},"last":16.608000000000001}]},{"osd":4,"last update":"Wed Dec 6 18:30:04 2023","interfaces":[{"interface":"back","average":{"1min":15.673,"5min":16.632000000000001,"15min":17.838000000000001},"min":{"1min":7.8070000000000004,"5min":6.609,"15min":6.609},"max":{"1min":60.332000000000001,"5min":97.088999999999999,"15min":97.088999999999999},"last":23.422000000000001},{"interface":"front","average":{"1min":14.141,"5min":18.652999999999999,"15min":20.876999999999999},"min":{"1min":6.3559999999999999,"5min":6.3559999999999999,"15min":6.3559999999999999},"max":{"1min":30.734999999999999,"5min":98.069000000000003,"15min":98.069000000000003},"last":8.0199999999999996}]},{"osd":5,"last update":"Wed Dec 6 18:30:04 2023","interfaces":[{"interface":"back","average":{"1min":17.09,"5min":17.640999999999998,"15min":18.077000000000002},"min":{"1min":9.8819999999999997,"5min":7.8170000000000002,"15min":7.8170000000000002},"max":{"1min":57.628999999999998,"5min":57.628999999999998,"15min":57.628999999999998},"last":22.111000000000001},{"interface":"front","average":{"1min":15.912000000000001,"5min":15.712999999999999,"15min":15.797000000000001},"min":{"1min":9.9930000000000003,"5min":8.843,"15min":8.843},"max":{"1min":47.749000000000002,"5min":47.749000000000002,"15min":47.749000000000002},"last":11.444000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:30:04 2023","interfaces":[{"interface":"back","average":{"1min":18.800000000000001,"5min":35.606000000000002,"15min":43.548999999999999},"min":{"1min":11.016,"5min":8.7289999999999992,"15min":8.7289999999999992},"max":{"1min":64.034999999999997,"5min":498.38499999999999,"15min":498.38499999999999},"last":18.835000000000001},{"interface":"front","average":{"1min":17.834,"5min":37.069000000000003,"15min":45.436999999999998},"min":{"1min":9.7230000000000008,"5min":9.7230000000000008,"15min":9.7230000000000008},"max":{"1min":54.920000000000002,"5min":496.17399999999998,"15min":496.17399999999998},"last":25.861000000000001}]},{"osd":7,"last update":"Wed Dec 6 18:27:57 2023","interfaces":[{"interface":"back","average":{"1min":27.364999999999998,"5min":27.364999999999998,"15min":27.364999999999998},"min":{"1min":12.321,"5min":12.321,"15min":12.321},"max":{"1min":182.93600000000001,"5min":182.93600000000001,"15min":182.93600000000001},"last":20.274000000000001},{"interface":"front","average":{"1min":45.905000000000001,"5min":45.905000000000001,"15min":45.905000000000001},"min":{"1min":13.724,"5min":13.724,"15min":13.724},"max":{"1min":497.67700000000002,"5min":497.67700000000002,"15min":497.67700000000002},"last":26.888000000000002}]}]},{"osd":1,"up_from":16,"seq":68719476779,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":950272,"data_stored":639137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":9.5990000000000002,"5min":18.391999999999999,"15min":22.372},"min":{"1min":6.0579999999999998,"5min":5.5700000000000003,"15min":5.5700000000000003},"max":{"1min":12.356999999999999,"5min":238.87100000000001,"15min":238.87100000000001},"last":18.282},{"interface":"front","average":{"1min":9.6799999999999997,"5min":34.107999999999997,"15min":44.859999999999999},"min":{"1min":5.5369999999999999,"5min":4.4269999999999996,"15min":4.4269999999999996},"max":{"1min":26.919,"5min":661.33600000000001,"15min":661.33600000000001},"last":14.526999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":9.5739999999999998,"5min":26.436,"15min":33.494},"min":{"1min":6.8040000000000003,"5min":6.6340000000000003,"15min":6.6340000000000003},"max":{"1min":15.69,"5min":384.315,"15min":384.315},"last":25.260000000000002},{"interface":"front","average":{"1min":14.73,"5min":37.917000000000002,"15min":48.116999999999997},"min":{"1min":10.743,"5min":5.7009999999999996,"15min":5.7009999999999996},"max":{"1min":18.289000000000001,"5min":656.98699999999997,"15min":656.98699999999997},"last":26.274999999999999}]},{"osd":3,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":11.050000000000001,"5min":12.286,"15min":13.162000000000001},"min":{"1min":5.9720000000000004,"5min":5.548,"15min":5.548},"max":{"1min":26.821000000000002,"5min":34.057000000000002,"15min":34.057000000000002},"last":21.591999999999999},{"interface":"front","average":{"1min":12.624000000000001,"5min":14.897,"15min":16.113},"min":{"1min":6.5730000000000004,"5min":6.5730000000000004,"15min":6.5730000000000004},"max":{"1min":24.925999999999998,"5min":35.832999999999998,"15min":35.832999999999998},"last":23.164000000000001}]},{"osd":4,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":11.384,"5min":32.529000000000003,"15min":41.786999999999999},"min":{"1min":6.7009999999999996,"5min":6.2930000000000001,"15min":6.2930000000000001},"max":{"1min":22.437999999999999,"5min":447.67200000000003,"15min":447.67200000000003},"last":16.388999999999999},{"interface":"front","average":{"1min":15.162000000000001,"5min":39.052,"15min":49.468000000000004},"min":{"1min":9.6129999999999995,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":24,"5min":630.572,"15min":630.572},"last":20.324999999999999}]},{"osd":5,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":15.984999999999999,"5min":35.731999999999999,"15min":44.042000000000002},"min":{"1min":8.9689999999999994,"5min":8.9689999999999994,"15min":8.9689999999999994},"max":{"1min":23.050000000000001,"5min":389.55000000000001,"15min":389.55000000000001},"last":24.338999999999999},{"interface":"front","average":{"1min":13.598000000000001,"5min":44.341000000000001,"15min":58.034999999999997},"min":{"1min":7.5359999999999996,"5min":7.0410000000000004,"15min":7.0410000000000004},"max":{"1min":26.456,"5min":658.48599999999999,"15min":658.48599999999999},"last":10.609999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":14.804,"5min":29.702999999999999,"15min":36.640000000000001},"min":{"1min":9.6579999999999995,"5min":8.4710000000000001,"15min":8.4710000000000001},"max":{"1min":31.664999999999999,"5min":439.56400000000002,"15min":439.56400000000002},"last":13.683999999999999},{"interface":"front","average":{"1min":17.984000000000002,"5min":31.940000000000001,"15min":37.909999999999997},"min":{"1min":10.641999999999999,"5min":10.641999999999999,"15min":10.641999999999999},"max":{"1min":33.786999999999999,"5min":386.71499999999997,"15min":386.71499999999997},"last":27.262}]},{"osd":7,"last update":"Wed Dec 6 18:27:58 2023","interfaces":[{"interface":"back","average":{"1min":43.825000000000003,"5min":43.825000000000003,"15min":43.825000000000003},"min":{"1min":11.693,"5min":11.693,"15min":11.693},"max":{"1min":431.46100000000001,"5min":431.46100000000001,"15min":431.46100000000001},"last":12.829000000000001},{"interface":"front","average":{"1min":42.889000000000003,"5min":42.889000000000003,"15min":42.889000000000003},"min":{"1min":11.731999999999999,"5min":11.731999999999999,"15min":11.731999999999999},"max":{"1min":388.52999999999997,"5min":388.52999999999997,"15min":388.52999999999997},"last":19.300999999999998}]}]},{"osd":0,"up_from":16,"seq":68719476780,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":548864,"data_stored":241297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 6 18:30:01 2023","interfaces":[{"interface":"back","average":{"1min":9.7509999999999994,"5min":15.065,"15min":17.385999999999999},"min":{"1min":3.2360000000000002,"5min":3.2360000000000002,"15min":3.2360000000000002},"max":{"1min":47.661000000000001,"5min":116.593,"15min":116.593},"last":8.7330000000000005},{"interface":"front","average":{"1min":12.177,"5min":16.544,"15min":18.117000000000001},"min":{"1min":4.9139999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"max":{"1min":50.127000000000002,"5min":117.896,"15min":117.896},"last":19.241}]},{"osd":2,"last update":"Wed Dec 6 18:30:01 2023","interfaces":[{"interface":"back","average":{"1min":13.339,"5min":14.113,"15min":13.630000000000001},"min":{"1min":5.694,"5min":5.694,"15min":5.694},"max":{"1min":43.709000000000003,"5min":43.752000000000002,"15min":43.752000000000002},"last":17.036999999999999},{"interface":"front","average":{"1min":11.170999999999999,"5min":12.009,"15min":11.318},"min":{"1min":6.2880000000000003,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":27.573,"5min":46.387999999999998,"15min":46.387999999999998},"last":10.082000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:30:07 2023","interfaces":[{"interface":"back","average":{"1min":13.433,"5min":15.228999999999999,"15min":16.306000000000001},"min":{"1min":6.1379999999999999,"5min":6.1379999999999999,"15min":6.1379999999999999},"max":{"1min":29.677,"5min":46.689999999999998,"15min":46.689999999999998},"last":13.933},{"interface":"front","average":{"1min":16.504999999999999,"5min":16.332999999999998,"15min":16.297000000000001},"min":{"1min":8.532,"5min":8.0329999999999995,"15min":8.0329999999999995},"max":{"1min":32.801000000000002,"5min":50.545999999999999,"15min":50.545999999999999},"last":11.106}]},{"osd":4,"last update":"Wed Dec 6 18:30:01 2023","interfaces":[{"interface":"back","average":{"1min":9.3989999999999991,"5min":17.760999999999999,"15min":20.173999999999999},"min":{"1min":5.9349999999999996,"5min":5.9349999999999996,"15min":5.9349999999999996},"max":{"1min":14.582000000000001,"5min":82.432000000000002,"15min":82.432000000000002},"last":12.483000000000001},{"interface":"front","average":{"1min":15.801,"5min":19.405000000000001,"15min":20.338000000000001},"min":{"1min":10.051,"5min":6.2629999999999999,"15min":6.2629999999999999},"max":{"1min":31.408000000000001,"5min":81.239999999999995,"15min":81.239999999999995},"last":21.504999999999999}]},{"osd":5,"last update":"Wed Dec 6 18:30:01 2023","interfaces":[{"interface":"back","average":{"1min":13.711,"5min":17.306000000000001,"15min":17.523},"min":{"1min":8.0099999999999998,"5min":8.0099999999999998,"15min":8.0099999999999998},"max":{"1min":25.613,"5min":71.367000000000004,"15min":71.367000000000004},"last":11.683},{"interface":"front","average":{"1min":13.176,"5min":17.669,"15min":18.024999999999999},"min":{"1min":8.5299999999999994,"5min":8.5299999999999994,"15min":8.5299999999999994},"max":{"1min":30.242000000000001,"5min":72.307000000000002,"15min":72.307000000000002},"last":18.007999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:30:01 2023","interfaces":[{"interface":"back","average":{"1min":17.878,"5min":19.167999999999999,"15min":19.061},"min":{"1min":11.423,"5min":7.7560000000000002,"15min":7.7560000000000002},"max":{"1min":29.899000000000001,"5min":62.932000000000002,"15min":62.932000000000002},"last":20.074999999999999},{"interface":"front","average":{"1min":15.651999999999999,"5min":18.478999999999999,"15min":19.190000000000001},"min":{"1min":9.6940000000000008,"5min":9.6940000000000008,"15min":9.6940000000000008},"max":{"1min":28.785,"5min":52.627000000000002,"15min":52.627000000000002},"last":15.988}]},{"osd":7,"last update":"Wed Dec 6 18:27:55 2023","interfaces":[{"interface":"back","average":{"1min":21.66,"5min":21.66,"15min":21.66},"min":{"1min":12.804,"5min":12.804,"15min":12.804},"max":{"1min":48.781999999999996,"5min":48.781999999999996,"15min":48.781999999999996},"last":22.117999999999999},{"interface":"front","average":{"1min":19.440000000000001,"5min":19.440000000000001,"15min":19.440000000000001},"min":{"1min":10.537000000000001,"5min":10.537000000000001,"15min":10.537000000000001},"max":{"1min":38.530000000000001,"5min":38.530000000000001,"15min":38.530000000000001},"last":14.666}]}]},{"osd":5,"up_from":15,"seq":64424509484,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":999424,"data_stored":640762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:30:03 2023","interfaces":[{"interface":"back","average":{"1min":7.0629999999999997,"5min":13.085000000000001,"15min":15.403},"min":{"1min":3.3660000000000001,"5min":3.3660000000000001,"15min":3.3660000000000001},"max":{"1min":19.173999999999999,"5min":208.291,"15min":208.291},"last":13.097},{"interface":"front","average":{"1min":10.037000000000001,"5min":16.419,"15min":19.463000000000001},"min":{"1min":4.758,"5min":4.4299999999999997,"15min":4.4299999999999997},"max":{"1min":15.025,"5min":199.94900000000001,"15min":199.94900000000001},"last":8.8040000000000003}]},{"osd":1,"last update":"Wed Dec 6 18:30:03 2023","interfaces":[{"interface":"back","average":{"1min":13.615,"5min":45.186999999999998,"15min":59.460000000000001},"min":{"1min":8.5809999999999995,"5min":8.1180000000000003,"15min":8.1180000000000003},"max":{"1min":41.234999999999999,"5min":1211.6010000000001,"15min":1211.6010000000001},"last":10.135999999999999},{"interface":"front","average":{"1min":10.398,"5min":40.906999999999996,"15min":54.581000000000003},"min":{"1min":5.3049999999999997,"5min":5.1340000000000003,"15min":5.1340000000000003},"max":{"1min":39.802999999999997,"5min":1189.3869999999999,"15min":1189.3869999999999},"last":9.5719999999999992}]},{"osd":2,"last update":"Wed Dec 6 18:30:03 2023","interfaces":[{"interface":"back","average":{"1min":14.023,"5min":21.236999999999998,"15min":24.780999999999999},"min":{"1min":6.5129999999999999,"5min":6.5129999999999999,"15min":6.5129999999999999},"max":{"1min":41.716000000000001,"5min":283.46800000000002,"15min":283.46800000000002},"last":10.744999999999999},{"interface":"front","average":{"1min":17.048999999999999,"5min":24.818000000000001,"15min":28.687000000000001},"min":{"1min":12.295999999999999,"5min":8.9350000000000005,"15min":8.9350000000000005},"max":{"1min":42.981999999999999,"5min":296.47800000000001,"15min":296.47800000000001},"last":16.443000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:30:03 2023","interfaces":[{"interface":"back","average":{"1min":15.493,"5min":17.984000000000002,"15min":19.076000000000001},"min":{"1min":9.1739999999999995,"5min":9.1739999999999995,"15min":9.1739999999999995},"max":{"1min":21.536999999999999,"5min":47.926000000000002,"15min":47.926000000000002},"last":14.406000000000001},{"interface":"front","average":{"1min":11.621,"5min":13.949,"15min":15.118},"min":{"1min":5.3979999999999997,"5min":5.3979999999999997,"15min":5.3979999999999997},"max":{"1min":19.492999999999999,"5min":66.176000000000002,"15min":66.176000000000002},"last":17.27}]},{"osd":4,"last update":"Wed Dec 6 18:30:03 2023","interfaces":[{"interface":"back","average":{"1min":16.521000000000001,"5min":18.350000000000001,"15min":19.611999999999998},"min":{"1min":12.817,"5min":10.286,"15min":10.286},"max":{"1min":27.754999999999999,"5min":67.248000000000005,"15min":67.248000000000005},"last":12.250999999999999},{"interface":"front","average":{"1min":9.8469999999999995,"5min":11.958,"15min":13.016},"min":{"1min":5.3300000000000001,"5min":5.3300000000000001,"15min":5.3300000000000001},"max":{"1min":30.934999999999999,"5min":77.266999999999996,"15min":77.266999999999996},"last":6.6779999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:30:03 2023","interfaces":[{"interface":"back","average":{"1min":10.468,"5min":14.179,"15min":15.666},"min":{"1min":6.1280000000000001,"5min":5.915,"15min":5.915},"max":{"1min":18.870000000000001,"5min":76.751999999999995,"15min":76.751999999999995},"last":8.5120000000000005},{"interface":"front","average":{"1min":17.943000000000001,"5min":19.835999999999999,"15min":20.946000000000002},"min":{"1min":13.004,"5min":8.8390000000000004,"15min":8.8390000000000004},"max":{"1min":36.256,"5min":57.101999999999997,"15min":57.101999999999997},"last":18.149999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:27:55 2023","interfaces":[{"interface":"back","average":{"1min":16.204000000000001,"5min":16.204000000000001,"15min":16.204000000000001},"min":{"1min":7.367,"5min":7.367,"15min":7.367},"max":{"1min":70.768000000000001,"5min":70.768000000000001,"15min":70.768000000000001},"last":15.602},{"interface":"front","average":{"1min":18.081,"5min":18.081,"15min":18.081},"min":{"1min":6.3150000000000004,"5min":6.3150000000000004,"15min":6.3150000000000004},"max":{"1min":63.991999999999997,"5min":63.991999999999997,"15min":63.991999999999997},"last":13.712999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":242641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:30:00 2023","interfaces":[{"interface":"back","average":{"1min":9.2739999999999991,"5min":12.443,"15min":13.609999999999999},"min":{"1min":3.7429999999999999,"5min":3.7429999999999999,"15min":3.7429999999999999},"max":{"1min":41.076000000000001,"5min":53.298999999999999,"15min":53.298999999999999},"last":14.099},{"interface":"front","average":{"1min":9.0860000000000003,"5min":10.368,"15min":10.833},"min":{"1min":4.8140000000000001,"5min":3.9670000000000001,"15min":3.9670000000000001},"max":{"1min":35.863,"5min":52.567,"15min":52.567},"last":17.042000000000002}]},{"osd":1,"last update":"Wed Dec 6 18:30:00 2023","interfaces":[{"interface":"back","average":{"1min":8.8810000000000002,"5min":12.872999999999999,"15min":13.939},"min":{"1min":6.4640000000000004,"5min":5.835,"15min":5.835},"max":{"1min":13.583,"5min":49.183,"15min":49.183},"last":6.2869999999999999},{"interface":"front","average":{"1min":10.976000000000001,"5min":14.564,"15min":15.682},"min":{"1min":7.516,"5min":5.96,"15min":5.96},"max":{"1min":14.849,"5min":45.412999999999997,"15min":45.412999999999997},"last":10.346}]},{"osd":2,"last update":"Wed Dec 6 18:30:00 2023","interfaces":[{"interface":"back","average":{"1min":11.057,"5min":18.984999999999999,"15min":22.416},"min":{"1min":6.2549999999999999,"5min":5.6390000000000002,"15min":5.6390000000000002},"max":{"1min":16.393999999999998,"5min":85.394000000000005,"15min":85.394000000000005},"last":9.1590000000000007},{"interface":"front","average":{"1min":13.888,"5min":18.390999999999998,"15min":20.571999999999999},"min":{"1min":8.7560000000000002,"5min":6.3639999999999999,"15min":6.3639999999999999},"max":{"1min":27.651,"5min":69.530000000000001,"15min":69.530000000000001},"last":19.039000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:30:06 2023","interfaces":[{"interface":"back","average":{"1min":13.596,"5min":14.331,"15min":14.377000000000001},"min":{"1min":8.5359999999999996,"5min":7.008,"15min":7.008},"max":{"1min":26.219000000000001,"5min":26.219000000000001,"15min":26.219000000000001},"last":20.195},{"interface":"front","average":{"1min":13.808,"5min":14.66,"15min":15.273999999999999},"min":{"1min":5.5549999999999997,"5min":5.5549999999999997,"15min":5.5549999999999997},"max":{"1min":22.760999999999999,"5min":29.902999999999999,"15min":29.902999999999999},"last":11.901999999999999}]},{"osd":5,"last update":"Wed Dec 6 18:30:00 2023","interfaces":[{"interface":"back","average":{"1min":11.284000000000001,"5min":15.779999999999999,"15min":17.044},"min":{"1min":6.2430000000000003,"5min":6.125,"15min":6.125},"max":{"1min":23.835000000000001,"5min":78.798000000000002,"15min":78.798000000000002},"last":15.260999999999999},{"interface":"front","average":{"1min":15.913,"5min":17.991,"15min":18.765000000000001},"min":{"1min":8.0519999999999996,"5min":6.4450000000000003,"15min":6.4450000000000003},"max":{"1min":25.544,"5min":55.814999999999998,"15min":55.814999999999998},"last":23.238}]},{"osd":6,"last update":"Wed Dec 6 18:30:00 2023","interfaces":[{"interface":"back","average":{"1min":16.356000000000002,"5min":17.545000000000002,"15min":17.936},"min":{"1min":9.6489999999999991,"5min":7.7850000000000001,"15min":7.7850000000000001},"max":{"1min":32.491,"5min":51.149000000000001,"15min":51.149000000000001},"last":22.629000000000001},{"interface":"front","average":{"1min":15.821999999999999,"5min":16.190000000000001,"15min":16.683},"min":{"1min":8.5630000000000006,"5min":7.8700000000000001,"15min":7.8700000000000001},"max":{"1min":34.929000000000002,"5min":40.426000000000002,"15min":40.426000000000002},"last":12.615}]},{"osd":7,"last update":"Wed Dec 6 18:27:57 2023","interfaces":[{"interface":"back","average":{"1min":18.751999999999999,"5min":18.751999999999999,"15min":18.751999999999999},"min":{"1min":11.249000000000001,"5min":11.249000000000001,"15min":11.249000000000001},"max":{"1min":33.067999999999998,"5min":33.067999999999998,"15min":33.067999999999998},"last":18.355},{"interface":"front","average":{"1min":17.401,"5min":17.401,"15min":17.401},"min":{"1min":7.6050000000000004,"5min":7.6050000000000004,"15min":7.6050000000000004},"max":{"1min":32.356999999999999,"5min":32.356999999999999,"15min":32.356999999999999},"last":20.920999999999999}]}]},{"osd":7,"up_from":61,"seq":261993005059,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":584,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":598016,"data_stored":242922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4395,"internal_metadata":28634837},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":59,"apply_latency_ms":59,"commit_latency_ns":59000000,"apply_latency_ns":59000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.2510000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.403}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.428999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.808}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.839}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.263999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.036}]}]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":512000,"data_stored":228220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:29:58 2023","interfaces":[{"interface":"back","average":{"1min":8.3689999999999998,"5min":9.6289999999999996,"15min":9.9359999999999999},"min":{"1min":5.4539999999999997,"5min":3.8519999999999999,"15min":3.8519999999999999},"max":{"1min":14.390000000000001,"5min":46.947000000000003,"15min":46.947000000000003},"last":9.8230000000000004},{"interface":"front","average":{"1min":8.3010000000000002,"5min":10.358000000000001,"15min":10.140000000000001},"min":{"1min":4.0650000000000004,"5min":4.0650000000000004,"15min":4.0650000000000004},"max":{"1min":19.611999999999998,"5min":45.953000000000003,"15min":45.953000000000003},"last":12.472}]},{"osd":1,"last update":"Wed Dec 6 18:29:58 2023","interfaces":[{"interface":"back","average":{"1min":9.7309999999999999,"5min":11.683999999999999,"15min":12.542},"min":{"1min":4.3760000000000003,"5min":4.3760000000000003,"15min":4.3760000000000003},"max":{"1min":66.313000000000002,"5min":66.313000000000002,"15min":66.313000000000002},"last":7.468},{"interface":"front","average":{"1min":14.108000000000001,"5min":14.211,"15min":14.603},"min":{"1min":7.1269999999999998,"5min":5.8540000000000001,"15min":5.8540000000000001},"max":{"1min":67.533000000000001,"5min":67.533000000000001,"15min":67.533000000000001},"last":15.423}]},{"osd":2,"last update":"Wed Dec 6 18:29:58 2023","interfaces":[{"interface":"back","average":{"1min":11.874000000000001,"5min":17.681000000000001,"15min":20.295000000000002},"min":{"1min":7.9169999999999998,"5min":6.7599999999999998,"15min":6.7599999999999998},"max":{"1min":21.646000000000001,"5min":200.88499999999999,"15min":200.88499999999999},"last":16.253},{"interface":"front","average":{"1min":15.34,"5min":16.172999999999998,"15min":16.414000000000001},"min":{"1min":10.709,"5min":7.4119999999999999,"15min":7.4119999999999999},"max":{"1min":23.888000000000002,"5min":48.005000000000003,"15min":48.005000000000003},"last":18.863}]},{"osd":3,"last update":"Wed Dec 6 18:30:01 2023","interfaces":[{"interface":"back","average":{"1min":11.470000000000001,"5min":13.114000000000001,"15min":13.734},"min":{"1min":7.298,"5min":7.298,"15min":7.298},"max":{"1min":17.768999999999998,"5min":25.190999999999999,"15min":25.190999999999999},"last":14.430999999999999},{"interface":"front","average":{"1min":13.824999999999999,"5min":15.862,"15min":16.193999999999999},"min":{"1min":10.824999999999999,"5min":7.6079999999999997,"15min":7.6079999999999997},"max":{"1min":18.117000000000001,"5min":47.006999999999998,"15min":47.006999999999998},"last":13.106999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:29:58 2023","interfaces":[{"interface":"back","average":{"1min":13.254,"5min":14.601000000000001,"15min":15.083},"min":{"1min":6.1859999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"max":{"1min":36.865000000000002,"5min":36.865000000000002,"15min":36.865000000000002},"last":11.699},{"interface":"front","average":{"1min":16.071999999999999,"5min":16.669,"15min":16.648},"min":{"1min":10.798,"5min":7.7169999999999996,"15min":7.7169999999999996},"max":{"1min":34.909999999999997,"5min":48.988999999999997,"15min":48.988999999999997},"last":17.445}]},{"osd":5,"last update":"Wed Dec 6 18:29:58 2023","interfaces":[{"interface":"back","average":{"1min":18.145,"5min":18.452999999999999,"15min":18.452999999999999},"min":{"1min":7.8879999999999999,"5min":7.8879999999999999,"15min":7.8879999999999999},"max":{"1min":49.390999999999998,"5min":57.308,"15min":57.308},"last":16.826000000000001},{"interface":"front","average":{"1min":13.266,"5min":14.016999999999999,"15min":14.914999999999999},"min":{"1min":6.0439999999999996,"5min":6.0129999999999999,"15min":6.0129999999999999},"max":{"1min":47.415999999999997,"5min":68.463999999999999,"15min":68.463999999999999},"last":8.8610000000000007}]},{"osd":7,"last update":"Wed Dec 6 18:27:54 2023","interfaces":[{"interface":"back","average":{"1min":17.899000000000001,"5min":17.899000000000001,"15min":17.899000000000001},"min":{"1min":10.449999999999999,"5min":10.449999999999999,"15min":10.449999999999999},"max":{"1min":45.374000000000002,"5min":45.374000000000002,"15min":45.374000000000002},"last":20.443000000000001},{"interface":"front","average":{"1min":14.888999999999999,"5min":14.888999999999999,"15min":14.888999999999999},"min":{"1min":8.7219999999999995,"5min":8.7219999999999995,"15min":8.7219999999999995},"max":{"1min":38.709000000000003,"5min":38.709000000000003,"15min":38.709000000000003},"last":19.629999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:30:17.537 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:30:17.557 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:17.536+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:17.609 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-12-06T18:30:17.653 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:17.649+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:17.722 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:17.723 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:17.723 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:17.723 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:17.734 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:30:17.734 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:17.730+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:17.837 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:17.833+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:17.935 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:17.931+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:17.952 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:17.952 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:17.952 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:17.952 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:17.952 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:17.964 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:30:18.036 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:18.032+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:18.097 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:30:18.094+0000 7f33bda11640 0 --2- 172.21.15.61:0/3645907858 >> [v2:172.21.15.61:3300/0,v1:172.21.15.61:6789/0] conn(0x7f33b8070950 0x7f33b8135be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-12-06T18:30:18.108 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:35.300 101796== 2023-12-06T18:30:18.108 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:35.300 101796== Exit program on first error (--exit-on-first-error=yes) 2023-12-06T18:30:18.117 INFO:tasks.ceph.mon.b.smithi152.stderr:daemon-helper: command failed with exit status 42 2023-12-06T18:30:18.154 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:30:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 176.33832125500001, 2023-12-06T18:30:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:30:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:30:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:30:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:30:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:30:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:30:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:30:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:30:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:30:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:30:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:30:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:30:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:30:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:30:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:30:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:30:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:18.159 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:18.165 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:30:18.165 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:18.161+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:18.237 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:18.234+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:18.308 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:18.308 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:18.309 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:18.309 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:18.319 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:30:18.363 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:18.359+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:18.431 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:30:18.432 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:30:18.444 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2023-12-06T18:30:18.249764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226684,"kb_used_data":5224,"kb_used_omap":15,"kb_used_meta":217008,"kb_avail":754748036,"statfs":{"total":773094113280,"available":772861988864,"internally_reserved":0,"allocated":5349376,"data_stored":2735009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15557,"internal_metadata":222217019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":143,"apply_latency_ms":143,"commit_latency_ns":143000000,"apply_latency_ns":143000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.503664"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-06T18:30:08.687970+0000","last_change":"2023-12-06T18:29:09.051293+0000","last_active":"2023-12-06T18:30:08.687970+0000","last_peered":"2023-12-06T18:30:08.687970+0000","last_clean":"2023-12-06T18:30:08.687970+0000","last_became_active":"2023-12-06T18:29:09.045331+0000","last_became_peered":"2023-12-06T18:29:09.045331+0000","last_unstale":"2023-12-06T18:30:08.687970+0000","last_undegraded":"2023-12-06T18:30:08.687970+0000","last_fullsized":"2023-12-06T18:30:08.687970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T02:46:01.854508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":50,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-06T18:30:08.224980+0000","last_change":"2023-12-06T18:27:04.883840+0000","last_active":"2023-12-06T18:30:08.224980+0000","last_peered":"2023-12-06T18:30:08.224980+0000","last_clean":"2023-12-06T18:30:08.224980+0000","last_became_active":"2023-12-06T18:27:04.858001+0000","last_became_peered":"2023-12-06T18:27:04.858001+0000","last_unstale":"2023-12-06T18:30:08.224980+0000","last_undegraded":"2023-12-06T18:30:08.224980+0000","last_fullsized":"2023-12-06T18:30:08.224980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:44:02.916244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":54,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-12-06T18:30:14.903675+0000","last_change":"2023-12-06T18:27:04.336778+0000","last_active":"2023-12-06T18:30:14.903675+0000","last_peered":"2023-12-06T18:30:14.903675+0000","last_clean":"2023-12-06T18:30:14.903675+0000","last_became_active":"2023-12-06T18:27:04.321269+0000","last_became_peered":"2023-12-06T18:27:04.321269+0000","last_unstale":"2023-12-06T18:30:14.903675+0000","last_undegraded":"2023-12-06T18:30:14.903675+0000","last_fullsized":"2023-12-06T18:30:14.903675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T22:25:15.703579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-06T18:30:08.676926+0000","last_change":"2023-12-06T18:28:47.898016+0000","last_active":"2023-12-06T18:30:08.676926+0000","last_peered":"2023-12-06T18:30:08.676926+0000","last_clean":"2023-12-06T18:30:08.676926+0000","last_became_active":"2023-12-06T18:28:47.891814+0000","last_became_peered":"2023-12-06T18:28:47.891814+0000","last_unstale":"2023-12-06T18:30:08.676926+0000","last_undegraded":"2023-12-06T18:30:08.676926+0000","last_fullsized":"2023-12-06T18:30:08.676926+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T00:49:06.937273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":54,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-12-06T18:30:14.898082+0000","last_change":"2023-12-06T18:27:04.408081+0000","last_active":"2023-12-06T18:30:14.898082+0000","last_peered":"2023-12-06T18:30:14.898082+0000","last_clean":"2023-12-06T18:30:14.898082+0000","last_became_active":"2023-12-06T18:27:04.399765+0000","last_became_peered":"2023-12-06T18:27:04.399765+0000","last_unstale":"2023-12-06T18:30:14.898082+0000","last_undegraded":"2023-12-06T18:30:14.898082+0000","last_fullsized":"2023-12-06T18:30:14.898082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:40:33.570114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-12-06T18:30:08.762262+0000","last_change":"2023-12-06T18:30:08.762262+0000","last_active":"2023-12-06T18:30:08.762262+0000","last_peered":"2023-12-06T18:30:08.762262+0000","last_clean":"2023-12-06T18:30:08.762262+0000","last_became_active":"2023-12-06T18:30:08.733421+0000","last_became_peered":"2023-12-06T18:30:08.733421+0000","last_unstale":"2023-12-06T18:30:08.762262+0000","last_undegraded":"2023-12-06T18:30:08.762262+0000","last_fullsized":"2023-12-06T18:30:08.762262+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:43:31.796397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":68,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-12-06T18:30:14.892335+0000","last_change":"2023-12-06T18:28:47.876556+0000","last_active":"2023-12-06T18:30:14.892335+0000","last_peered":"2023-12-06T18:30:14.892335+0000","last_clean":"2023-12-06T18:30:14.892335+0000","last_became_active":"2023-12-06T18:28:47.865169+0000","last_became_peered":"2023-12-06T18:28:47.865169+0000","last_unstale":"2023-12-06T18:30:14.892335+0000","last_undegraded":"2023-12-06T18:30:14.892335+0000","last_fullsized":"2023-12-06T18:30:14.892335+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:16:40.095542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":54,"reported_epoch":67,"state":"active+clean","last_fresh":"2023-12-06T18:30:14.899925+0000","last_change":"2023-12-06T18:27:04.609381+0000","last_active":"2023-12-06T18:30:14.899925+0000","last_peered":"2023-12-06T18:30:14.899925+0000","last_clean":"2023-12-06T18:30:14.899925+0000","last_became_active":"2023-12-06T18:27:04.598701+0000","last_became_peered":"2023-12-06T18:27:04.598701+0000","last_unstale":"2023-12-06T18:30:14.899925+0000","last_undegraded":"2023-12-06T18:30:14.899925+0000","last_fullsized":"2023-12-06T18:30:14.899925+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:29:58.962692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":189,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-12-06T18:30:14.034503+0000","last_change":"2023-12-06T18:28:48.207134+0000","last_active":"2023-12-06T18:30:14.034503+0000","last_peered":"2023-12-06T18:30:14.034503+0000","last_clean":"2023-12-06T18:30:14.034503+0000","last_became_active":"2023-12-06T18:28:47.875258+0000","last_became_peered":"2023-12-06T18:28:47.875258+0000","last_unstale":"2023-12-06T18:30:14.034503+0000","last_undegraded":"2023-12-06T18:30:14.034503+0000","last_fullsized":"2023-12-06T18:30:14.034503+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:14:32.667039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1204224,"data_stored":1193520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411371,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":561152,"data_stored":245656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":13.292,"5min":10.651,"15min":10.521000000000001},"min":{"1min":6.0229999999999997,"5min":5.4009999999999998,"15min":5.4009999999999998},"max":{"1min":108.57599999999999,"5min":108.57599999999999,"15min":108.57599999999999},"last":7.5490000000000004},{"interface":"front","average":{"1min":14.936,"5min":12.337,"15min":12.619},"min":{"1min":5.2030000000000003,"5min":3.7200000000000002,"15min":3.7200000000000002},"max":{"1min":128.09399999999999,"5min":128.09399999999999,"15min":128.09399999999999},"last":15.664}]},{"osd":1,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":12.457000000000001,"5min":12.502000000000001,"15min":12.871},"min":{"1min":6.2539999999999996,"5min":6.2539999999999996,"15min":6.2539999999999996},"max":{"1min":70.072000000000003,"5min":70.072000000000003,"15min":70.072000000000003},"last":25.940999999999999},{"interface":"front","average":{"1min":12.776,"5min":14.673,"15min":15.680999999999999},"min":{"1min":6.444,"5min":6.444,"15min":6.444},"max":{"1min":27.934000000000001,"5min":32.920000000000002,"15min":32.920000000000002},"last":18.939}]},{"osd":2,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":15.362,"5min":14.803000000000001,"15min":14.759},"min":{"1min":8.5649999999999995,"5min":7.5149999999999997,"15min":7.5149999999999997},"max":{"1min":75.125,"5min":75.125,"15min":75.125},"last":10.257999999999999},{"interface":"front","average":{"1min":17.251000000000001,"5min":17.606000000000002,"15min":18.052},"min":{"1min":9.6590000000000007,"5min":8.8290000000000006,"15min":8.8290000000000006},"max":{"1min":69.403999999999996,"5min":69.403999999999996,"15min":69.403999999999996},"last":14.669}]},{"osd":4,"last update":"Wed Dec 6 18:30:04 2023","interfaces":[{"interface":"back","average":{"1min":10.486000000000001,"5min":11.387,"15min":11.696999999999999},"min":{"1min":5.8879999999999999,"5min":5.8879999999999999,"15min":5.8879999999999999},"max":{"1min":21.064,"5min":21.064,"15min":21.064},"last":16.904},{"interface":"front","average":{"1min":16.756,"5min":15.946,"15min":16.045999999999999},"min":{"1min":6.3099999999999996,"5min":6.3099999999999996,"15min":6.3099999999999996},"max":{"1min":71.718000000000004,"5min":71.718000000000004,"15min":71.718000000000004},"last":8.3740000000000006}]},{"osd":5,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":18.776,"5min":18.231999999999999,"15min":17.882000000000001},"min":{"1min":10.853999999999999,"5min":8.3179999999999996,"15min":8.3179999999999996},"max":{"1min":75.751999999999995,"5min":75.751999999999995,"15min":75.751999999999995},"last":13.849},{"interface":"front","average":{"1min":16.117000000000001,"5min":17.902000000000001,"15min":18.187000000000001},"min":{"1min":7.9269999999999996,"5min":7.9269999999999996,"15min":7.9269999999999996},"max":{"1min":70.933000000000007,"5min":70.933000000000007,"15min":70.933000000000007},"last":11.571}]},{"osd":6,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":13.388,"5min":15.525,"15min":16.145},"min":{"1min":8.2050000000000001,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":28.111999999999998,"5min":29.245000000000001,"15min":29.245000000000001},"last":12.720000000000001},{"interface":"front","average":{"1min":20.686,"5min":20.766999999999999,"15min":21.321000000000002},"min":{"1min":12.061,"5min":10.973000000000001,"15min":10.973000000000001},"max":{"1min":74.224000000000004,"5min":74.224000000000004,"15min":74.224000000000004},"last":22.341999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:27:59 2023","interfaces":[{"interface":"back","average":{"1min":20.715,"5min":20.715,"15min":20.715},"min":{"1min":12.949999999999999,"5min":12.949999999999999,"15min":12.949999999999999},"max":{"1min":34.771000000000001,"5min":34.771000000000001,"15min":34.771000000000001},"last":19.757000000000001},{"interface":"front","average":{"1min":19.077999999999999,"5min":19.077999999999999,"15min":19.077999999999999},"min":{"1min":12.787000000000001,"5min":12.787000000000001,"15min":12.787000000000001},"max":{"1min":32.286000000000001,"5min":32.286000000000001,"15min":32.286000000000001},"last":23.440999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476778,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":536576,"data_stored":236938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:30:04 2023","interfaces":[{"interface":"back","average":{"1min":8.673,"5min":10.326000000000001,"15min":11.119999999999999},"min":{"1min":4.9989999999999997,"5min":4.609,"15min":4.609},"max":{"1min":24.404,"5min":38.740000000000002,"15min":38.740000000000002},"last":14.67},{"interface":"front","average":{"1min":11.497,"5min":10.827999999999999,"15min":11.194000000000001},"min":{"1min":4.6550000000000002,"5min":4.6550000000000002,"15min":4.6550000000000002},"max":{"1min":33.387,"5min":33.387,"15min":33.387},"last":9.7319999999999993}]},{"osd":1,"last update":"Wed Dec 6 18:30:04 2023","interfaces":[{"interface":"back","average":{"1min":13.208,"5min":11.56,"15min":11.606999999999999},"min":{"1min":5.9139999999999997,"5min":5.9139999999999997,"15min":5.9139999999999997},"max":{"1min":42.920999999999999,"5min":42.920999999999999,"15min":42.920999999999999},"last":9.1440000000000001},{"interface":"front","average":{"1min":13.545,"5min":13.532999999999999,"15min":13.851000000000001},"min":{"1min":7.4930000000000003,"5min":6.6260000000000003,"15min":6.6260000000000003},"max":{"1min":34.607999999999997,"5min":34.607999999999997,"15min":34.607999999999997},"last":15.356}]},{"osd":3,"last update":"Wed Dec 6 18:30:04 2023","interfaces":[{"interface":"back","average":{"1min":17.064,"5min":17.663,"15min":19.145},"min":{"1min":9.5180000000000007,"5min":6.1260000000000003,"15min":6.1260000000000003},"max":{"1min":49.609000000000002,"5min":112.129,"15min":112.129},"last":12.630000000000001},{"interface":"front","average":{"1min":17.344000000000001,"5min":21.335999999999999,"15min":23.93},"min":{"1min":8.4559999999999995,"5min":7.5510000000000002,"15min":7.5510000000000002},"max":{"1min":61.243000000000002,"5min":181.601,"15min":181.601},"last":16.608000000000001}]},{"osd":4,"last update":"Wed Dec 6 18:30:04 2023","interfaces":[{"interface":"back","average":{"1min":15.673,"5min":16.632000000000001,"15min":17.838000000000001},"min":{"1min":7.8070000000000004,"5min":6.609,"15min":6.609},"max":{"1min":60.332000000000001,"5min":97.088999999999999,"15min":97.088999999999999},"last":23.422000000000001},{"interface":"front","average":{"1min":14.141,"5min":18.652999999999999,"15min":20.876999999999999},"min":{"1min":6.3559999999999999,"5min":6.3559999999999999,"15min":6.3559999999999999},"max":{"1min":30.734999999999999,"5min":98.069000000000003,"15min":98.069000000000003},"last":8.0199999999999996}]},{"osd":5,"last update":"Wed Dec 6 18:30:04 2023","interfaces":[{"interface":"back","average":{"1min":17.09,"5min":17.640999999999998,"15min":18.077000000000002},"min":{"1min":9.8819999999999997,"5min":7.8170000000000002,"15min":7.8170000000000002},"max":{"1min":57.628999999999998,"5min":57.628999999999998,"15min":57.628999999999998},"last":22.111000000000001},{"interface":"front","average":{"1min":15.912000000000001,"5min":15.712999999999999,"15min":15.797000000000001},"min":{"1min":9.9930000000000003,"5min":8.843,"15min":8.843},"max":{"1min":47.749000000000002,"5min":47.749000000000002,"15min":47.749000000000002},"last":11.444000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:30:04 2023","interfaces":[{"interface":"back","average":{"1min":18.800000000000001,"5min":35.606000000000002,"15min":43.548999999999999},"min":{"1min":11.016,"5min":8.7289999999999992,"15min":8.7289999999999992},"max":{"1min":64.034999999999997,"5min":498.38499999999999,"15min":498.38499999999999},"last":18.835000000000001},{"interface":"front","average":{"1min":17.834,"5min":37.069000000000003,"15min":45.436999999999998},"min":{"1min":9.7230000000000008,"5min":9.7230000000000008,"15min":9.7230000000000008},"max":{"1min":54.920000000000002,"5min":496.17399999999998,"15min":496.17399999999998},"last":25.861000000000001}]},{"osd":7,"last update":"Wed Dec 6 18:27:57 2023","interfaces":[{"interface":"back","average":{"1min":27.364999999999998,"5min":27.364999999999998,"15min":27.364999999999998},"min":{"1min":12.321,"5min":12.321,"15min":12.321},"max":{"1min":182.93600000000001,"5min":182.93600000000001,"15min":182.93600000000001},"last":20.274000000000001},{"interface":"front","average":{"1min":45.905000000000001,"5min":45.905000000000001,"15min":45.905000000000001},"min":{"1min":13.724,"5min":13.724,"15min":13.724},"max":{"1min":497.67700000000002,"5min":497.67700000000002,"15min":497.67700000000002},"last":26.888000000000002}]}]},{"osd":1,"up_from":16,"seq":68719476779,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":950272,"data_stored":639137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":9.5990000000000002,"5min":18.391999999999999,"15min":22.372},"min":{"1min":6.0579999999999998,"5min":5.5700000000000003,"15min":5.5700000000000003},"max":{"1min":12.356999999999999,"5min":238.87100000000001,"15min":238.87100000000001},"last":18.282},{"interface":"front","average":{"1min":9.6799999999999997,"5min":34.107999999999997,"15min":44.859999999999999},"min":{"1min":5.5369999999999999,"5min":4.4269999999999996,"15min":4.4269999999999996},"max":{"1min":26.919,"5min":661.33600000000001,"15min":661.33600000000001},"last":14.526999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":9.5739999999999998,"5min":26.436,"15min":33.494},"min":{"1min":6.8040000000000003,"5min":6.6340000000000003,"15min":6.6340000000000003},"max":{"1min":15.69,"5min":384.315,"15min":384.315},"last":25.260000000000002},{"interface":"front","average":{"1min":14.73,"5min":37.917000000000002,"15min":48.116999999999997},"min":{"1min":10.743,"5min":5.7009999999999996,"15min":5.7009999999999996},"max":{"1min":18.289000000000001,"5min":656.98699999999997,"15min":656.98699999999997},"last":26.274999999999999}]},{"osd":3,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":11.050000000000001,"5min":12.286,"15min":13.162000000000001},"min":{"1min":5.9720000000000004,"5min":5.548,"15min":5.548},"max":{"1min":26.821000000000002,"5min":34.057000000000002,"15min":34.057000000000002},"last":21.591999999999999},{"interface":"front","average":{"1min":12.624000000000001,"5min":14.897,"15min":16.113},"min":{"1min":6.5730000000000004,"5min":6.5730000000000004,"15min":6.5730000000000004},"max":{"1min":24.925999999999998,"5min":35.832999999999998,"15min":35.832999999999998},"last":23.164000000000001}]},{"osd":4,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":11.384,"5min":32.529000000000003,"15min":41.786999999999999},"min":{"1min":6.7009999999999996,"5min":6.2930000000000001,"15min":6.2930000000000001},"max":{"1min":22.437999999999999,"5min":447.67200000000003,"15min":447.67200000000003},"last":16.388999999999999},{"interface":"front","average":{"1min":15.162000000000001,"5min":39.052,"15min":49.468000000000004},"min":{"1min":9.6129999999999995,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":24,"5min":630.572,"15min":630.572},"last":20.324999999999999}]},{"osd":5,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":15.984999999999999,"5min":35.731999999999999,"15min":44.042000000000002},"min":{"1min":8.9689999999999994,"5min":8.9689999999999994,"15min":8.9689999999999994},"max":{"1min":23.050000000000001,"5min":389.55000000000001,"15min":389.55000000000001},"last":24.338999999999999},{"interface":"front","average":{"1min":13.598000000000001,"5min":44.341000000000001,"15min":58.034999999999997},"min":{"1min":7.5359999999999996,"5min":7.0410000000000004,"15min":7.0410000000000004},"max":{"1min":26.456,"5min":658.48599999999999,"15min":658.48599999999999},"last":10.609999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:30:02 2023","interfaces":[{"interface":"back","average":{"1min":14.804,"5min":29.702999999999999,"15min":36.640000000000001},"min":{"1min":9.6579999999999995,"5min":8.4710000000000001,"15min":8.4710000000000001},"max":{"1min":31.664999999999999,"5min":439.56400000000002,"15min":439.56400000000002},"last":13.683999999999999},{"interface":"front","average":{"1min":17.984000000000002,"5min":31.940000000000001,"15min":37.909999999999997},"min":{"1min":10.641999999999999,"5min":10.641999999999999,"15min":10.641999999999999},"max":{"1min":33.786999999999999,"5min":386.71499999999997,"15min":386.71499999999997},"last":27.262}]},{"osd":7,"last update":"Wed Dec 6 18:27:58 2023","interfaces":[{"interface":"back","average":{"1min":43.825000000000003,"5min":43.825000000000003,"15min":43.825000000000003},"min":{"1min":11.693,"5min":11.693,"15min":11.693},"max":{"1min":431.46100000000001,"5min":431.46100000000001,"15min":431.46100000000001},"last":12.829000000000001},{"interface":"front","average":{"1min":42.889000000000003,"5min":42.889000000000003,"15min":42.889000000000003},"min":{"1min":11.731999999999999,"5min":11.731999999999999,"15min":11.731999999999999},"max":{"1min":388.52999999999997,"5min":388.52999999999997,"15min":388.52999999999997},"last":19.300999999999998}]}]},{"osd":0,"up_from":16,"seq":68719476780,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":548864,"data_stored":241297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 6 18:30:01 2023","interfaces":[{"interface":"back","average":{"1min":9.7509999999999994,"5min":15.065,"15min":17.385999999999999},"min":{"1min":3.2360000000000002,"5min":3.2360000000000002,"15min":3.2360000000000002},"max":{"1min":47.661000000000001,"5min":116.593,"15min":116.593},"last":8.7330000000000005},{"interface":"front","average":{"1min":12.177,"5min":16.544,"15min":18.117000000000001},"min":{"1min":4.9139999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"max":{"1min":50.127000000000002,"5min":117.896,"15min":117.896},"last":19.241}]},{"osd":2,"last update":"Wed Dec 6 18:30:01 2023","interfaces":[{"interface":"back","average":{"1min":13.339,"5min":14.113,"15min":13.630000000000001},"min":{"1min":5.694,"5min":5.694,"15min":5.694},"max":{"1min":43.709000000000003,"5min":43.752000000000002,"15min":43.752000000000002},"last":17.036999999999999},{"interface":"front","average":{"1min":11.170999999999999,"5min":12.009,"15min":11.318},"min":{"1min":6.2880000000000003,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":27.573,"5min":46.387999999999998,"15min":46.387999999999998},"last":10.082000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:30:07 2023","interfaces":[{"interface":"back","average":{"1min":13.433,"5min":15.228999999999999,"15min":16.306000000000001},"min":{"1min":6.1379999999999999,"5min":6.1379999999999999,"15min":6.1379999999999999},"max":{"1min":29.677,"5min":46.689999999999998,"15min":46.689999999999998},"last":13.933},{"interface":"front","average":{"1min":16.504999999999999,"5min":16.332999999999998,"15min":16.297000000000001},"min":{"1min":8.532,"5min":8.0329999999999995,"15min":8.0329999999999995},"max":{"1min":32.801000000000002,"5min":50.545999999999999,"15min":50.545999999999999},"last":11.106}]},{"osd":4,"last update":"Wed Dec 6 18:30:01 2023","interfaces":[{"interface":"back","average":{"1min":9.3989999999999991,"5min":17.760999999999999,"15min":20.173999999999999},"min":{"1min":5.9349999999999996,"5min":5.9349999999999996,"15min":5.9349999999999996},"max":{"1min":14.582000000000001,"5min":82.432000000000002,"15min":82.432000000000002},"last":12.483000000000001},{"interface":"front","average":{"1min":15.801,"5min":19.405000000000001,"15min":20.338000000000001},"min":{"1min":10.051,"5min":6.2629999999999999,"15min":6.2629999999999999},"max":{"1min":31.408000000000001,"5min":81.239999999999995,"15min":81.239999999999995},"last":21.504999999999999}]},{"osd":5,"last update":"Wed Dec 6 18:30:01 2023","interfaces":[{"interface":"back","average":{"1min":13.711,"5min":17.306000000000001,"15min":17.523},"min":{"1min":8.0099999999999998,"5min":8.0099999999999998,"15min":8.0099999999999998},"max":{"1min":25.613,"5min":71.367000000000004,"15min":71.367000000000004},"last":11.683},{"interface":"front","average":{"1min":13.176,"5min":17.669,"15min":18.024999999999999},"min":{"1min":8.5299999999999994,"5min":8.5299999999999994,"15min":8.5299999999999994},"max":{"1min":30.242000000000001,"5min":72.307000000000002,"15min":72.307000000000002},"last":18.007999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:30:01 2023","interfaces":[{"interface":"back","average":{"1min":17.878,"5min":19.167999999999999,"15min":19.061},"min":{"1min":11.423,"5min":7.7560000000000002,"15min":7.7560000000000002},"max":{"1min":29.899000000000001,"5min":62.932000000000002,"15min":62.932000000000002},"last":20.074999999999999},{"interface":"front","average":{"1min":15.651999999999999,"5min":18.478999999999999,"15min":19.190000000000001},"min":{"1min":9.6940000000000008,"5min":9.6940000000000008,"15min":9.6940000000000008},"max":{"1min":28.785,"5min":52.627000000000002,"15min":52.627000000000002},"last":15.988}]},{"osd":7,"last update":"Wed Dec 6 18:27:55 2023","interfaces":[{"interface":"back","average":{"1min":21.66,"5min":21.66,"15min":21.66},"min":{"1min":12.804,"5min":12.804,"15min":12.804},"max":{"1min":48.781999999999996,"5min":48.781999999999996,"15min":48.781999999999996},"last":22.117999999999999},{"interface":"front","average":{"1min":19.440000000000001,"5min":19.440000000000001,"15min":19.440000000000001},"min":{"1min":10.537000000000001,"5min":10.537000000000001,"15min":10.537000000000001},"max":{"1min":38.530000000000001,"5min":38.530000000000001,"15min":38.530000000000001},"last":14.666}]}]},{"osd":5,"up_from":15,"seq":64424509484,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":999424,"data_stored":640762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:30:03 2023","interfaces":[{"interface":"back","average":{"1min":7.0629999999999997,"5min":13.085000000000001,"15min":15.403},"min":{"1min":3.3660000000000001,"5min":3.3660000000000001,"15min":3.3660000000000001},"max":{"1min":19.173999999999999,"5min":208.291,"15min":208.291},"last":13.097},{"interface":"front","average":{"1min":10.037000000000001,"5min":16.419,"15min":19.463000000000001},"min":{"1min":4.758,"5min":4.4299999999999997,"15min":4.4299999999999997},"max":{"1min":15.025,"5min":199.94900000000001,"15min":199.94900000000001},"last":8.8040000000000003}]},{"osd":1,"last update":"Wed Dec 6 18:30:03 2023","interfaces":[{"interface":"back","average":{"1min":13.615,"5min":45.186999999999998,"15min":59.460000000000001},"min":{"1min":8.5809999999999995,"5min":8.1180000000000003,"15min":8.1180000000000003},"max":{"1min":41.234999999999999,"5min":1211.6010000000001,"15min":1211.6010000000001},"last":10.135999999999999},{"interface":"front","average":{"1min":10.398,"5min":40.906999999999996,"15min":54.581000000000003},"min":{"1min":5.3049999999999997,"5min":5.1340000000000003,"15min":5.1340000000000003},"max":{"1min":39.802999999999997,"5min":1189.3869999999999,"15min":1189.3869999999999},"last":9.5719999999999992}]},{"osd":2,"last update":"Wed Dec 6 18:30:03 2023","interfaces":[{"interface":"back","average":{"1min":14.023,"5min":21.236999999999998,"15min":24.780999999999999},"min":{"1min":6.5129999999999999,"5min":6.5129999999999999,"15min":6.5129999999999999},"max":{"1min":41.716000000000001,"5min":283.46800000000002,"15min":283.46800000000002},"last":10.744999999999999},{"interface":"front","average":{"1min":17.048999999999999,"5min":24.818000000000001,"15min":28.687000000000001},"min":{"1min":12.295999999999999,"5min":8.9350000000000005,"15min":8.9350000000000005},"max":{"1min":42.981999999999999,"5min":296.47800000000001,"15min":296.47800000000001},"last":16.443000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:30:03 2023","interfaces":[{"interface":"back","average":{"1min":15.493,"5min":17.984000000000002,"15min":19.076000000000001},"min":{"1min":9.1739999999999995,"5min":9.1739999999999995,"15min":9.1739999999999995},"max":{"1min":21.536999999999999,"5min":47.926000000000002,"15min":47.926000000000002},"last":14.406000000000001},{"interface":"front","average":{"1min":11.621,"5min":13.949,"15min":15.118},"min":{"1min":5.3979999999999997,"5min":5.3979999999999997,"15min":5.3979999999999997},"max":{"1min":19.492999999999999,"5min":66.176000000000002,"15min":66.176000000000002},"last":17.27}]},{"osd":4,"last update":"Wed Dec 6 18:30:03 2023","interfaces":[{"interface":"back","average":{"1min":16.521000000000001,"5min":18.350000000000001,"15min":19.611999999999998},"min":{"1min":12.817,"5min":10.286,"15min":10.286},"max":{"1min":27.754999999999999,"5min":67.248000000000005,"15min":67.248000000000005},"last":12.250999999999999},{"interface":"front","average":{"1min":9.8469999999999995,"5min":11.958,"15min":13.016},"min":{"1min":5.3300000000000001,"5min":5.3300000000000001,"15min":5.3300000000000001},"max":{"1min":30.934999999999999,"5min":77.266999999999996,"15min":77.266999999999996},"last":6.6779999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:30:03 2023","interfaces":[{"interface":"back","average":{"1min":10.468,"5min":14.179,"15min":15.666},"min":{"1min":6.1280000000000001,"5min":5.915,"15min":5.915},"max":{"1min":18.870000000000001,"5min":76.751999999999995,"15min":76.751999999999995},"last":8.5120000000000005},{"interface":"front","average":{"1min":17.943000000000001,"5min":19.835999999999999,"15min":20.946000000000002},"min":{"1min":13.004,"5min":8.8390000000000004,"15min":8.8390000000000004},"max":{"1min":36.256,"5min":57.101999999999997,"15min":57.101999999999997},"last":18.149999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:27:55 2023","interfaces":[{"interface":"back","average":{"1min":16.204000000000001,"5min":16.204000000000001,"15min":16.204000000000001},"min":{"1min":7.367,"5min":7.367,"15min":7.367},"max":{"1min":70.768000000000001,"5min":70.768000000000001,"15min":70.768000000000001},"last":15.602},{"interface":"front","average":{"1min":18.081,"5min":18.081,"15min":18.081},"min":{"1min":6.3150000000000004,"5min":6.3150000000000004,"15min":6.3150000000000004},"max":{"1min":63.991999999999997,"5min":63.991999999999997,"15min":63.991999999999997},"last":13.712999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":593920,"data_stored":242641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:30:00 2023","interfaces":[{"interface":"back","average":{"1min":9.2739999999999991,"5min":12.443,"15min":13.609999999999999},"min":{"1min":3.7429999999999999,"5min":3.7429999999999999,"15min":3.7429999999999999},"max":{"1min":41.076000000000001,"5min":53.298999999999999,"15min":53.298999999999999},"last":14.099},{"interface":"front","average":{"1min":9.0860000000000003,"5min":10.368,"15min":10.833},"min":{"1min":4.8140000000000001,"5min":3.9670000000000001,"15min":3.9670000000000001},"max":{"1min":35.863,"5min":52.567,"15min":52.567},"last":17.042000000000002}]},{"osd":1,"last update":"Wed Dec 6 18:30:00 2023","interfaces":[{"interface":"back","average":{"1min":8.8810000000000002,"5min":12.872999999999999,"15min":13.939},"min":{"1min":6.4640000000000004,"5min":5.835,"15min":5.835},"max":{"1min":13.583,"5min":49.183,"15min":49.183},"last":6.2869999999999999},{"interface":"front","average":{"1min":10.976000000000001,"5min":14.564,"15min":15.682},"min":{"1min":7.516,"5min":5.96,"15min":5.96},"max":{"1min":14.849,"5min":45.412999999999997,"15min":45.412999999999997},"last":10.346}]},{"osd":2,"last update":"Wed Dec 6 18:30:00 2023","interfaces":[{"interface":"back","average":{"1min":11.057,"5min":18.984999999999999,"15min":22.416},"min":{"1min":6.2549999999999999,"5min":5.6390000000000002,"15min":5.6390000000000002},"max":{"1min":16.393999999999998,"5min":85.394000000000005,"15min":85.394000000000005},"last":9.1590000000000007},{"interface":"front","average":{"1min":13.888,"5min":18.390999999999998,"15min":20.571999999999999},"min":{"1min":8.7560000000000002,"5min":6.3639999999999999,"15min":6.3639999999999999},"max":{"1min":27.651,"5min":69.530000000000001,"15min":69.530000000000001},"last":19.039000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:30:06 2023","interfaces":[{"interface":"back","average":{"1min":13.596,"5min":14.331,"15min":14.377000000000001},"min":{"1min":8.5359999999999996,"5min":7.008,"15min":7.008},"max":{"1min":26.219000000000001,"5min":26.219000000000001,"15min":26.219000000000001},"last":20.195},{"interface":"front","average":{"1min":13.808,"5min":14.66,"15min":15.273999999999999},"min":{"1min":5.5549999999999997,"5min":5.5549999999999997,"15min":5.5549999999999997},"max":{"1min":22.760999999999999,"5min":29.902999999999999,"15min":29.902999999999999},"last":11.901999999999999}]},{"osd":5,"last update":"Wed Dec 6 18:30:00 2023","interfaces":[{"interface":"back","average":{"1min":11.284000000000001,"5min":15.779999999999999,"15min":17.044},"min":{"1min":6.2430000000000003,"5min":6.125,"15min":6.125},"max":{"1min":23.835000000000001,"5min":78.798000000000002,"15min":78.798000000000002},"last":15.260999999999999},{"interface":"front","average":{"1min":15.913,"5min":17.991,"15min":18.765000000000001},"min":{"1min":8.0519999999999996,"5min":6.4450000000000003,"15min":6.4450000000000003},"max":{"1min":25.544,"5min":55.814999999999998,"15min":55.814999999999998},"last":23.238}]},{"osd":6,"last update":"Wed Dec 6 18:30:00 2023","interfaces":[{"interface":"back","average":{"1min":16.356000000000002,"5min":17.545000000000002,"15min":17.936},"min":{"1min":9.6489999999999991,"5min":7.7850000000000001,"15min":7.7850000000000001},"max":{"1min":32.491,"5min":51.149000000000001,"15min":51.149000000000001},"last":22.629000000000001},{"interface":"front","average":{"1min":15.821999999999999,"5min":16.190000000000001,"15min":16.683},"min":{"1min":8.5630000000000006,"5min":7.8700000000000001,"15min":7.8700000000000001},"max":{"1min":34.929000000000002,"5min":40.426000000000002,"15min":40.426000000000002},"last":12.615}]},{"osd":7,"last update":"Wed Dec 6 18:27:57 2023","interfaces":[{"interface":"back","average":{"1min":18.751999999999999,"5min":18.751999999999999,"15min":18.751999999999999},"min":{"1min":11.249000000000001,"5min":11.249000000000001,"15min":11.249000000000001},"max":{"1min":33.067999999999998,"5min":33.067999999999998,"15min":33.067999999999998},"last":18.355},{"interface":"front","average":{"1min":17.401,"5min":17.401,"15min":17.401},"min":{"1min":7.6050000000000004,"5min":7.6050000000000004,"15min":7.6050000000000004},"max":{"1min":32.356999999999999,"5min":32.356999999999999,"15min":32.356999999999999},"last":20.920999999999999}]}]},{"osd":7,"up_from":61,"seq":261993005059,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":584,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":598016,"data_stored":242922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4395,"internal_metadata":28634837},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":59,"apply_latency_ms":59,"commit_latency_ns":59000000,"apply_latency_ns":59000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.2510000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.403}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":16.428999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":19.808}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.839}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":12.263999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.036}]}]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":561152,"data_stored":245656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:29:58 2023","interfaces":[{"interface":"back","average":{"1min":8.3689999999999998,"5min":9.6289999999999996,"15min":9.9359999999999999},"min":{"1min":5.4539999999999997,"5min":3.8519999999999999,"15min":3.8519999999999999},"max":{"1min":14.390000000000001,"5min":46.947000000000003,"15min":46.947000000000003},"last":5.3929999999999998},{"interface":"front","average":{"1min":8.3010000000000002,"5min":10.358000000000001,"15min":10.140000000000001},"min":{"1min":4.0650000000000004,"5min":4.0650000000000004,"15min":4.0650000000000004},"max":{"1min":19.611999999999998,"5min":45.953000000000003,"15min":45.953000000000003},"last":6.3019999999999996}]},{"osd":1,"last update":"Wed Dec 6 18:29:58 2023","interfaces":[{"interface":"back","average":{"1min":9.7309999999999999,"5min":11.683999999999999,"15min":12.542},"min":{"1min":4.3760000000000003,"5min":4.3760000000000003,"15min":4.3760000000000003},"max":{"1min":66.313000000000002,"5min":66.313000000000002,"15min":66.313000000000002},"last":6.9009999999999998},{"interface":"front","average":{"1min":14.108000000000001,"5min":14.211,"15min":14.603},"min":{"1min":7.1269999999999998,"5min":5.8540000000000001,"15min":5.8540000000000001},"max":{"1min":67.533000000000001,"5min":67.533000000000001,"15min":67.533000000000001},"last":16.629000000000001}]},{"osd":2,"last update":"Wed Dec 6 18:29:58 2023","interfaces":[{"interface":"back","average":{"1min":11.874000000000001,"5min":17.681000000000001,"15min":20.295000000000002},"min":{"1min":7.9169999999999998,"5min":6.7599999999999998,"15min":6.7599999999999998},"max":{"1min":21.646000000000001,"5min":200.88499999999999,"15min":200.88499999999999},"last":8.3889999999999993},{"interface":"front","average":{"1min":15.34,"5min":16.172999999999998,"15min":16.414000000000001},"min":{"1min":10.709,"5min":7.4119999999999999,"15min":7.4119999999999999},"max":{"1min":23.888000000000002,"5min":48.005000000000003,"15min":48.005000000000003},"last":23.443999999999999}]},{"osd":3,"last update":"Wed Dec 6 18:30:01 2023","interfaces":[{"interface":"back","average":{"1min":11.470000000000001,"5min":13.114000000000001,"15min":13.734},"min":{"1min":7.298,"5min":7.298,"15min":7.298},"max":{"1min":17.768999999999998,"5min":25.190999999999999,"15min":25.190999999999999},"last":36.283999999999999},{"interface":"front","average":{"1min":13.824999999999999,"5min":15.862,"15min":16.193999999999999},"min":{"1min":10.824999999999999,"5min":7.6079999999999997,"15min":7.6079999999999997},"max":{"1min":18.117000000000001,"5min":47.006999999999998,"15min":47.006999999999998},"last":34.317}]},{"osd":4,"last update":"Wed Dec 6 18:29:58 2023","interfaces":[{"interface":"back","average":{"1min":13.254,"5min":14.601000000000001,"15min":15.083},"min":{"1min":6.1859999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"max":{"1min":36.865000000000002,"5min":36.865000000000002,"15min":36.865000000000002},"last":17.826000000000001},{"interface":"front","average":{"1min":16.071999999999999,"5min":16.669,"15min":16.648},"min":{"1min":10.798,"5min":7.7169999999999996,"15min":7.7169999999999996},"max":{"1min":34.909999999999997,"5min":48.988999999999997,"15min":48.988999999999997},"last":24.372}]},{"osd":5,"last update":"Wed Dec 6 18:29:58 2023","interfaces":[{"interface":"back","average":{"1min":18.145,"5min":18.452999999999999,"15min":18.452999999999999},"min":{"1min":7.8879999999999999,"5min":7.8879999999999999,"15min":7.8879999999999999},"max":{"1min":49.390999999999998,"5min":57.308,"15min":57.308},"last":25.257999999999999},{"interface":"front","average":{"1min":13.266,"5min":14.016999999999999,"15min":14.914999999999999},"min":{"1min":6.0439999999999996,"5min":6.0129999999999999,"15min":6.0129999999999999},"max":{"1min":47.415999999999997,"5min":68.463999999999999,"15min":68.463999999999999},"last":9.9320000000000004}]},{"osd":7,"last update":"Wed Dec 6 18:27:54 2023","interfaces":[{"interface":"back","average":{"1min":17.899000000000001,"5min":17.899000000000001,"15min":17.899000000000001},"min":{"1min":10.449999999999999,"5min":10.449999999999999,"15min":10.449999999999999},"max":{"1min":45.374000000000002,"5min":45.374000000000002,"15min":45.374000000000002},"last":21.754000000000001},{"interface":"front","average":{"1min":14.888999999999999,"5min":14.888999999999999,"15min":14.888999999999999},"min":{"1min":8.7219999999999995,"5min":8.7219999999999995,"15min":8.7219999999999995},"max":{"1min":38.709000000000003,"5min":38.709000000000003,"15min":38.709000000000003},"last":26.178999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:30:18.445 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-12-06T18:30:18.446 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:30:18.446 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2023-12-06T18:30:18.446 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2023-12-06T18:30:18.446 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2023-12-06T18:30:18.446 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2023-12-06T18:30:18.448 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:18.445+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:18.460 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:18.460 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:18.461 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:18.461 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:18.461 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:18.470 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:30:18.527 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:30:18.538 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-12-06T18:30:18.540 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:18.536+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:18.613 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:18.613 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2023-12-06T18:30:18.613 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:18.641 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:18.637+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:18.657 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:18.657 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:18.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:18.658 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:18.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.658 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:18.658 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:18.658 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 57.559958743000003, 2023-12-06T18:30:18.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:30:18.658 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:18.658 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:18.658 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:18.658 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:18.658 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:18.658 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:30:18.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:18.658 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:18.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:18.659 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:18.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:18.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:18.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:18.659 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:18.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:18.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:18.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:18.659 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:30:18.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:30:18.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:18.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:18.660 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:30:18.660 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:30:18.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:18.660 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:18.660 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:30:18.660 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:30:18.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:18.660 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:18.660 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:30:18.660 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:30:18.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:18.660 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:18.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:30:18.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:30:18.661 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:18.661 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:18.661 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:18.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:30:18.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:30:18.661 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:18.661 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:18.661 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:18.661 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:18.661 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:18.661 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:18.668 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:30:18.741 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:18.738+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:18.800 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:18.800 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:18.800 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:18.800 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:18.809 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:30:18.853 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:18.849+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:18.937 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:18.937 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:18.937 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:18.937 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:18.937 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:18.943 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:18.939+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:18.946 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:30:19.045 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:19.041+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:19.116 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:19.116 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:19.116 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:19.116 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:19.116 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.116 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:19.116 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:19.117 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 117.622883335, 2023-12-06T18:30:19.117 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:30:19.117 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.117 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:19.117 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:19.117 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:19.117 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:19.117 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:30:19.117 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.117 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.117 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.117 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:19.117 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.117 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:19.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:30:19.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:30:19.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:30:19.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:30:19.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:30:19.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:30:19.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:30:19.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:30:19.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:30:19.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:30:19.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.120 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.120 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:30:19.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:30:19.120 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.120 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.120 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.120 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.120 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.120 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:19.125 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:30:19.169 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:19.166+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:19.246 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:19.243+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:19.253 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:19.254 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:19.254 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:19.254 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:19.264 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:30:19.351 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:19.347+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:19.435 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:19.436 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:19.436 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:19.436 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:19.436 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:19.445 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:30:19.486 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:19.483+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:19.549 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:19.546+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:19.650 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:19.646+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:19.654 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:19.654 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:19.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:19.655 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:19.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.655 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:30:19.655 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:19.655 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 203.03836143000001, 2023-12-06T18:30:19.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:30:19.655 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.655 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.655 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:19.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:19.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:30:19.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:30:19.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:30:19.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:30:19.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:30:19.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:30:19.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:30:19.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:30:19.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:30:19.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:30:19.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:30:19.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:30:19.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:30:19.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:30:19.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:30:19.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:30:19.659 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.659 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.659 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.659 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:30:19.660 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:19.660 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 202.01553442100001, 2023-12-06T18:30:19.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:30:19.660 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.660 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.660 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:19.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:19.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:30:19.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:30:19.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:30:19.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:30:19.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:30:19.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:30:19.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:30:19.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:30:19.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:30:19.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:30:19.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:30:19.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:30:19.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:30:19.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:30:19.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:30:19.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:30:19.663 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.663 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.663 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.663 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:30:19.663 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:19.664 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 202.003487783, 2023-12-06T18:30:19.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:30:19.664 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.664 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.664 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:19.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:19.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.665 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.665 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:30:19.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:30:19.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.665 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.665 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:30:19.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:30:19.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.665 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.665 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:30:19.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:30:19.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:30:19.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:30:19.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:30:19.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:30:19.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:30:19.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:30:19.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:30:19.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:30:19.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:30:19.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:30:19.667 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.668 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.668 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.668 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:30:19.668 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:19.668 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 201.98208717599999, 2023-12-06T18:30:19.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:30:19.668 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.668 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.668 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:19.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:19.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:30:19.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:30:19.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:30:19.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:30:19.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:30:19.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:30:19.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:30:19.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:30:19.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:30:19.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:30:19.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:30:19.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:30:19.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:30:19.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:30:19.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:30:19.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:30:19.672 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.672 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.672 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.672 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:30:19.672 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:19.672 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 201.98090785299999, 2023-12-06T18:30:19.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:30:19.672 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.672 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.673 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:19.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:19.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:30:19.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:30:19.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:30:19.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:30:19.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:30:19.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:30:19.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:30:19.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:30:19.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:30:19.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:30:19.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:30:19.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:30:19.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:30:19.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:30:19.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:30:19.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:30:19.676 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.676 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.676 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.676 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:30:19.676 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:19.676 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 201.98033587200001, 2023-12-06T18:30:19.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:30:19.677 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.677 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.677 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.677 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.677 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.677 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:19.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.677 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:19.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:30:19.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:30:19.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:30:19.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:30:19.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:30:19.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:30:19.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:30:19.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:30:19.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:30:19.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:30:19.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:30:19.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:30:19.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:30:19.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:30:19.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:30:19.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:30:19.681 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.681 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.681 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.681 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:30:19.681 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:19.681 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 201.53450040199999, 2023-12-06T18:30:19.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:30:19.681 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.681 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.681 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.681 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:19.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:19.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:30:19.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:30:19.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:30:19.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:30:19.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:30:19.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:30:19.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:30:19.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:30:19.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:30:19.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:30:19.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:30:19.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:30:19.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:30:19.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:30:19.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.686 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.686 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.686 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:30:19.686 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:30:19.686 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.686 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.686 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.687 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.687 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:30:19.687 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:19.687 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 201.53290054600001, 2023-12-06T18:30:19.687 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:30:19.687 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.687 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.688 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.688 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.688 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.688 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:19.688 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.688 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.688 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.688 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:19.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.689 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:30:19.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:30:19.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.689 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:30:19.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:30:19.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.689 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:30:19.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:30:19.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:30:19.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:30:19.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:30:19.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:30:19.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:30:19.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:30:19.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:30:19.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:30:19.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:30:19.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:30:19.692 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.692 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.692 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.692 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:30:19.692 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:19.692 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 201.15369493, 2023-12-06T18:30:19.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:30:19.692 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.692 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.692 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.693 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:19.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.693 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:19.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.693 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:30:19.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:30:19.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:30:19.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:30:19.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:30:19.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:30:19.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:30:19.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:30:19.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:30:19.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:30:19.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:30:19.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:30:19.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:30:19.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:30:19.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:30:19.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:30:19.696 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.696 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.696 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.696 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:30:19.696 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:19.696 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 200.91619274799999, 2023-12-06T18:30:19.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:30:19.697 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.697 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.697 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:19.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:19.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:30:19.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:30:19.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.698 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:30:19.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:30:19.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.698 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:30:19.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:30:19.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.699 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:30:19.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:30:19.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.699 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:30:19.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:30:19.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:30:19.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:30:19.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:30:19.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:30:19.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:30:19.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:30:19.700 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.700 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.700 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.701 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.701 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.701 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:30:19.701 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:19.701 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 200.91546474899999, 2023-12-06T18:30:19.701 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:30:19.701 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.701 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.701 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.701 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.701 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.701 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:19.701 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.701 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.702 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.702 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.702 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:19.702 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.702 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.702 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.702 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.702 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:30:19.702 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:30:19.702 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.702 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.702 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.702 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:30:19.702 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:30:19.703 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.703 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.703 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.703 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:30:19.703 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:30:19.703 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.703 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.703 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.703 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:30:19.703 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:30:19.703 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.703 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.703 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:30:19.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:30:19.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:30:19.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:30:19.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:30:19.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:30:19.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:30:19.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:30:19.705 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.705 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.705 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.705 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:30:19.705 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:19.705 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 200.91045233899999, 2023-12-06T18:30:19.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:30:19.705 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.705 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.705 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:19.706 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.706 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.706 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.706 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.706 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:19.706 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.706 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.706 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.706 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.706 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:30:19.706 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:30:19.706 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.706 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.706 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.707 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:30:19.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:30:19.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.707 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.707 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:30:19.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:30:19.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.707 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.707 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:30:19.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:30:19.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:30:19.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:30:19.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:30:19.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:30:19.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:30:19.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:30:19.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:30:19.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:30:19.709 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.709 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.709 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.710 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:30:19.710 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:19.710 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 200.90492526, 2023-12-06T18:30:19.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:30:19.710 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.710 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.710 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.710 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.710 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:19.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.711 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:19.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.711 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:30:19.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:30:19.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.711 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:30:19.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:30:19.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:30:19.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:30:19.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:30:19.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:30:19.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:30:19.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:30:19.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:30:19.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:30:19.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:30:19.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:30:19.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:30:19.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:30:19.713 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.713 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.714 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.714 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:30:19.714 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:19.714 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 200.90441442299999, 2023-12-06T18:30:19.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:30:19.714 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.714 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.714 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.714 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.714 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:19.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:19.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:30:19.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:30:19.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:30:19.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:30:19.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:30:19.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:30:19.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:30:19.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:30:19.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:30:19.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:30:19.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:30:19.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:30:19.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:30:19.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:30:19.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:30:19.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:30:19.718 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.718 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.718 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.718 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:30:19.718 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:19.718 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 200.90395516199999, 2023-12-06T18:30:19.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:30:19.718 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.718 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.718 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.718 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:19.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:19.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:30:19.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:30:19.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:30:19.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:30:19.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:30:19.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:30:19.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:30:19.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:30:19.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:30:19.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:30:19.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:30:19.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:30:19.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:30:19.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:30:19.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:30:19.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:30:19.722 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.722 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.723 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.723 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:30:19.723 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:19.723 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 194.324039355, 2023-12-06T18:30:19.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:30:19.723 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.723 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.723 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.723 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:19.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:19.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:30:19.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:30:19.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:30:19.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:30:19.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:30:19.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:30:19.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:30:19.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:30:19.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:30:19.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:30:19.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:30:19.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:30:19.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:30:19.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:30:19.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.727 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.727 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:30:19.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:30:19.727 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.727 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.727 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.727 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:30:19.727 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:19.727 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 194.31780155000001, 2023-12-06T18:30:19.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:30:19.727 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.727 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.727 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:19.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:19.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:30:19.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:30:19.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:30:19.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:30:19.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:30:19.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:30:19.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:30:19.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:30:19.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:30:19.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:30:19.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:30:19.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:30:19.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:30:19.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:30:19.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:30:19.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:30:19.732 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.732 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.732 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.732 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:30:19.732 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:19.732 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 194.31173847100001, 2023-12-06T18:30:19.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:30:19.732 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.733 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.733 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:19.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:19.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:30:19.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:30:19.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:30:19.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:30:19.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:30:19.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:30:19.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:30:19.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:30:19.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:30:19.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:30:19.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:30:19.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:30:19.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:30:19.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:30:19.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:30:19.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:30:19.736 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.736 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.736 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.736 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:30:19.736 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:19.736 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 194.17435138600001, 2023-12-06T18:30:19.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:30:19.736 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.737 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.737 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:19.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:19.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:30:19.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:30:19.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:30:19.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:30:19.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:30:19.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:30:19.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:30:19.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:30:19.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:30:19.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:30:19.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:30:19.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:30:19.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:30:19.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:30:19.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:30:19.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:30:19.741 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.741 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.741 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.741 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:30:19.741 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:19.741 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 194.16768065900001, 2023-12-06T18:30:19.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:30:19.741 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:19.741 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:19.741 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:19.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:19.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:19.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:19.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:19.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:19.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:19.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:30:19.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:30:19.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:19.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:30:19.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:30:19.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:19.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:30:19.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:30:19.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:19.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:30:19.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:30:19.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:19.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:30:19.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:30:19.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:19.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:30:19.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:30:19.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:19.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:30:19.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:30:19.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:19.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:19.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:19.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:30:19.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:30:19.745 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.745 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.745 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.745 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:19.745 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:19.745 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:19.746 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:30:19.788 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:19.784+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:19.852 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:19.848+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:19.916 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:19.916 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:19.916 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:19.916 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:19.925 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:30:19.966 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:19.963+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:20.053 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:20.050+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:20.093 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:20.093 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:20.093 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:20.093 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:20.093 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:20.101 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:30:20.154 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:20.151+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:20.228 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:20.228 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:20.228 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:20.228 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:30:20.228 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:20.237 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:30:20.281 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:20.277+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:20.356 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:20.353+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:20.365 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:20.365 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:20.365 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:20.365 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:20.375 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:30:20.460 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:20.456+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:20.547 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:20.547 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:20.547 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:20.547 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:20.547 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:20.556 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:30:20.598 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:20.594+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:20.660 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:20.657+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:20.724 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:20.724 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:20.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:20.724 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:30:20.725 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:20.735 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:30:20.761 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:20.757+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:20.862 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:20.858+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:20.883 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:20.883 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:20.883 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:20.884 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:20.893 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:30:20.977 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:20.974+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:21.064 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:21.060+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:21.069 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:21.069 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:21.069 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:21.070 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:21.070 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:21.079 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:30:21.165 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:21.161+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:21.257 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:21.257 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:21.257 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:21.257 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:30:21.258 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:21.265 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:21.262+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:21.267 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:30:21.367 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:21.363+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:21.443 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:21.443 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:21.443 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:21.444 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:21.452 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:30:21.497 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:21.494+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:21.568 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:21.564+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:21.589 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:21.589 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:21.589 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:21.598 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:30:21.669 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:21.665+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:21.770 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:21.767+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:21.834 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:21.834 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:21.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:21.835 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:21.835 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.835 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:30:21.835 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:21.835 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 206.42362094200001, 2023-12-06T18:30:21.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:30:21.835 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.835 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.835 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.835 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.835 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.835 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:30:21.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.835 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.836 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.836 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.836 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:21.836 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.836 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.836 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.836 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:21.836 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.836 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.836 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.836 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:30:21.836 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:30:21.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.837 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.837 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:30:21.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:30:21.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.837 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.837 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:30:21.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:30:21.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.837 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.837 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:30:21.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:30:21.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:30:21.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:30:21.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:30:21.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:30:21.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:30:21.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:30:21.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:30:21.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.839 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:30:21.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:30:21.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.839 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:30:21.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:30:21.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.839 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:30:21.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:30:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:30:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:30:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:30:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:30:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:30:21.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:30:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 204.315047703, 2023-12-06T18:30:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:30:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.841 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:30:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.842 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:30:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:30:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:30:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:30:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:30:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:30:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:30:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:30:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:30:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:30:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:30:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:30:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:30:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:30:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:30:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:30:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:30:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:30:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.845 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:30:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:30:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:30:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:30:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:21.846 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 204.31430247099999, 2023-12-06T18:30:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:30:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:30:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.847 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:30:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:30:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:30:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:30:21.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:30:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:30:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:30:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:30:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:30:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:30:21.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:30:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:30:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:30:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:30:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:30:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:30:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:30:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:30:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:30:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:30:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.851 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:30:21.852 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:30:21.852 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.852 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.852 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.852 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.852 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.852 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:21.852 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:21.852 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 204.31361871600001, 2023-12-06T18:30:21.852 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:30:21.852 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.852 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.852 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.852 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.853 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.853 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:30:21.853 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.853 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.853 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.853 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.853 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:21.853 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.853 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.853 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.853 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.853 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:21.853 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.853 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.853 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:30:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:30:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:30:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:30:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:30:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:30:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.854 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.855 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:30:21.855 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:30:21.855 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.855 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.855 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.855 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:30:21.855 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:30:21.855 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.855 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.855 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.855 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:30:21.855 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:30:21.855 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.855 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.855 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.856 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:30:21.856 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:30:21.856 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.856 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.856 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.856 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:30:21.856 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:30:21.856 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.856 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.856 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.856 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:30:21.856 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:30:21.856 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.856 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.856 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.856 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:30:21.857 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:30:21.857 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.857 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.857 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.857 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:30:21.857 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:30:21.857 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.857 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.857 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.857 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.857 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.857 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:21.857 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:21.857 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 204.30804546100001, 2023-12-06T18:30:21.857 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:30:21.857 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.858 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.858 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.858 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.858 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.858 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:30:21.858 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.858 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.858 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.858 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.858 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:21.858 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.858 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.858 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.858 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.858 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:21.859 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.859 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:30:21.859 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:30:21.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.859 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:30:21.859 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:30:21.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.859 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:30:21.859 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:30:21.860 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.860 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.860 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.860 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:30:21.860 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:30:21.860 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.860 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.860 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.860 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:30:21.860 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:30:21.860 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.860 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.860 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.860 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:30:21.861 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:30:21.861 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.861 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.861 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.861 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:30:21.861 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:30:21.861 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.861 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.861 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.861 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:30:21.861 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:30:21.861 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.861 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.861 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.861 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:30:21.861 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:30:21.862 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.862 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.862 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.862 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:30:21.862 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:30:21.862 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.862 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.862 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.862 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:30:21.862 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:30:21.862 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.862 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.862 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.862 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.863 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.863 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:21.863 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:21.863 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 204.307371821, 2023-12-06T18:30:21.863 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:30:21.863 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.863 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.863 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.863 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.863 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.863 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:30:21.863 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.863 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.863 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.863 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.864 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:21.864 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.864 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.864 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.864 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:21.864 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.864 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.864 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.864 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:30:21.864 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:30:21.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.864 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.864 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.864 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:30:21.865 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:30:21.865 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.865 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.865 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.865 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:30:21.865 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:30:21.865 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.865 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.865 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.865 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:30:21.865 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:30:21.865 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.865 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.865 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.865 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:30:21.866 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:30:21.866 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.866 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.866 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.866 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:30:21.866 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:30:21.866 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.866 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.866 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.866 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:30:21.866 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:30:21.866 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.866 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.866 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.867 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:30:21.867 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:30:21.867 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.867 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.867 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.867 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:30:21.867 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:30:21.867 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.867 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.867 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.867 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:30:21.867 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:30:21.868 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.868 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.868 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.868 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:30:21.868 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:30:21.868 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.868 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.868 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.868 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.868 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.868 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:21.868 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:21.868 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 204.306709622, 2023-12-06T18:30:21.869 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:30:21.869 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.869 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.869 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.869 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.869 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:30:21.869 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.869 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.869 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.869 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.869 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:21.869 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.869 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.869 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.870 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.870 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:21.870 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.870 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.870 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.870 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.870 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:30:21.870 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:30:21.870 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.870 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.870 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.870 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:30:21.870 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:30:21.870 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.870 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.871 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.871 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:30:21.871 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:30:21.871 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.871 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.871 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.871 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:30:21.871 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:30:21.871 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.871 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.871 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.871 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:30:21.871 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:30:21.871 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.871 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.871 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.872 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:30:21.872 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:30:21.872 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.872 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.872 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.872 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:30:21.872 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:30:21.872 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.872 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.872 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.872 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:30:21.872 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:30:21.872 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.872 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.872 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.873 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:30:21.873 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:30:21.873 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.873 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.873 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.873 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:30:21.873 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:30:21.873 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.873 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.873 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.873 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:30:21.873 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:30:21.873 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.873 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.873 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.874 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.874 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.874 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:21.874 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:21.874 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 204.23400018699999, 2023-12-06T18:30:21.874 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:30:21.874 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.874 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.874 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.874 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.874 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.874 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:30:21.874 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.874 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.874 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.875 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.875 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:21.875 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.875 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.875 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.875 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.875 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:21.875 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.875 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.875 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.875 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.875 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:30:21.875 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:30:21.875 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.875 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.876 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.876 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:30:21.876 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:30:21.876 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.876 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.876 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.876 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:30:21.876 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:30:21.876 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.876 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.876 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.876 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:30:21.876 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:30:21.876 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.876 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.876 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.877 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:30:21.877 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:30:21.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.877 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.877 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.877 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:30:21.877 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:30:21.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.877 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.877 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.877 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:30:21.877 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:30:21.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.877 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.877 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.878 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:30:21.878 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:30:21.878 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.878 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.878 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.878 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:30:21.878 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:30:21.878 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.878 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.878 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.878 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:30:21.878 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:30:21.878 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.878 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.878 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.879 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:30:21.879 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:30:21.879 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.879 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.879 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.879 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.879 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.879 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:21.879 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:21.879 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 204.233318478, 2023-12-06T18:30:21.879 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:30:21.879 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.879 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.879 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.879 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.880 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.880 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:30:21.880 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.880 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.880 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.880 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.880 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:21.880 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.880 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.880 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.880 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.880 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:21.880 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.880 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.881 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.881 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.881 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:30:21.881 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:30:21.881 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.881 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.881 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.881 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:30:21.881 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:30:21.881 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.881 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.881 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.881 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:30:21.881 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:30:21.881 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.881 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:30:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:30:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:30:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:30:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:30:21.882 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:30:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:30:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:30:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:30:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:30:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:30:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:30:21.883 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:30:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:30:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:30:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:30:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.884 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 203.15451439, 2023-12-06T18:30:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:30:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:30:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:30:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:30:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:30:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:30:21.886 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.887 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.887 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.887 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:30:21.887 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:30:21.887 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.887 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.887 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.887 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:30:21.887 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:30:21.887 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.887 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.887 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.887 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:30:21.887 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:30:21.887 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.888 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.888 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.888 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:30:21.888 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:30:21.888 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.888 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.888 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.888 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:30:21.888 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:30:21.888 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.888 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.888 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.888 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:30:21.888 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:30:21.888 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.889 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.889 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.889 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:30:21.889 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:30:21.889 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.889 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.889 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.889 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:30:21.889 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:30:21.889 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.889 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.889 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.889 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:30:21.889 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:30:21.889 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.889 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.890 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.890 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.890 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.890 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:21.890 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:21.890 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 203.15309781600001, 2023-12-06T18:30:21.890 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:30:21.890 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.890 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.890 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.890 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.890 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.890 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:30:21.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.891 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.891 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.891 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:21.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.891 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.891 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:21.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.891 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.891 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:30:21.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:30:21.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.892 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.892 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:30:21.892 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:30:21.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.892 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.892 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:30:21.892 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:30:21.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.892 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.892 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:30:21.892 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:30:21.893 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.893 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.893 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.893 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:30:21.893 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:30:21.893 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.893 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.893 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.893 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:30:21.893 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:30:21.893 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.893 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.893 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.893 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:30:21.893 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:30:21.893 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.894 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:30:21.894 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:30:21.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.894 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:30:21.894 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:30:21.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.894 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:30:21.894 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:30:21.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.895 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.895 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.895 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:30:21.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:30:21.895 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.895 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.895 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.895 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.895 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.895 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:21.895 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:21.895 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 203.15201269100001, 2023-12-06T18:30:21.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:30:21.895 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.895 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.895 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.896 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.896 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.896 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:30:21.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.896 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.896 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.896 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.896 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:21.896 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.896 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.896 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.896 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:21.896 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.897 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.897 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.897 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:30:21.897 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:30:21.897 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.897 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.897 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.897 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:30:21.897 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:30:21.897 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.897 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.897 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.897 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:30:21.897 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:30:21.897 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.897 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:30:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:30:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:30:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:30:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:30:21.898 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:30:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:30:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:30:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:30:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:30:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:30:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:30:21.899 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.900 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.900 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.900 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:30:21.900 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:30:21.900 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.900 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.900 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.900 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:30:21.900 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:30:21.900 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.900 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.900 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.900 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.900 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.900 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:21.901 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:21.901 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 203.11686358899999, 2023-12-06T18:30:21.901 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:30:21.901 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.901 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.901 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.901 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.901 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.901 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:30:21.901 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.901 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.901 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.901 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.901 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:21.901 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.901 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.902 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.902 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.902 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:21.902 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.902 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.902 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.902 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.902 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:30:21.902 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:30:21.902 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.902 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.902 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.902 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:30:21.902 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:30:21.902 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.903 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.903 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.903 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:30:21.903 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:30:21.903 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.903 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.903 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.903 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:30:21.903 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:30:21.903 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.903 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.903 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.903 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:30:21.903 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:30:21.903 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.903 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.904 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.904 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:30:21.904 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:30:21.904 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.904 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.904 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.904 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:30:21.904 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:30:21.904 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.904 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.904 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.904 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:30:21.904 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:30:21.904 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.904 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.905 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.905 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:30:21.905 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:30:21.905 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.905 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.905 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.905 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:30:21.905 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:30:21.905 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.905 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.905 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.905 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:30:21.905 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:30:21.905 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.905 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.906 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.906 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.906 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.906 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:21.906 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:21.906 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 196.56713306399999, 2023-12-06T18:30:21.906 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:30:21.906 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.906 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.906 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.906 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.906 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.906 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:30:21.906 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.906 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.907 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.907 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.907 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:21.907 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.907 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.907 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.907 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.907 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:21.907 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.907 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.907 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.907 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.907 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:30:21.907 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:30:21.907 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.908 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.908 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.908 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:30:21.908 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:30:21.908 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.908 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.908 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.908 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:30:21.908 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:30:21.908 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.908 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.908 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.908 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:30:21.908 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:30:21.908 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.908 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.909 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.909 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:30:21.909 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:30:21.909 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.909 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.909 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.909 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:30:21.909 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:30:21.909 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.909 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.909 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.909 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:30:21.909 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:30:21.909 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.909 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.910 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.910 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:30:21.910 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:30:21.910 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.910 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.910 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.910 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:30:21.910 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:30:21.910 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.910 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.910 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.910 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:30:21.910 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:30:21.910 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.910 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.911 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.911 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:30:21.911 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:30:21.911 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.911 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.911 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.911 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.911 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.911 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:21.911 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:21.911 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 196.56632745600001, 2023-12-06T18:30:21.911 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:30:21.911 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.911 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.911 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.912 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.912 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.912 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:30:21.912 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.912 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.912 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.912 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.912 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:21.912 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.912 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.912 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.912 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.912 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:21.912 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.912 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.913 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.913 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.913 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:30:21.913 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:30:21.913 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.913 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.913 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.913 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:30:21.913 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:30:21.913 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.913 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.913 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.913 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:30:21.913 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:30:21.913 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.914 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.914 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.914 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:30:21.914 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:30:21.914 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.914 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.914 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.914 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:30:21.914 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:30:21.914 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.914 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.914 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.914 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:30:21.914 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:30:21.914 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.915 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.915 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.915 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:30:21.915 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:30:21.915 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.915 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.915 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.915 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:30:21.915 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:30:21.915 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.915 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.915 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.915 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:30:21.915 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:30:21.915 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.916 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.916 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.916 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:30:21.916 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:30:21.916 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.916 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.916 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.916 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:30:21.916 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:30:21.916 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.916 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.916 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.916 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.916 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.916 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:21.916 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:21.917 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 196.56563664699999, 2023-12-06T18:30:21.917 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:30:21.917 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.917 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.917 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.917 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.917 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.917 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:30:21.917 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.917 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.917 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.917 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.917 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:21.917 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.917 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.918 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.918 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.918 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:21.918 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.918 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.918 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.918 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.918 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:30:21.918 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:30:21.918 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.918 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.918 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.918 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:30:21.918 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:30:21.919 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.919 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.919 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.919 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:30:21.919 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:30:21.919 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.919 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.919 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.919 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:30:21.919 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:30:21.919 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.919 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.919 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.919 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:30:21.919 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:30:21.919 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.920 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.920 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.920 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:30:21.920 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:30:21.920 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.920 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.920 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.920 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:30:21.920 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:30:21.920 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.920 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.920 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.920 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:30:21.920 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:30:21.920 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.921 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.921 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.921 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:30:21.921 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:30:21.921 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.921 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.921 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.921 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:30:21.921 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:30:21.921 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.921 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.921 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.921 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:30:21.921 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:30:21.921 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.921 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.922 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.922 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.922 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.922 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:21.922 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:21.922 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 196.56133913900001, 2023-12-06T18:30:21.922 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:30:21.922 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.922 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.922 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.922 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.922 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.922 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:30:21.922 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.923 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.923 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.923 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.923 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:21.923 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.923 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.923 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.923 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.923 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:21.923 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.923 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.923 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.923 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.923 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:30:21.924 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:30:21.924 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.924 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.924 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.924 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:30:21.924 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:30:21.924 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.924 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.924 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.924 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:30:21.924 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:30:21.924 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.924 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.924 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.924 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:30:21.925 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:30:21.925 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.925 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.925 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.925 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:30:21.925 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:30:21.925 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.925 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.925 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.925 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:30:21.925 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:30:21.925 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.925 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.925 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.925 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:30:21.926 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:30:21.926 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.926 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.926 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.926 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:30:21.926 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:30:21.926 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.926 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.926 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.926 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:30:21.926 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:30:21.926 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.926 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.926 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.927 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:30:21.927 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:30:21.927 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.927 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.927 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.927 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:30:21.927 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:30:21.927 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.927 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.927 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.927 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.927 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.927 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:21.927 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:21.927 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 196.560671464, 2023-12-06T18:30:21.927 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:30:21.928 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.928 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.928 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.928 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.928 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.928 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:30:21.928 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.928 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.928 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.928 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.928 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:21.928 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.928 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.928 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.928 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.929 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:21.929 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.929 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.929 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.929 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.929 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:30:21.929 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:30:21.929 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.929 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.929 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.929 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:30:21.929 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:30:21.929 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.929 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.929 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.929 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:30:21.930 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:30:21.930 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.930 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.930 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.930 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:30:21.930 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:30:21.930 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.930 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.930 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.930 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:30:21.930 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:30:21.930 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.930 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.930 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.931 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:30:21.931 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:30:21.931 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.931 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.931 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.931 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:30:21.931 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:30:21.931 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.931 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.931 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.931 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:30:21.931 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:30:21.931 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.931 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.931 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:30:21.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:30:21.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.932 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:30:21.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:30:21.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.932 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:30:21.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:30:21.932 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.932 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.932 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 196.56001603600001, 2023-12-06T18:30:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:30:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:30:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:30:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:30:21.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.935 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.935 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:30:21.935 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:30:21.935 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.935 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.935 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:30:21.935 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:30:21.935 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.935 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.935 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:30:21.935 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:30:21.935 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.936 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.936 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:30:21.936 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:30:21.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.936 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.936 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:30:21.936 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:30:21.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.936 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.936 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:30:21.936 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:30:21.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.937 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.937 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:30:21.937 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:30:21.937 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.937 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.937 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.937 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:30:21.937 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:30:21.937 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.937 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.937 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.937 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:30:21.937 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:30:21.937 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.937 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.937 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.938 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:30:21.938 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:30:21.938 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.938 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.938 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.938 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.938 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.938 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:21.938 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:21.938 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 196.519361489, 2023-12-06T18:30:21.938 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:30:21.938 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:21.938 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:21.938 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:21.938 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:21.939 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:21.939 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:30:21.939 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.939 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:21.939 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.939 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:21.939 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:21.939 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.939 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.939 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.939 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:21.939 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:21.939 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:21.939 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.939 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.939 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:21.940 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:30:21.940 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:30:21.940 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.940 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.940 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:21.940 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:30:21.940 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:30:21.940 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.940 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.940 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:21.940 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:30:21.940 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:30:21.940 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.940 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.940 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:21.940 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:30:21.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:30:21.941 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.941 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:21.941 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:30:21.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:30:21.941 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.941 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:21.941 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:30:21.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:30:21.941 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.941 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:21.941 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:30:21.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:30:21.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:21.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:30:21.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:30:21.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:21.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:30:21.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:30:21.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:21.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:30:21.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:30:21.943 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:21.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:21.943 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:21.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:30:21.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:30:21.943 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.943 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.943 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.943 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:21.943 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:21.943 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:21.945 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:30:21.945 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:21.942+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:22.046 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:22.042+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:22.122 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:22.122 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:22.122 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:22.122 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:22.131 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:30:22.147 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:22.143+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:22.248 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:22.244+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:22.270 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:22.271 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:22.271 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:22.271 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:22.271 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:22.280 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:30:22.349 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:22.346+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:22.418 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:22.418 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:22.418 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:22.418 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:22.418 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.418 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:30:22.419 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:22.419 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 180.60218074299999, 2023-12-06T18:30:22.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:30:22.419 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:22.419 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:30:22.419 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:22.419 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.419 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:22.419 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:22.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:22.419 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.419 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.419 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:22.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:22.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:22.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:22.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:30:22.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:30:22.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:22.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:30:22.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:30:22.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.421 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:22.421 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:30:22.421 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:30:22.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.421 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.421 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:22.421 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:30:22.421 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:30:22.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.421 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.421 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:22.421 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:30:22.421 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:30:22.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.422 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.422 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:22.422 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:30:22.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:30:22.422 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.422 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.422 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:22.422 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:30:22.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:30:22.422 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:22.422 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:22.422 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:22.423 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:22.423 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:22.423 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:22.428 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:30:22.473 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:22.470+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:22.550 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:22.547+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:22.566 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:22.566 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:22.567 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:22.567 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:22.575 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:30:22.651 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:22.648+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:22.753 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:22.749+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:22.753 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:22.754 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:22.754 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:22.754 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:22.754 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:22.763 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:30:22.853 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:22.850+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:22.910 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:22.910 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:22.910 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:22.910 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:22.910 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.911 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:22.911 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:22.911 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 61.810636911000003, 2023-12-06T18:30:22.911 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:30:22.911 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:22.911 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:22.911 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:22.911 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:22.911 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:22.911 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:30:22.911 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.911 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:22.911 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.911 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:22.911 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:22.912 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:22.912 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.912 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.912 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:22.912 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:22.912 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:22.912 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.912 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.912 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:22.912 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:30:22.912 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:30:22.912 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.912 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.912 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:22.912 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:30:22.912 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:30:22.913 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.913 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.913 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:22.913 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:30:22.913 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:30:22.913 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.913 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.913 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:22.913 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:30:22.913 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:30:22.913 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.913 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.913 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:22.913 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:30:22.913 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:30:22.913 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.914 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.914 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:22.914 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:30:22.914 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:30:22.914 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:22.914 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:22.914 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:22.914 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.914 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.914 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:30:22.914 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:22.914 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 1.7923796090000002, 2023-12-06T18:30:22.914 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:30:22.914 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:22.915 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:22.915 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:22.915 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:22.915 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:22.915 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:30:22.915 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.915 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:22.915 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.915 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:22.915 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:22.915 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:22.915 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.915 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.915 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:22.915 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:22.916 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:22.916 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.916 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.916 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:22.916 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:30:22.916 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:30:22.916 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.916 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.916 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:22.916 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:30:22.916 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:30:22.916 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.916 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.916 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:22.916 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:30:22.916 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:30:22.917 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.917 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.917 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:22.917 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:30:22.917 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:30:22.917 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.917 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.917 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:22.917 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:30:22.917 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:30:22.917 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:22.917 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:22.917 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:22.917 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:30:22.917 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:30:22.917 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:22.918 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:22.918 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:22.918 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:22.918 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:22.918 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:22.919 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:30:22.961 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:22.958+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:23.054 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:23.051+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:23.060 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:23.061 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:23.061 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:23.061 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:23.071 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:30:23.156 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:23.153+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:23.240 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:23.240 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:23.240 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:23.240 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:23.240 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:23.250 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:30:23.256 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:23.254+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:23.357 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:23.354+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:23.380 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 121.887506238, 2023-12-06T18:30:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:30:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:23.381 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:30:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.382 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:30:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:30:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:30:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:30:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:30:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:30:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.383 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:30:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:30:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:30:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:30:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:30:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:30:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:23.384 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:23.385 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:23.385 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:23.385 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:23.391 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:30:23.459 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:23.456+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:23.532 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-12-06T18:30:23.533 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi152 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b" 2023-12-06T18:30:23.533 INFO:tasks.ceph.mon.b:Stopped 2023-12-06T18:30:23.533 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-12-06T18:30:23.533 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:30:23.549 INFO:tasks.ceph.mon.c.smithi061.stderr:2023-12-06T18:30:23.535+0000 9f54640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 106647) UID: 0 2023-12-06T18:30:23.549 INFO:tasks.ceph.mon.c.smithi061.stderr:2023-12-06T18:30:23.537+0000 9f54640 -1 mon.c@2(leader) e1 *** Got Signal Terminated *** 2023-12-06T18:30:23.560 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:23.556+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:23.560 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:23.561 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:23.561 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:23.561 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:23.570 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:30:23.627 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:30:23.627 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:30:23.627 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2023-12-06T18:30:23.627 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2023-12-06T18:30:23.627 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2023-12-06T18:30:23.628 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2023-12-06T18:30:23.698 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:23.695+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:23.744 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:23.744 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:23.744 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:23.744 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:23.745 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:23.754 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:30:23.795 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:23.792+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:23.838 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:23.838 INFO:teuthology.orchestra.run.smithi152.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2023-12-06T18:30:23.838 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:23.862 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:23.860+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:23.964 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:23.961+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:23.988 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:23.988 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:23.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:23.988 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:23.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.988 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:30:23.988 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:23.988 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 207.371526295, 2023-12-06T18:30:23.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:30:23.989 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:23.989 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:23.989 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:23.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:23.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:23.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:23.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:23.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:23.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:23.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:23.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:30:23.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:30:23.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:23.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:30:23.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:30:23.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:23.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:30:23.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:30:23.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:23.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:30:23.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:30:23.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:23.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:30:23.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:30:23.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:23.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:30:23.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:30:23.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:23.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:30:23.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:30:23.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:23.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:30:23.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:30:23.992 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:23.992 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:23.992 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:23.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.992 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:30:23.992 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:23.992 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 206.348699286, 2023-12-06T18:30:23.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:30:23.992 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:23.992 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:23.992 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:23.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:23.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:23.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:23.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:23.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:23.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:23.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:23.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:30:23.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:30:23.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:23.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:30:23.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:30:23.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:23.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:30:23.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:30:23.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:23.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:30:23.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:30:23.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:23.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:30:23.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:30:23.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:23.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:30:23.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:30:23.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:23.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:30:23.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:30:23.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:23.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:30:23.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:30:23.996 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:23.996 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:23.996 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:23.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.996 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:30:23.996 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:23.996 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 206.33665264800001, 2023-12-06T18:30:23.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:30:23.996 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:23.996 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:23.996 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:23.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:23.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:23.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:23.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:23.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:23.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:23.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:23.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:30:23.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:30:23.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:23.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:30:23.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:30:23.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:23.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:30:23.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:30:23.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:23.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:30:23.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:30:23.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:23.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:30:23.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:30:23.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:23.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:30:23.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:30:23.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:23.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:30:23.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:30:23.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:23.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:23.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:23.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:30:24.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:30:24.000 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.000 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.000 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.000 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:30:24.000 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:24.000 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 206.31525204100001, 2023-12-06T18:30:24.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:30:24.000 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:24.000 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:24.000 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:24.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:24.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:24.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:24.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:24.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:24.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:30:24.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:30:24.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.001 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:24.001 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:30:24.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:30:24.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:24.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:30:24.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:30:24.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:24.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:30:24.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:30:24.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:24.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:30:24.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:30:24.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:24.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:30:24.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:30:24.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:24.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:30:24.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:30:24.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:24.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:30:24.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:30:24.003 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.004 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.004 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.004 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:30:24.004 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:24.004 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 206.31407271800001, 2023-12-06T18:30:24.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:30:24.004 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:24.004 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:24.004 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:24.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:24.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:24.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:24.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:24.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:24.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:30:24.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:30:24.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:24.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:30:24.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:30:24.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:24.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:30:24.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:30:24.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:24.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:30:24.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:30:24.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:24.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:30:24.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:30:24.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:24.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:30:24.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:30:24.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:24.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:30:24.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:30:24.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:24.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:30:24.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:30:24.007 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.008 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.008 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.008 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.008 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.008 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:30:24.008 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:24.008 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 206.313500737, 2023-12-06T18:30:24.008 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:30:24.008 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:24.008 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:24.008 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:24.008 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.008 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:24.008 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:24.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:24.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:24.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:24.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:30:24.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:30:24.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:24.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:30:24.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:30:24.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:24.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:30:24.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:30:24.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:24.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:30:24.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:30:24.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:24.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:30:24.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:30:24.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:24.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:30:24.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:30:24.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:24.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:30:24.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:30:24.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:24.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:30:24.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:30:24.011 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.012 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.012 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.012 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:30:24.012 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:24.012 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 205.86766526700001, 2023-12-06T18:30:24.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:30:24.012 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:24.012 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:24.012 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:24.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:24.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:24.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.013 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:24.013 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:24.013 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.013 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:24.013 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:30:24.013 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:30:24.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.013 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:24.013 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:30:24.013 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:30:24.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.014 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:24.014 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:30:24.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:30:24.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.014 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:24.014 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:30:24.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:30:24.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.014 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:24.014 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:30:24.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:30:24.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:24.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:30:24.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:30:24.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:24.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:30:24.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:30:24.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:24.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:30:24.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:30:24.016 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.016 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.016 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.016 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:30:24.016 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:24.016 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 205.86606541099999, 2023-12-06T18:30:24.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:30:24.016 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:24.016 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:24.016 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:24.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:24.016 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:24.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:24.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:24.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:24.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:30:24.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:30:24.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:24.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:30:24.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:30:24.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:24.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:30:24.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:30:24.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:24.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:30:24.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:30:24.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:24.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:30:24.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:30:24.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:24.019 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:30:24.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:30:24.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:24.019 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:30:24.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:30:24.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:24.019 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:30:24.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:30:24.020 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.020 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.020 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.020 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:30:24.020 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:24.020 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 205.48685979499999, 2023-12-06T18:30:24.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:30:24.020 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:24.020 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:24.020 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:24.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:24.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:24.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.021 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:24.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:24.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.021 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:24.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:30:24.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:30:24.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.021 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:24.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:30:24.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:30:24.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:24.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:30:24.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:30:24.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:24.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:30:24.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:30:24.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:24.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:30:24.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:30:24.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:24.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:30:24.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:30:24.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:24.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:30:24.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:30:24.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:24.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:30:24.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:30:24.023 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.024 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.024 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.024 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.024 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:30:24.024 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:24.024 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 205.249357613, 2023-12-06T18:30:24.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:30:24.024 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:24.024 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:24.024 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:24.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.024 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:24.024 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:24.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.025 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.025 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:24.025 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:24.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.025 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.025 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:24.025 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:30:24.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:30:24.025 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.025 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:24.025 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:30:24.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:30:24.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:24.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:30:24.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:30:24.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:24.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:30:24.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:30:24.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:24.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:30:24.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:30:24.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:24.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:30:24.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:30:24.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:24.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:30:24.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:30:24.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:24.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:30:24.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:30:24.028 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.028 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.028 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.028 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.028 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:30:24.028 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:24.028 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 205.24862961400001, 2023-12-06T18:30:24.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:30:24.028 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:24.028 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:24.028 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:24.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:24.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:24.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.029 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:24.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:24.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.029 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:24.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:30:24.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:30:24.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.029 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:24.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:30:24.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:30:24.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:24.030 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:30:24.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:30:24.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:24.030 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:30:24.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:30:24.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:24.030 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:30:24.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:30:24.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:24.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:30:24.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:30:24.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:24.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:30:24.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:30:24.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:24.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:30:24.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:30:24.032 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.032 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.032 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.032 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:30:24.032 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:24.032 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 205.243617204, 2023-12-06T18:30:24.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:30:24.032 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:24.032 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:24.032 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:24.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:24.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:24.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:24.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:24.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:24.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:30:24.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:30:24.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:24.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:30:24.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:30:24.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.034 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:24.034 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:30:24.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:30:24.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.034 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:24.034 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:30:24.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:30:24.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.034 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:24.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:30:24.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:30:24.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:24.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:30:24.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:30:24.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:24.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:30:24.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:30:24.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:24.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:30:24.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:30:24.036 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.036 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.036 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.036 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:30:24.036 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:24.036 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 205.23809012500001, 2023-12-06T18:30:24.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:30:24.036 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:24.036 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:24.036 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:24.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:24.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:24.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:24.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:24.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:24.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:30:24.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:30:24.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:24.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:30:24.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:30:24.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:24.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:30:24.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:30:24.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:24.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:30:24.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:30:24.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:24.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:30:24.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:30:24.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:24.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:30:24.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:30:24.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:24.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:30:24.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:30:24.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:24.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:30:24.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:30:24.040 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.040 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.040 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.040 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:30:24.040 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:24.040 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 205.23757928800001, 2023-12-06T18:30:24.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:30:24.040 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:24.040 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:24.040 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:24.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:24.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:24.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:24.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:24.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:24.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:30:24.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:30:24.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:24.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:30:24.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:30:24.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:24.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:30:24.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:30:24.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:24.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:30:24.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:30:24.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:24.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:30:24.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:30:24.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:24.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:30:24.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:30:24.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:24.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:30:24.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:30:24.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:24.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:30:24.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:30:24.044 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.044 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.044 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.044 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:30:24.044 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:24.044 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 205.237120027, 2023-12-06T18:30:24.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:30:24.044 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:24.045 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:24.045 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:24.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:24.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:24.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:24.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:24.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:24.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:30:24.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:30:24.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:24.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:30:24.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:30:24.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:24.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:30:24.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:30:24.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:24.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:30:24.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:30:24.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:24.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:30:24.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:30:24.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:24.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:30:24.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:30:24.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:24.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:30:24.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:30:24.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:24.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:30:24.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:30:24.048 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.048 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.048 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.048 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:30:24.048 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:24.048 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 198.65720422000001, 2023-12-06T18:30:24.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:30:24.048 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:24.048 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:24.048 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:24.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:24.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:24.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:24.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:24.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:24.049 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:30:24.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:30:24.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:24.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:30:24.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:30:24.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:24.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:30:24.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:30:24.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:24.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:30:24.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:30:24.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:24.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:30:24.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:30:24.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:24.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:30:24.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:30:24.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:24.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:30:24.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:30:24.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:24.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:30:24.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:30:24.052 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.052 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.052 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.052 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:30:24.052 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:24.052 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 198.650966415, 2023-12-06T18:30:24.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:30:24.052 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:24.052 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:24.052 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:24.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:24.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:24.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:24.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:24.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:24.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:30:24.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:30:24.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:24.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:30:24.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:30:24.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:24.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:30:24.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:30:24.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:24.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:30:24.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:30:24.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:24.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:30:24.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:30:24.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:24.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:30:24.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:30:24.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:24.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:30:24.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:30:24.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:24.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:30:24.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:30:24.056 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.056 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.056 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.056 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:30:24.056 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:24.056 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 198.644903336, 2023-12-06T18:30:24.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:30:24.056 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:24.056 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:24.056 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:24.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:24.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:24.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:24.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:24.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:24.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:30:24.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:30:24.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:24.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:30:24.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:30:24.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:24.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:30:24.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:30:24.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:24.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:30:24.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:30:24.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:24.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:30:24.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:30:24.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:24.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:30:24.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:30:24.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:24.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:30:24.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:30:24.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:24.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:30:24.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:30:24.060 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.060 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.060 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.060 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:30:24.060 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:24.060 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 198.507516251, 2023-12-06T18:30:24.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:30:24.060 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:24.060 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:24.060 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:24.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:24.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:24.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:24.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:24.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:24.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:30:24.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:30:24.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:24.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:30:24.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:30:24.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:24.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:30:24.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:30:24.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:24.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:30:24.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:30:24.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:24.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:30:24.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:30:24.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:24.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:30:24.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:30:24.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:24.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:30:24.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:30:24.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:24.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:30:24.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:30:24.064 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.064 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.064 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.064 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:30:24.064 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:24.064 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 198.500845524, 2023-12-06T18:30:24.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:30:24.064 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:24.064 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:24.064 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:24.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:24.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:24.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:24.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:24.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:24.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:24.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:30:24.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:30:24.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:24.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:30:24.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:30:24.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:24.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:30:24.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:30:24.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:24.066 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:30:24.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:30:24.066 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.066 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:24.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:30:24.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:30:24.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:24.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:30:24.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:30:24.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:24.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:30:24.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:30:24.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:24.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:24.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:24.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:30:24.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:30:24.068 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.068 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.068 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.068 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:24.068 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:24.068 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:24.069 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:30:24.070 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:24.066+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:24.184 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:24.180+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:24.198 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:24.198 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:24.198 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:24.198 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:24.208 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:30:24.271 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:24.269+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:24.373 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:24.369+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:24.377 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:24.377 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:24.377 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:24.377 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:24.377 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:24.386 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:30:24.474 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:24.471+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:24.558 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:24.558 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:24.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:24.558 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:30:24.558 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:24.568 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:30:24.575 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:24.572+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:24.682 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:24.678+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:24.696 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:24.696 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:24.696 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:24.696 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:24.706 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:30:24.775 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-12-06T18:30:24.791 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:24.787+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:24.874 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:24.875 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:24.875 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:24.875 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:24.875 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:24.877 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:24.875+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:24.884 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:30:24.978 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:24.975+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:25.066 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:25.066 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:25.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:25.066 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:25.066 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:25.066 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:30:25.066 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:25.066 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 3.3932008659999999, 2023-12-06T18:30:25.066 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:30:25.066 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:25.067 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:25.067 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:25.067 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:25.067 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:25.067 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:30:25.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:25.067 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:25.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:25.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:25.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:25.067 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:25.067 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:25.067 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:25.067 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:25.067 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:25.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:25.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:25.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:25.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:25.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:30:25.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:30:25.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:25.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:25.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:25.068 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:30:25.068 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:30:25.068 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:25.068 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:25.068 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:25.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:30:25.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:30:25.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:25.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:25.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:25.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:30:25.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:30:25.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:25.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:25.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:25.069 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:30:25.069 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:30:25.069 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:25.069 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:25.069 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:25.070 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:30:25.070 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:30:25.070 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:25.070 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:25.070 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:25.070 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:25.070 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:25.070 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:25.078 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:30:25.079 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:25.077+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:25.180 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:25.178+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:25.231 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:25.231 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:25.231 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:25.231 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:25.240 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:30:25.282 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:25.279+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:25.383 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:25.380+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:25.444 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:25.445 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:25.445 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:25.445 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:25.445 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:25.456 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:30:25.484 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:25.480+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:25.585 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:25.582+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:25.634 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:25.634 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:25.634 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:25.634 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:30:25.634 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:25.643 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:30:25.686 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:25.684+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:25.807 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:25.804+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:25.822 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:25.822 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:25.822 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:25.822 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:25.831 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:30:25.916 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:25.913+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:25.988 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:25.986+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:26.008 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:26.008 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:26.008 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:26.008 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:26.008 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:26.017 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:30:26.102 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:26.099+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:26.191 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:26.187+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:26.254 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:26.255 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:26.255 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:26.255 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:26.255 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.255 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:30:26.255 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:26.255 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 210.842048957, 2023-12-06T18:30:26.255 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:30:26.255 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.255 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.255 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.255 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.255 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.255 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:30:26.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.256 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.256 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.256 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.256 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:26.256 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.256 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.256 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.256 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:26.256 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.256 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.256 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.256 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:30:26.256 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:30:26.257 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.257 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.257 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.257 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:30:26.257 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:30:26.257 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.257 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.257 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.257 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:30:26.257 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:30:26.257 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.257 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.257 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.257 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:30:26.258 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:30:26.258 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.258 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.258 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.258 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:30:26.258 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:30:26.258 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.258 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.258 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:30:26.258 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:30:26.258 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:30:26.258 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.258 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.258 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.259 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:30:26.259 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:30:26.259 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.259 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.259 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.259 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:30:26.259 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:30:26.259 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.259 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.259 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.259 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:30:26.259 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:30:26.259 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.259 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.260 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.260 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:30:26.260 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:30:26.260 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.260 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.260 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.260 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:30:26.260 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:30:26.260 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.260 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.260 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.260 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:30:26.260 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:30:26.260 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.260 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.260 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.261 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:30:26.261 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:30:26.261 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.261 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.261 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.261 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.261 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.261 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:26.261 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:26.261 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 208.73347571799999, 2023-12-06T18:30:26.261 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:30:26.261 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.261 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.261 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.261 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.261 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.262 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:30:26.262 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.262 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.262 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.262 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.262 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:26.262 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.262 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.262 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.262 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.262 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:26.262 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.262 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.262 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.262 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.263 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:30:26.263 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:30:26.263 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.263 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.263 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.263 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:30:26.263 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:30:26.263 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.263 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.263 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.263 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:30:26.263 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:30:26.263 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.263 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.263 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.263 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:30:26.264 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:30:26.264 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.264 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.264 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.264 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:30:26.264 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:30:26.264 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.264 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.264 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.264 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:30:26.264 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:30:26.264 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.264 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.264 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.264 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:30:26.265 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:30:26.265 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.265 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.265 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.265 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:30:26.265 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:30:26.265 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.265 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.265 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.265 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:30:26.265 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:30:26.265 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.265 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.265 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.265 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:30:26.266 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:30:26.266 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.266 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.266 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.266 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:30:26.266 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:30:26.266 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.266 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.266 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.266 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.266 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.266 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:26.266 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:26.266 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 208.73273048600001, 2023-12-06T18:30:26.267 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:30:26.267 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.267 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.267 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.267 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.267 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.267 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:30:26.267 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.267 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.267 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.267 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.267 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:26.267 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.267 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.268 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.268 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.268 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:26.268 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.268 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.268 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.268 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.268 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:30:26.268 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:30:26.268 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.268 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.268 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.268 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:30:26.268 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:30:26.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.269 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.269 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:30:26.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:30:26.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.269 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.269 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:30:26.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:30:26.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.269 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.270 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:30:26.270 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:30:26.270 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.270 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.270 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.270 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:30:26.270 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:30:26.270 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.270 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.270 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.270 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:30:26.270 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:30:26.270 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.270 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.271 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.271 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:30:26.271 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:30:26.271 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.271 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.271 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.271 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:30:26.271 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:30:26.271 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.271 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.271 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.271 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:30:26.271 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:30:26.272 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.272 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.272 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.272 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:30:26.272 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:30:26.272 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.272 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.272 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.272 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.272 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.272 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:26.272 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:26.272 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 208.732046731, 2023-12-06T18:30:26.272 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:30:26.273 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.273 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.273 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.273 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.273 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.273 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:30:26.273 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.273 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.273 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.273 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.273 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:26.273 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.273 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.273 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.273 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.274 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:26.274 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.274 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.274 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.274 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.274 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:30:26.274 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:30:26.274 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.274 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.274 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.274 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:30:26.274 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:30:26.275 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.275 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.275 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:30:26.275 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:30:26.275 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.275 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.275 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:30:26.275 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:30:26.275 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.275 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.275 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:30:26.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:30:26.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.276 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:30:26.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:30:26.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.276 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:30:26.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:30:26.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.276 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:30:26.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:30:26.277 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.277 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.277 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.277 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:30:26.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:30:26.277 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.277 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.277 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.277 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:30:26.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:30:26.277 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.277 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.277 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.277 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:30:26.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:30:26.278 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.278 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.278 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.278 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.278 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.278 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:26.278 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:26.278 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 208.726473476, 2023-12-06T18:30:26.278 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:30:26.278 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.278 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.278 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.278 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.278 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.278 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:30:26.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.279 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.279 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:26.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.279 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:26.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.279 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:30:26.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:30:26.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:30:26.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:30:26.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:30:26.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:30:26.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:30:26.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:30:26.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.281 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:30:26.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:30:26.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.281 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:30:26.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:30:26.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.281 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:30:26.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:30:26.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:30:26.282 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:30:26.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:30:26.282 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:30:26.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:30:26.282 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:30:26.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.283 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.283 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:30:26.283 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:30:26.283 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.283 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.283 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.283 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.283 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.283 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:26.283 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:26.283 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 208.72579983599999, 2023-12-06T18:30:26.283 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:30:26.283 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.283 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.283 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.283 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.284 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.284 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:30:26.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.284 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.284 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.284 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:26.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.284 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.284 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:26.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.285 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.285 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:30:26.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:30:26.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.285 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.285 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:30:26.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:30:26.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.285 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.285 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:30:26.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:30:26.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.286 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:30:26.286 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:30:26.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.286 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:30:26.286 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:30:26.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.286 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:30:26.286 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:30:26.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:30:26.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:30:26.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:30:26.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:30:26.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:30:26.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:30:26.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.288 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.288 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.288 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:30:26.288 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:30:26.288 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.288 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.288 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.288 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:30:26.288 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:30:26.288 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.288 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.288 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.288 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.288 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.288 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:26.289 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:26.289 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 208.72513763699999, 2023-12-06T18:30:26.289 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:30:26.289 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.289 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.289 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.289 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.289 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.289 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:30:26.289 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.289 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.289 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.289 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.289 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:26.290 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.290 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.290 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.290 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:26.290 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.290 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.290 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.290 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:30:26.290 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:30:26.290 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.290 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.291 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:30:26.291 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:30:26.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.291 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.291 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.291 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:30:26.291 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:30:26.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.291 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.291 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.291 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:30:26.291 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:30:26.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.291 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.292 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.292 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:30:26.292 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:30:26.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.292 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.292 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:30:26.292 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:30:26.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.292 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.292 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:30:26.293 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:30:26.293 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.293 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.293 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.293 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:30:26.293 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:30:26.293 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.293 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.293 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.293 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:30:26.293 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:30:26.293 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.293 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.293 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.294 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:30:26.294 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:30:26.294 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.294 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.294 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.294 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:30:26.294 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:30:26.294 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.294 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.294 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.294 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.294 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.294 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:26.295 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:26.295 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 208.65242820200001, 2023-12-06T18:30:26.295 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:30:26.295 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.295 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.295 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.295 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.295 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.295 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:30:26.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.295 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.295 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.295 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.295 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:26.296 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.296 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.296 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.296 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.296 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:26.296 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.296 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.296 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.296 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.296 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:30:26.296 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:30:26.296 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.296 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.297 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.297 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:30:26.297 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:30:26.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.297 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.297 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.297 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:30:26.297 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:30:26.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.297 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.297 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.297 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:30:26.297 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:30:26.298 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.298 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.298 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.298 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:30:26.298 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:30:26.298 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.298 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.298 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.298 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:30:26.298 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:30:26.298 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.298 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.298 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.298 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:30:26.299 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:30:26.299 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.299 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.299 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.299 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:30:26.299 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:30:26.299 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.299 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.299 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.299 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:30:26.299 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:30:26.299 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.300 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.300 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.300 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:30:26.300 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:30:26.300 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.300 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.300 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.300 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:30:26.300 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:30:26.300 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.300 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.300 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.301 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.301 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.301 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:26.301 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:26.301 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 208.65174649299999, 2023-12-06T18:30:26.301 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:30:26.301 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.301 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.301 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.301 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.301 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.301 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:30:26.301 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.301 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.301 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.302 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.302 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:26.302 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.302 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.302 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.302 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.302 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:26.302 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.302 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.302 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.302 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.302 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:30:26.302 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:30:26.302 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.302 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.303 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:30:26.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:30:26.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.303 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:30:26.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:30:26.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.303 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:30:26.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:30:26.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:30:26.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:30:26.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:30:26.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:30:26.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:30:26.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:30:26.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:30:26.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:30:26.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:30:26.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:30:26.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:30:26.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:30:26.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:30:26.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:30:26.306 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.306 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.306 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.306 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.306 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:26.306 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:26.306 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 207.57294240499999, 2023-12-06T18:30:26.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:30:26.306 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.306 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.306 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.307 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.307 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.307 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:30:26.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.307 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.307 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.307 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.307 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:26.307 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.307 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.307 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.308 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:26.308 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.308 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.308 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.308 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.308 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:30:26.308 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:30:26.308 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.308 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.308 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.308 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:30:26.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:30:26.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.309 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.309 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:30:26.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:30:26.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.309 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.309 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:30:26.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:30:26.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.309 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.309 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:30:26.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:30:26.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.310 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.310 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.310 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:30:26.310 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:30:26.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.310 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.310 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.310 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:30:26.310 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:30:26.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.310 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.310 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.310 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:30:26.310 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:30:26.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.311 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:30:26.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:30:26.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.311 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:30:26.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:30:26.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.311 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:30:26.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:30:26.312 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.312 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.312 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.312 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:26.312 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:26.312 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 207.571525831, 2023-12-06T18:30:26.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:30:26.312 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.312 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.312 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.312 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.312 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.313 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:30:26.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.313 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.313 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:26.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.313 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:26.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:30:26.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:30:26.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:30:26.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:30:26.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:30:26.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:30:26.315 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.315 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.315 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.315 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:30:26.315 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:30:26.315 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.315 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.317 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.317 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:30:26.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:30:26.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.317 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.317 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:30:26.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:30:26.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:30:26.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:30:26.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:30:26.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:30:26.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:30:26.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:30:26.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:30:26.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:30:26.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:30:26.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:30:26.320 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.320 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.320 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.320 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:26.320 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:26.320 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 207.570440706, 2023-12-06T18:30:26.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:30:26.320 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.321 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.321 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.321 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.321 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.321 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:30:26.321 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.321 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.321 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.321 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.321 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:26.321 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.321 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.322 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.322 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.322 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:26.322 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.322 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.322 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.322 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.322 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:30:26.322 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:30:26.322 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.322 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.322 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.323 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:30:26.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:30:26.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.323 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.323 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.323 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:30:26.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:30:26.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.323 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.323 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.323 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:30:26.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:30:26.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.324 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:30:26.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:30:26.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.324 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:30:26.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:30:26.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:30:26.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:30:26.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:30:26.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:30:26.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:30:26.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:30:26.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.326 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:30:26.326 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:30:26.326 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.326 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.326 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.326 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:30:26.326 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:30:26.326 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.326 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.326 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.326 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.326 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.326 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:26.326 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:26.327 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 207.53529160400001, 2023-12-06T18:30:26.327 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:30:26.327 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.327 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.327 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.327 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.327 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.327 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:30:26.327 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.327 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.327 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.327 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.327 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:26.327 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.327 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.328 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.328 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.328 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:26.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.328 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.328 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.328 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:30:26.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:30:26.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.328 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.328 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.328 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:30:26.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:30:26.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.329 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:30:26.329 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:30:26.329 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.329 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:30:26.329 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:30:26.329 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.329 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:30:26.329 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:30:26.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:30:26.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:30:26.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:30:26.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:30:26.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.331 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:30:26.331 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:30:26.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.331 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.331 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.331 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:30:26.331 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:30:26.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.331 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.331 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.331 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:30:26.331 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:30:26.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.331 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.331 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.332 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:30:26.332 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:30:26.332 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.332 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.332 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.332 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.332 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.332 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:26.332 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:26.332 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 200.98556107900001, 2023-12-06T18:30:26.332 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:30:26.332 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.332 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.332 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.332 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.333 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.333 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:30:26.333 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.333 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.333 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.333 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.333 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:26.333 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.333 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.333 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.333 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.333 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:26.333 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.333 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.333 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.334 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.334 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:30:26.334 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:30:26.334 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.334 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.334 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.334 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:30:26.334 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:30:26.334 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.334 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.334 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.334 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:30:26.335 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:30:26.335 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.335 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.335 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.335 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:30:26.335 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:30:26.335 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.335 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.335 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.335 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:30:26.335 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:30:26.335 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.335 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.335 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.335 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:30:26.336 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:30:26.336 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.336 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.336 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.336 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:30:26.336 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:30:26.336 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.336 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.336 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.336 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:30:26.336 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:30:26.336 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.336 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.336 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.337 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:30:26.337 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:30:26.337 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.337 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.337 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.337 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:30:26.337 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:30:26.337 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.337 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.337 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.337 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:30:26.337 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:30:26.337 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.337 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.338 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.338 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.338 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.338 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:26.338 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:26.338 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 200.984755471, 2023-12-06T18:30:26.338 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:30:26.338 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.338 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.338 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.338 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.338 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.338 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:30:26.339 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.339 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.339 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.339 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.339 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:26.339 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.339 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.339 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.339 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.339 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:26.339 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.339 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.339 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.339 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.339 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:30:26.340 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:30:26.340 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.340 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.340 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.340 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:30:26.340 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:30:26.340 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.340 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.340 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.340 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:30:26.340 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:30:26.340 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.340 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.341 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.341 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:30:26.341 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:30:26.341 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.341 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.341 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.341 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:30:26.341 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:30:26.341 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.341 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.341 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.341 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:30:26.341 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:30:26.342 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.342 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.342 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.342 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:30:26.342 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:30:26.342 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.342 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.342 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.342 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:30:26.342 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:30:26.342 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.342 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.342 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.342 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:30:26.343 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:30:26.343 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.343 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.343 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.343 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:30:26.343 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:30:26.343 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.343 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.343 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.343 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:30:26.343 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:30:26.343 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.343 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.343 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.344 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.344 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.344 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:26.344 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:26.344 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 200.98406466200001, 2023-12-06T18:30:26.344 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:30:26.344 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.344 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.344 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.344 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.344 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.344 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:30:26.344 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.344 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.344 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.345 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.345 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:26.345 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.345 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.345 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.345 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.345 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:26.345 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.345 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.345 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.345 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.345 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:30:26.345 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:30:26.345 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.345 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.345 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.346 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:30:26.346 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:30:26.346 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.346 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.346 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.346 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:30:26.346 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:30:26.346 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.346 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.346 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.346 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:30:26.346 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:30:26.346 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.346 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.347 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.347 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:30:26.347 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:30:26.347 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.347 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.347 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.347 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:30:26.347 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:30:26.347 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.347 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.347 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.347 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:30:26.347 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:30:26.347 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.347 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.347 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.347 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:30:26.348 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:30:26.348 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.348 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.348 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.348 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:30:26.348 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:30:26.348 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.348 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.348 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.348 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:30:26.348 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:30:26.348 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.348 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.348 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.348 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:30:26.349 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:30:26.349 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.349 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.349 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.349 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.349 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.349 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:26.349 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:26.349 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 200.979767154, 2023-12-06T18:30:26.349 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:30:26.349 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.349 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.349 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.349 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.349 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.349 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:30:26.350 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.350 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.350 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.350 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.350 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:26.350 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.350 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.350 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.350 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.350 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:26.350 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.350 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.350 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.351 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:30:26.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:30:26.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.351 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:30:26.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:30:26.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.351 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:30:26.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:30:26.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:30:26.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:30:26.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:30:26.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:30:26.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:30:26.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:30:26.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:30:26.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:30:26.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:30:26.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:30:26.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:30:26.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:30:26.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:30:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:30:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:30:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:30:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.354 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 200.97909947900001, 2023-12-06T18:30:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:30:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:30:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.355 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:30:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:30:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.356 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:30:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:30:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:30:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:30:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:30:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:30:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:30:26.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:30:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:30:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:30:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:30:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:30:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.358 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:30:26.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:30:26.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.359 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.359 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.359 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:30:26.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:30:26.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.359 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.359 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.359 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:30:26.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:30:26.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.359 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.359 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.359 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:30:26.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:30:26.360 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.360 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.360 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.360 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.360 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.360 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:26.360 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:26.360 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 200.978444051, 2023-12-06T18:30:26.360 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:30:26.360 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.360 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.360 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.360 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.360 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.361 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:30:26.361 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.361 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.361 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.361 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.361 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:26.361 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.361 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.361 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.361 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.361 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:26.361 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.361 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.361 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.361 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.361 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:30:26.362 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:30:26.362 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.362 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.362 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:30:26.362 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:30:26.362 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.362 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.362 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:30:26.362 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:30:26.362 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.362 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.362 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:30:26.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:30:26.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:30:26.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:30:26.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:30:26.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:30:26.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:30:26.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:30:26.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:30:26.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:30:26.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:30:26.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:30:26.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:30:26.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:30:26.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:30:26.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:30:26.365 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.365 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.365 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.365 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.365 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:26.365 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:26.365 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 200.93778950399999, 2023-12-06T18:30:26.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:30:26.365 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.365 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:26.365 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:26.365 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:26.365 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:26.365 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:30:26.366 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.366 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.366 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.366 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.366 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:26.366 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.366 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.366 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.366 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.366 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:26.366 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.366 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.366 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.367 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.367 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:30:26.367 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:30:26.367 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.367 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.367 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.367 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:30:26.367 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:30:26.367 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.367 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.367 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.367 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:30:26.367 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:30:26.367 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.368 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.368 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:30:26.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:30:26.368 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.368 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.368 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:30:26.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:30:26.368 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.368 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.368 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:30:26.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:30:26.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.369 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.369 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:26.369 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:30:26.369 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:30:26.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.369 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.369 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:26.369 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:30:26.369 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:30:26.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.369 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.369 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:26.370 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:30:26.370 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:30:26.370 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.370 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.370 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:26.370 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:30:26.370 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:30:26.370 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.370 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.370 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.370 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:30:26.370 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:30:26.370 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.370 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.371 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.371 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.371 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.371 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:26.372 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:30:26.373 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:26.369+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:26.474 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:26.471+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:26.549 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:26.550 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:26.550 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:26.550 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:26.558 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:30:26.574 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:26.571+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:26.675 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:26.673+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:26.735 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:26.735 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:26.736 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:26.736 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:26.736 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:26.744 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:30:26.787 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:26.783+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:26.879 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:26.876+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:26.941 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:30:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 185.12490486999999, 2023-12-06T18:30:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:30:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:26.941 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:30:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:30:26.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:30:26.943 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.943 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:26.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:30:26.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:30:26.943 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.943 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:26.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:30:26.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:30:26.943 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.943 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:26.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:30:26.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:30:26.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.944 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.944 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:26.944 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:30:26.944 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:30:26.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.944 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.944 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:26.944 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:30:26.944 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:30:26.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:26.944 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:26.944 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:26.944 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:30:26.945 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:30:26.945 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.945 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.945 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.945 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:26.945 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:26.945 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:26.950 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:30:26.994 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:26.992+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:27.080 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:27.076+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:27.087 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:27.087 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:27.087 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:27.088 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:27.097 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:30:27.180 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:27.177+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:27.273 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:27.273 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:27.273 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:27.273 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:27.273 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:27.281 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:27.278+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:27.282 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:30:27.382 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:27.379+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:27.423 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:27.423 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:27.423 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:27.423 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:27.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:27.424 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:27.424 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:27.424 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 66.323835697000007, 2023-12-06T18:30:27.424 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:30:27.424 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:27.424 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:27.424 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:27.424 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:27.424 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:27.424 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:30:27.424 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:27.424 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:27.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:27.424 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:27.424 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:27.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:27.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:27.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:27.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:27.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:27.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:27.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:27.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:27.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:27.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:30:27.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:30:27.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:27.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:27.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:27.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:30:27.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:30:27.426 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:27.426 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:27.426 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:27.426 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:30:27.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:30:27.426 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:27.426 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:27.426 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:27.426 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:30:27.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:30:27.426 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:27.426 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:27.426 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:27.427 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:30:27.427 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:30:27.427 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:27.427 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:27.427 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:27.427 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:30:27.427 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:30:27.427 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:27.427 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:27.427 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:27.427 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:27.427 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:27.427 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:30:27.427 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:27.427 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 6.3055783950000004, 2023-12-06T18:30:27.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:30:27.428 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:27.428 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:27.428 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:27.428 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:27.428 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:27.428 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:30:27.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:27.428 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:27.428 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:27.428 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:27.428 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:27.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:27.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:27.428 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:27.429 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:27.429 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:27.429 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:27.429 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:27.429 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:27.429 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:27.429 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:30:27.429 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:30:27.429 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:27.429 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:27.429 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:27.429 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:30:27.429 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:30:27.429 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:27.429 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:27.429 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:27.430 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:30:27.430 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:30:27.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:27.430 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:27.430 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:27.430 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:30:27.430 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:30:27.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:27.430 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:27.430 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:27.430 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:30:27.430 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:30:27.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:27.430 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:27.431 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:27.431 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:30:27.431 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:30:27.431 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:27.431 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:27.431 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:27.431 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:27.431 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:27.431 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:27.433 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:30:27.483 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:27.480+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:27.576 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:27.576 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:27.576 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:27.576 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:27.584 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:27.580+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:27.585 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:30:27.685 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:27.681+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:27.754 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:27.754 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:27.754 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:27.754 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:27.754 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:27.763 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:30:27.785 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:27.783+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:27.886 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:27.884+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:27.938 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:27.938 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:27.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:27.938 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:27.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:27.938 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:27.938 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:27.938 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 126.444659801, 2023-12-06T18:30:27.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:30:27.939 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:27.939 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:27.939 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:27.939 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:27.939 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:27.939 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:30:27.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:27.939 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:27.939 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:27.939 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:27.939 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:27.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:27.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:27.939 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:27.939 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:27.939 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:27.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:27.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:27.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:27.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:27.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:30:27.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:30:27.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:27.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:27.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:27.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:30:27.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:30:27.940 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:27.940 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:27.940 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:27.940 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:30:27.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:30:27.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:27.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:27.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:27.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:30:27.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:30:27.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:27.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:27.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:27.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:30:27.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:30:27.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:27.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:27.941 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:27.941 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:30:27.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:30:27.942 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:27.942 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:27.942 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:27.942 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:27.942 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:27.942 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:27.950 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:30:27.993 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:27.990+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:28.078 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:28.078 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:28.078 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:28.078 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:28.088 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:30:28.095 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:28.092+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:28.189 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:28.186+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:28.256 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:28.256 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:28.257 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:28.257 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:28.257 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:28.267 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:30:28.289 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:28.287+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:28.390 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:28.388+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:28.436 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:28.436 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:28.436 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:28.436 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:28.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.436 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:30:28.436 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:28.437 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 211.82114958, 2023-12-06T18:30:28.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:30:28.437 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.437 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.437 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:28.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:28.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:30:28.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:30:28.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:30:28.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:30:28.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:30:28.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:30:28.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:30:28.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:30:28.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:30:28.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:30:28.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:30:28.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:30:28.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:30:28.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:30:28.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:30:28.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:30:28.440 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.440 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.440 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.441 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:30:28.441 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:28.441 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 210.798322571, 2023-12-06T18:30:28.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:30:28.441 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.441 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.441 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:28.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:28.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:30:28.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:30:28.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:30:28.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:30:28.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:30:28.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:30:28.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:30:28.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:30:28.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:30:28.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:30:28.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:30:28.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:30:28.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:30:28.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:30:28.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:30:28.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:30:28.444 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.444 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.444 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.445 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:30:28.445 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:28.445 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 210.78627593300001, 2023-12-06T18:30:28.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:30:28.445 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.445 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.445 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:28.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:28.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:30:28.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:30:28.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:30:28.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:30:28.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:30:28.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:30:28.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:30:28.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:30:28.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:30:28.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:30:28.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:30:28.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:30:28.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.448 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:30:28.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:30:28.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.448 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:30:28.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:30:28.448 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.448 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.448 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.448 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:30:28.449 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:28.449 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 210.76487532600001, 2023-12-06T18:30:28.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:30:28.449 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.449 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.449 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:28.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:28.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:30:28.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:30:28.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:30:28.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:30:28.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:30:28.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:30:28.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:30:28.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:30:28.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:30:28.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:30:28.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:30:28.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:30:28.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:30:28.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:30:28.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:30:28.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:30:28.453 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.453 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.453 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.453 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:30:28.453 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:28.453 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 210.76369600300001, 2023-12-06T18:30:28.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:30:28.453 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.453 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.453 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:28.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:28.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:30:28.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:30:28.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:30:28.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:30:28.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:30:28.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:30:28.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:30:28.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:30:28.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:30:28.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:30:28.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:30:28.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:30:28.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:30:28.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:30:28.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:30:28.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:30:28.457 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.457 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.457 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.457 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:30:28.457 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:28.457 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 210.763124022, 2023-12-06T18:30:28.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:30:28.457 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.457 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.457 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:28.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:28.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:30:28.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:30:28.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:30:28.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:30:28.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:30:28.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:30:28.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:30:28.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:30:28.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:30:28.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:30:28.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:30:28.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:30:28.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:30:28.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:30:28.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:30:28.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:30:28.461 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.461 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.461 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.461 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:30:28.461 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:28.461 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 210.31728855200001, 2023-12-06T18:30:28.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:30:28.461 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.461 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.461 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:28.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:28.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:30:28.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:30:28.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:30:28.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:30:28.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:30:28.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:30:28.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:30:28.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:30:28.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:30:28.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:30:28.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:30:28.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:30:28.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:30:28.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:30:28.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:30:28.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:30:28.465 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.465 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.465 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.465 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:30:28.465 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:28.465 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 210.315688696, 2023-12-06T18:30:28.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:30:28.466 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.466 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.466 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:28.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:28.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:30:28.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:30:28.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:30:28.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:30:28.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:30:28.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:30:28.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:30:28.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:30:28.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:30:28.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:30:28.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:30:28.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:30:28.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:30:28.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:30:28.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:30:28.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:30:28.469 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.469 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.469 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.470 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:30:28.470 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:28.470 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 209.93648307999999, 2023-12-06T18:30:28.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:30:28.470 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.470 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.470 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.470 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.470 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:28.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:28.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:30:28.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:30:28.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:30:28.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:30:28.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:30:28.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:30:28.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:30:28.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:30:28.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:30:28.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:30:28.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:30:28.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:30:28.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:30:28.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:30:28.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:30:28.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:30:28.473 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.474 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.474 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.474 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:30:28.474 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:28.474 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 209.698980898, 2023-12-06T18:30:28.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:30:28.474 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.474 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.474 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:28.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:28.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:30:28.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:30:28.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:30:28.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:30:28.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:30:28.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:30:28.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:30:28.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:30:28.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:30:28.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:30:28.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:30:28.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:30:28.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:30:28.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:30:28.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:30:28.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:30:28.478 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.478 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.478 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.478 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:30:28.478 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:28.478 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 209.69825289900001, 2023-12-06T18:30:28.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:30:28.479 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.479 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.479 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:28.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:28.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:30:28.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:30:28.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:30:28.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:30:28.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:30:28.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:30:28.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:30:28.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:30:28.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:30:28.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:30:28.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:30:28.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:30:28.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:30:28.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:30:28.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:30:28.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:30:28.482 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.482 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.482 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.483 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:30:28.483 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:28.483 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 209.693240489, 2023-12-06T18:30:28.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:30:28.483 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.483 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.483 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:28.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:28.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:30:28.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:30:28.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:30:28.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:30:28.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:30:28.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:30:28.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:30:28.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:30:28.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:30:28.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:30:28.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:30:28.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:30:28.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:30:28.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:30:28.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:30:28.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:30:28.486 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.486 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.486 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.487 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:30:28.487 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:28.487 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 209.68771340999999, 2023-12-06T18:30:28.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:30:28.487 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.487 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.487 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.487 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.487 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:28.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.487 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:28.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:30:28.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:30:28.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:30:28.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:30:28.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:30:28.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:30:28.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:30:28.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:30:28.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:30:28.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:30:28.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:30:28.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:30:28.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:30:28.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:30:28.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.491 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:30:28.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:30:28.491 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.491 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.491 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.491 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.491 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:30:28.491 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:28.491 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 209.68720257300001, 2023-12-06T18:30:28.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:30:28.491 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.491 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.491 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.491 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.491 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:28.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:28.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:30:28.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:30:28.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:30:28.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:30:28.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:30:28.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:30:28.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:30:28.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:30:28.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:30:28.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:30:28.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:30:28.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:30:28.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:30:28.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:30:28.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:30:28.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:30:28.495 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.495 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.495 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.495 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:30:28.495 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:28.495 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 209.686743312, 2023-12-06T18:30:28.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:30:28.495 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.495 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.495 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:28.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:28.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:30:28.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:30:28.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:30:28.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:30:28.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:30:28.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:30:28.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:30:28.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:30:28.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:30:28.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:30:28.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:30:28.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:30:28.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:30:28.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:30:28.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.499 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.499 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:30:28.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:30:28.499 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.499 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.499 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.499 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:30:28.499 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:28.499 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 203.10682750500001, 2023-12-06T18:30:28.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:30:28.499 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.500 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.500 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:28.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:28.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:30:28.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:30:28.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:30:28.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:30:28.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:30:28.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:30:28.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:30:28.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:30:28.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:30:28.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:30:28.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:30:28.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:30:28.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:30:28.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:30:28.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:30:28.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:30:28.503 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.503 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.503 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.504 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:30:28.504 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:28.504 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 203.1005897, 2023-12-06T18:30:28.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:30:28.504 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.504 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.504 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:28.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:28.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:30:28.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:30:28.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:30:28.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:30:28.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:30:28.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:30:28.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:30:28.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:30:28.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:30:28.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:30:28.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:30:28.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:30:28.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:30:28.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:30:28.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:30:28.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:30:28.507 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.507 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.508 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.508 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:30:28.508 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:28.508 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 203.094526621, 2023-12-06T18:30:28.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:30:28.508 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.508 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.508 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:28.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:28.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:30:28.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:30:28.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:30:28.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:30:28.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:30:28.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:30:28.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:30:28.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:30:28.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:30:28.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:30:28.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:30:28.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:30:28.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:30:28.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:30:28.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:30:28.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:30:28.511 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.511 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.512 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.512 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:30:28.512 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:28.512 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 202.957139536, 2023-12-06T18:30:28.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:30:28.512 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.512 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.512 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:28.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:28.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:30:28.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:30:28.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:30:28.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:30:28.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:30:28.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:30:28.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:30:28.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:30:28.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:30:28.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:30:28.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:30:28.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:30:28.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:30:28.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:30:28.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:30:28.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:30:28.516 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.516 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.516 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.516 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:30:28.516 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:28.516 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 202.950468809, 2023-12-06T18:30:28.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:30:28.516 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:28.516 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:28.516 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:28.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:28.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:28.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:28.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:28.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:28.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:28.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:30:28.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:30:28.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:28.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:30:28.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:30:28.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:28.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:30:28.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:30:28.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:28.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:30:28.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:30:28.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:28.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:30:28.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:30:28.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:28.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:30:28.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:30:28.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:28.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:30:28.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:30:28.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:28.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:28.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:28.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:30:28.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:30:28.520 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.520 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.520 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.520 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:28.520 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:28.520 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:28.522 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:30:28.522 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:28.518+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:28.622 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:28.620+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:28.690 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:28.690 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:28.691 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:28.691 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:28.700 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:30:28.723 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:28.721+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:28.824 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:28.822+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:28.827 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:28.827 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:28.827 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:28.827 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:28.827 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:28.836 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:30:28.848 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:30:28.848 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:30:28.848 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2023-12-06T18:30:28.848 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2023-12-06T18:30:28.848 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2023-12-06T18:30:28.848 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2023-12-06T18:30:28.925 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:28.922+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:28.967 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:28.968 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:28.968 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:28.968 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:30:28.968 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:28.977 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:30:29.015 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:29.015 INFO:teuthology.orchestra.run.smithi152.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2023-12-06T18:30:29.016 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:29.026 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:29.023+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:29.108 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:29.109 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:29.109 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:29.109 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:29.119 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:30:29.126 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:29.124+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:29.227 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:29.224+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:29.246 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:29.246 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:29.246 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:29.246 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:29.246 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:29.256 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:30:29.328 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:29.326+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:29.389 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:29.389 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:29.390 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:29.390 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:29.390 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:29.390 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:30:29.390 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:29.390 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 7.727485776, 2023-12-06T18:30:29.390 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:30:29.390 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:29.390 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:29.390 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:29.390 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:29.391 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:29.391 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:30:29.391 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:29.391 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:29.391 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:29.391 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:29.391 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:29.391 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:29.391 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:29.391 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:29.391 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:29.392 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:29.392 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:29.392 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:29.392 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:29.392 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:29.392 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:30:29.392 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:30:29.392 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:29.392 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:29.392 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:29.392 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:30:29.392 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:30:29.392 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:29.392 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:29.392 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:29.392 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:30:29.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:30:29.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:29.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:29.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:29.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:30:29.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:30:29.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:29.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:29.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:29.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:30:29.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:30:29.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:29.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:29.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:29.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:30:29.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:30:29.394 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:29.394 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:29.394 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:29.394 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:29.394 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:29.394 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:29.399 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:30:29.430 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:29.426+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:29.530 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:29.528+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:29.534 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:29.535 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:29.535 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:29.535 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:29.544 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:30:29.631 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:29.629+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:29.721 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:29.721 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:29.722 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:29.722 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:29.722 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:29.730 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:30:29.732 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:29.729+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:29.834 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:29.831+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:29.909 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:29.909 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:29.909 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:29.910 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:30:29.910 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:29.918 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:30:29.961 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:29.957+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:30.035 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:30.032+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:30.094 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:30.094 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:30.094 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:30.095 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:30.103 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:30:30.146 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:30.142+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:30.237 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:30.234+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:30.280 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:30.280 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:30.280 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:30.280 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:30.281 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:30.289 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:30:30.338 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:30.334+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:30.451 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:30.448+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:30.524 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:30.524 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:30.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:30.525 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:30.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.525 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:30:30.525 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:30.525 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 215.113832941, 2023-12-06T18:30:30.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:30:30.525 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.525 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.525 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.525 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.525 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.525 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:30:30.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.526 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:30.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:30.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:30:30.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:30:30.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:30:30.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:30:30.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:30:30.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:30:30.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:30:30.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:30:30.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:30:30.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:30:30.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:30:30.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:30:30.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:30:30.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:30:30.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:30:30.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:30:30.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:30:30.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:30:30.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:30:30.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:30:30.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:30:30.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:30:30.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:30:30.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:30:30.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:30:30.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:30:30.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:30:30.534 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.534 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.534 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.535 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:30.535 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:30.535 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 213.00525970199999, 2023-12-06T18:30:30.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:30:30.535 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.536 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.536 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.536 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.536 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.536 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:30:30.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.537 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:30.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.538 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:30.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:30:30.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:30:30.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:30:30.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:30:30.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:30:30.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:30:30.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:30:30.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:30:30.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:30:30.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:30:30.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:30:30.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:30:30.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:30:30.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:30:30.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:30:30.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:30:30.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:30:30.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:30:30.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:30:30.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:30:30.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:30:30.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:30:30.546 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.546 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.546 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.547 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:30.547 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:30.547 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 213.00451447, 2023-12-06T18:30:30.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:30:30.547 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.547 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.547 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.548 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.548 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:30:30.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.548 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:30.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:30.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:30:30.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:30:30.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:30:30.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:30:30.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:30:30.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:30:30.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:30:30.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:30:30.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:30:30.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:30:30.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:30:30.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:30:30.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:30:30.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:30:30.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:30:30.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:30:30.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:30:30.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:30:30.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:30:30.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:30:30.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:30:30.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:30:30.556 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.556 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.556 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.556 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:30.556 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:30.556 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 213.00383071499999, 2023-12-06T18:30:30.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:30:30.556 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.556 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.556 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.556 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.556 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.556 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:30:30.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.557 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:30.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:30.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:30:30.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:30:30.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:30:30.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:30:30.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:30:30.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:30:30.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:30:30.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:30:30.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:30:30.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:30:30.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:30:30.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:30:30.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:30:30.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:30:30.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:30:30.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:30:30.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:30:30.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:30:30.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:30:30.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:30:30.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:30:30.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:30:30.561 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.561 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.561 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.561 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:30.561 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:30.561 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 212.99825745999999, 2023-12-06T18:30:30.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:30:30.562 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.562 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.562 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.562 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.562 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.562 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:30:30.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.562 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:30.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:30.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:30:30.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:30:30.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:30:30.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:30:30.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:30:30.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:30:30.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:30:30.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:30:30.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:30:30.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:30:30.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:30:30.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:30:30.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:30:30.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:30:30.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:30:30.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:30:30.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:30:30.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:30:30.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.567 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:30:30.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:30:30.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.567 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.567 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:30:30.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:30:30.567 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.567 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.567 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.567 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:30.567 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:30.567 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 212.99758381999999, 2023-12-06T18:30:30.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:30:30.568 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.568 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.568 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.568 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.568 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.568 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:30:30.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.568 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.568 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.568 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:30.568 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.568 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:30.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:30:30.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:30:30.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:30:30.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:30:30.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.570 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:30:30.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:30:30.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.570 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.570 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:30:30.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:30:30.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.570 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.570 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:30:30.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:30:30.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:30:30.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:30:30.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:30:30.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:30:30.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:30:30.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:30:30.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.572 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:30:30.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:30:30.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.572 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:30:30.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:30:30.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.572 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:30:30.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:30:30.572 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.572 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.573 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.573 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:30.573 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:30.573 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 212.99692162100001, 2023-12-06T18:30:30.573 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:30:30.573 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.573 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.573 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.573 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.573 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.573 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:30:30.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.573 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:30.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.574 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:30.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.574 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:30:30.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:30:30.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.575 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.575 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:30:30.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:30:30.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.576 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.576 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:30:30.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:30:30.576 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.576 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.576 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.576 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:30:30.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:30:30.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.577 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.577 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:30:30.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:30:30.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.577 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.577 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:30:30.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:30:30.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:30:30.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:30:30.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:30:30.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:30:30.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:30:30.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:30:30.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.579 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.579 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.579 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:30:30.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:30:30.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.579 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.579 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.579 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:30:30.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:30:30.579 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.579 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.579 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.579 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.580 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:30.580 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:30.580 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 212.92421218600001, 2023-12-06T18:30:30.580 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:30:30.580 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.580 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.580 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.580 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.580 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.580 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:30:30.580 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.580 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.580 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.580 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:30.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.581 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.581 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:30.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.581 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.581 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:30:30.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:30:30.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.581 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.581 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:30:30.582 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:30:30.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.582 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.582 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:30:30.582 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:30:30.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.582 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.582 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:30:30.582 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:30:30.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.583 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:30:30.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:30:30.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.583 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:30:30.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:30:30.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.583 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:30:30.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:30:30.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.584 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:30:30.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:30:30.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.584 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:30:30.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:30:30.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.584 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:30:30.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:30:30.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:30:30.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:30:30.585 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.585 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.585 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.585 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.585 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:30.585 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:30.585 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 212.92353047700001, 2023-12-06T18:30:30.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:30:30.585 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.585 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.585 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.585 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.586 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.586 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:30:30.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.586 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:30.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:30.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:30:30.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:30:30.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:30:30.588 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:30:30.588 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.588 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.588 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.588 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:30:30.588 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:30:30.589 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.589 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.589 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:30:30.589 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:30:30.589 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.590 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.590 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.590 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:30:30.590 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:30:30.590 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.590 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.591 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.591 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:30:30.591 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:30:30.591 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.591 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.591 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.592 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:30:30.592 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:30:30.592 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.592 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.592 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.592 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:30:30.593 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:30:30.593 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.593 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.593 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:30:30.593 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:30:30.594 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.594 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.594 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.594 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:30:30.594 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:30:30.594 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.595 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.595 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.595 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:30:30.595 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:30:30.595 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.595 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.595 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.595 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.596 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.596 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:30.596 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:30.596 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 211.84472638899999, 2023-12-06T18:30:30.596 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:30:30.596 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.596 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.597 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.597 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.597 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.597 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:30:30.597 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.597 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.597 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.598 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:30.598 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.598 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:30.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.599 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:30:30.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:30:30.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.600 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.600 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.600 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:30:30.600 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:30:30.600 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.600 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.600 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.600 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:30:30.601 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:30:30.601 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.601 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.601 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.601 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:30:30.601 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:30:30.602 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.602 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.602 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.602 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:30:30.602 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:30:30.602 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.603 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.603 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.603 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:30:30.603 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:30:30.603 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.603 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.604 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.604 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:30:30.604 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:30:30.604 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.604 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.604 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.604 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:30:30.604 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:30:30.604 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.604 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.605 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.605 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:30:30.605 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:30:30.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.605 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.605 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.605 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:30:30.605 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:30:30.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.606 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.606 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.606 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:30:30.606 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:30:30.606 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.606 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.606 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.606 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.607 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.607 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:30.607 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:30.607 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 211.843309815, 2023-12-06T18:30:30.607 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:30:30.607 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.607 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.607 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.608 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.608 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.608 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:30:30.608 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.608 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.608 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.608 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.609 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:30.609 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.609 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.609 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.609 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:30.609 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.610 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.610 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.610 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:30:30.610 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:30:30.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.611 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.611 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.611 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:30:30.611 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:30:30.611 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.611 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.611 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.611 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:30:30.611 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:30:30.611 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.612 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.612 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.612 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:30:30.612 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:30:30.612 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.612 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.612 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.612 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:30:30.612 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:30:30.612 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.612 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.612 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.612 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:30:30.612 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:30:30.612 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.613 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.613 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.613 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:30:30.613 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:30:30.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.613 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.613 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.613 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:30:30.613 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:30:30.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.613 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.613 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.613 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:30:30.613 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:30:30.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.614 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:30:30.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:30:30.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.614 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:30:30.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:30:30.614 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.614 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.614 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.614 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:30.615 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:30.615 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 211.84222468999999, 2023-12-06T18:30:30.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:30:30.615 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.615 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.615 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.615 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.615 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.615 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:30:30.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.615 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.615 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:30.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:30.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:30:30.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:30:30.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:30:30.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:30:30.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:30:30.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:30:30.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:30:30.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:30:30.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:30:30.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:30:30.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:30:30.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:30:30.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:30:30.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:30:30.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:30:30.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:30:30.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.619 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.619 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:30:30.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:30:30.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.619 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.619 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:30:30.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:30:30.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.619 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.619 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:30:30.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:30:30.620 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.620 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.620 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.620 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.620 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.620 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:30.620 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:30.620 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 211.807075588, 2023-12-06T18:30:30.620 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:30:30.620 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.620 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.620 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.620 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.620 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.620 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:30:30.621 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.621 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.621 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.621 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.621 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:30.621 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.621 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.621 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.621 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.621 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:30.621 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.621 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.622 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.622 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.622 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:30:30.622 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:30:30.622 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.622 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.622 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.622 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:30:30.622 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:30:30.622 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.622 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.623 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.623 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:30:30.623 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:30:30.623 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.623 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.623 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.623 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:30:30.623 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:30:30.623 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.623 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.623 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.623 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:30:30.623 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:30:30.623 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.623 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.623 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.624 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:30:30.624 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:30:30.624 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.624 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.624 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.624 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:30:30.624 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:30:30.624 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.624 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.624 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.624 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:30:30.624 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:30:30.624 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.624 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.624 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.624 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:30:30.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:30:30.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:30:30.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:30:30.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:30:30.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:30:30.625 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.625 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.625 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.626 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.626 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:30.626 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:30.626 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 205.257345063, 2023-12-06T18:30:30.626 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:30:30.626 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.626 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.626 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.626 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.626 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.626 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:30:30.626 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.626 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.626 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.627 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:30.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.627 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:30.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.627 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:30:30.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:30:30.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:30:30.628 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:30:30.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.628 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.628 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:30:30.628 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:30:30.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.628 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.628 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:30:30.628 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:30:30.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.628 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.628 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.629 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:30:30.629 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:30:30.629 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.629 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.629 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.629 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:30:30.629 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:30:30.629 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.629 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.629 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.629 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:30:30.629 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:30:30.629 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.629 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.629 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.629 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:30:30.630 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:30:30.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.630 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.630 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.630 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:30:30.630 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:30:30.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.630 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.630 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.630 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:30:30.630 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:30:30.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.630 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.631 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.631 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:30:30.631 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:30:30.631 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.631 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.631 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.631 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.631 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:30.631 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:30.631 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 205.256539455, 2023-12-06T18:30:30.631 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:30:30.631 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.631 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.631 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.632 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.632 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.632 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:30:30.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.632 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.632 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.632 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:30.632 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.633 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.633 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:30.633 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.633 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.633 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.633 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.634 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:30:30.634 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:30:30.634 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.634 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.634 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.634 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:30:30.634 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:30:30.634 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.634 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.634 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.634 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:30:30.635 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:30:30.635 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.635 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.635 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.635 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:30:30.635 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:30:30.635 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.635 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.635 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.635 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:30:30.635 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:30:30.635 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.635 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.635 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.635 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:30:30.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:30:30.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:30:30.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:30:30.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:30:30.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:30:30.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:30:30.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:30:30.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:30:30.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:30:30.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:30:30.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:30:30.637 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.637 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.637 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.638 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:30.638 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:30.638 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 205.255848646, 2023-12-06T18:30:30.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:30:30.638 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.638 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.638 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.638 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.638 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.638 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:30:30.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.638 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:30.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:30.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:30:30.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:30:30.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:30:30.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:30:30.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:30:30.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:30:30.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:30:30.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:30:30.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:30:30.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:30:30.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:30:30.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:30:30.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:30:30.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:30:30.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:30:30.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:30:30.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.642 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.642 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.642 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:30:30.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:30:30.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.642 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.642 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.642 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:30:30.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:30:30.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.642 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.642 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.643 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:30:30.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:30:30.643 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.643 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.643 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.643 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.643 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.643 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:30.643 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:30.643 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 205.251551138, 2023-12-06T18:30:30.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:30:30.643 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.643 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.643 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.643 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.643 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.644 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:30:30.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.644 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:30.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:30.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.645 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.645 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:30:30.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:30:30.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:30:30.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:30:30.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.647 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.647 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:30:30.647 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:30:30.647 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.647 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.648 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.648 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:30:30.648 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:30:30.648 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.648 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.648 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.649 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:30:30.649 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:30:30.649 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.649 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.649 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.649 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:30:30.650 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:30:30.650 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.650 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.650 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:30:30.651 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:30:30.651 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.651 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.651 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.651 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:30:30.651 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:30:30.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.652 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:30:30.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:30:30.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.653 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.653 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:30:30.653 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:30:30.653 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.653 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.653 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.653 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:30:30.654 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:30:30.654 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.654 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.654 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.654 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.654 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.654 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:30.655 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:30.655 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 205.25088346300001, 2023-12-06T18:30:30.655 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:30:30.655 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.655 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.655 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.655 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.656 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.656 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:30:30.656 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.656 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.656 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.656 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.656 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:30.656 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.656 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.657 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.657 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.657 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:30.657 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.657 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.657 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.657 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:30:30.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:30:30.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.658 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:30:30.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:30:30.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.659 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:30:30.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:30:30.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.660 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:30:30.660 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:30:30.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.660 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:30:30.661 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:30:30.661 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.661 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.661 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.661 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:30:30.662 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:30:30.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.662 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.662 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.662 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:30:30.662 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:30:30.662 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.662 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.663 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.663 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:30:30.663 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:30:30.663 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.663 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.663 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.663 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:30:30.663 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:30:30.663 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.664 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.664 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.664 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:30:30.664 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:30:30.664 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.664 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.664 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.664 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:30:30.664 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:30:30.665 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.665 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.665 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.665 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.665 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.665 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:30.665 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:30.665 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 205.25022803499999, 2023-12-06T18:30:30.665 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:30:30.666 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.666 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.666 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.666 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.666 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.666 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:30:30.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.667 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.667 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.667 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.667 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:30.667 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.668 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.668 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:30.668 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.668 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.668 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.669 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:30:30.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:30:30.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.669 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.669 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:30:30.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:30:30.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.670 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.670 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:30:30.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:30:30.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.670 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.670 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:30:30.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:30:30.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.670 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.670 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:30:30.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:30:30.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:30:30.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:30:30.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:30:30.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:30:30.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:30:30.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:30:30.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:30:30.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:30:30.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:30:30.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:30:30.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:30:30.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:30:30.673 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.673 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.673 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.673 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.673 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.673 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:30.673 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:30.673 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 205.20957348799999, 2023-12-06T18:30:30.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:30:30.673 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:30.673 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:30.673 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:30.673 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:30.674 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:30.674 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:30:30.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.674 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:30.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:30.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:30.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:30.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:30.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:30.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:30.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:30:30.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:30:30.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:30.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:30:30.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:30:30.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:30.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:30:30.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:30:30.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.676 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:30.676 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:30:30.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:30:30.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.676 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.676 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:30.676 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:30:30.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:30:30.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.676 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.676 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:30.676 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:30:30.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:30:30.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.676 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:30.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:30:30.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:30:30.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:30.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:30:30.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:30:30.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:30.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:30:30.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:30:30.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:30.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:30:30.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:30:30.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:30.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:30.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:30.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:30:30.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:30:30.678 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.678 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.678 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.679 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:30.679 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:30.679 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:30.680 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:30:30.681 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:30.677+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:30.782 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:30.778+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:30.859 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:30.859 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:30.859 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:30.859 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:30.868 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:30:30.882 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:30.879+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:30.983 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:30.979+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:31.007 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:31.007 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:31.007 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:31.007 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:31.007 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:31.017 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:30:31.085 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:31.081+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:31.186 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:31.183+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:31.195 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:31.195 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:31.195 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:31.195 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:31.196 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.196 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:30:31.196 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:31.196 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 189.37931785800001, 2023-12-06T18:30:31.196 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:30:31.196 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:31.196 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:30:31.196 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:31.197 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.197 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:31.197 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:31.197 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:31.197 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.197 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.197 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:31.198 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:31.198 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:31.198 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.198 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.198 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:31.198 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:30:31.198 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:30:31.199 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.199 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.199 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:31.199 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:30:31.199 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:30:31.199 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.199 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.200 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:31.200 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:30:31.200 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:30:31.200 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.200 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.200 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:31.200 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:30:31.200 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:30:31.201 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.201 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.201 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:31.201 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:30:31.201 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:30:31.201 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.201 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.201 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:31.201 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:30:31.201 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:30:31.201 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.201 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.201 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:31.201 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:30:31.202 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:30:31.202 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:31.202 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:31.202 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:31.202 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:31.202 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:31.202 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:31.205 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:30:31.291 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:31.287+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:31.349 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:31.349 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:31.349 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:31.349 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:31.358 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:30:31.402 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:31.398+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:31.489 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:31.485+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:31.495 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:31.495 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:31.495 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:31.495 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:31.495 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:31.505 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:30:31.590 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:31.586+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:31.687 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:31.687 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:31.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:31.687 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:31.687 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.687 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:31.687 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:31.688 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 70.587762983999994, 2023-12-06T18:30:31.688 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:30:31.688 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:31.688 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:31.688 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:31.688 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:31.688 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:31.688 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:30:31.688 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.688 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:31.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.688 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:31.688 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:31.688 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:31.688 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.689 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:31.689 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:31.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:31.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.689 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:31.689 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:30:31.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:30:31.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.689 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:31.689 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:30:31.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:30:31.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:31.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:30:31.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:30:31.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:31.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:30:31.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:30:31.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:31.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:30:31.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:30:31.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:31.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:30:31.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:30:31.691 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:31.691 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:31.691 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:31.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.691 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:30:31.691 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:31.691 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 10.569505681999999, 2023-12-06T18:30:31.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:30:31.691 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:31.691 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:31.691 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:31.691 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:31.692 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:31.692 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:30:31.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.692 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:31.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:31.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:31.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:31.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:31.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:31.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:31.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:31.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:30:31.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:30:31.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:31.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:30:31.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:30:31.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:31.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:30:31.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:30:31.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:31.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:30:31.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:30:31.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:31.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:30:31.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:30:31.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:31.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:31.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:31.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:30:31.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:30:31.694 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:31.694 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:31.694 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:31.695 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:31.695 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:31.695 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:31.696 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:30:31.697 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:31.692+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:31.797 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:31.793+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:31.834 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:31.834 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:31.834 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:31.834 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:31.844 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:30:31.898 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:31.895+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:31.972 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:31.972 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:31.972 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:31.972 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:31.973 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:31.981 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:30:32.000 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:31.996+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:32.101 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:32.097+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:32.154 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:32.154 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:32.155 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:32.155 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:32.155 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.155 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:32.155 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:32.155 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 130.66111301999999, 2023-12-06T18:30:32.155 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:30:32.155 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.155 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:32.155 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:32.155 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:32.155 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:32.155 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:30:32.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.155 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.156 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.156 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.156 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:32.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.156 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.156 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.156 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:32.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.156 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.156 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.156 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:30:32.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:30:32.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.157 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.157 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.157 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:30:32.157 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:30:32.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.157 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.157 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.157 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:30:32.157 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:30:32.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.157 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.157 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.157 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:30:32.157 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:30:32.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.158 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.158 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.158 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:30:32.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:30:32.158 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.158 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.158 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.158 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:30:32.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:30:32.158 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.158 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.159 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.159 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.159 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.159 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:32.163 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:30:32.202 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:32.198+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:32.292 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:32.292 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:32.292 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:32.292 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:32.301 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:30:32.303 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:32.299+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:32.404 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:32.401+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:32.430 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:32.430 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:32.430 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:32.430 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:32.430 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:32.439 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:30:32.505 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:32.501+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:32.606 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:32.602+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:32.654 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:32.654 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:32.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:32.655 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:32.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.655 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:30:32.655 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:32.655 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 216.039428816, 2023-12-06T18:30:32.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:30:32.655 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.655 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.655 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:32.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:32.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:30:32.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:30:32.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:30:32.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:30:32.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:30:32.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:30:32.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:30:32.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:30:32.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:30:32.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:30:32.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:30:32.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:30:32.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:30:32.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:30:32.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:30:32.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:30:32.658 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.658 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.659 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.659 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:30:32.659 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:32.659 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 215.016601807, 2023-12-06T18:30:32.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:30:32.659 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.659 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.659 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:32.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:32.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:30:32.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:30:32.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:30:32.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:30:32.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:30:32.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:30:32.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:30:32.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:30:32.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:30:32.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:30:32.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:30:32.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:30:32.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:30:32.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:30:32.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:30:32.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:30:32.662 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.662 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.663 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.663 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:30:32.663 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:32.663 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 215.00455516900001, 2023-12-06T18:30:32.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:30:32.663 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.663 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.663 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:32.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:32.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:30:32.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:30:32.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:30:32.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:30:32.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.665 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:30:32.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:30:32.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.665 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.665 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:30:32.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:30:32.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.665 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.665 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:30:32.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:30:32.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.665 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:30:32.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:30:32.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:30:32.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:30:32.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:30:32.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:30:32.666 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.666 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.666 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.667 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:30:32.667 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:32.667 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 214.98315456200001, 2023-12-06T18:30:32.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:30:32.667 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.667 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.667 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:32.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:32.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:30:32.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:30:32.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:30:32.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:30:32.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:30:32.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:30:32.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:30:32.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:30:32.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:30:32.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:30:32.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:30:32.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:30:32.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:30:32.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:30:32.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:30:32.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:30:32.670 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.670 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.670 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.671 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:30:32.671 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:32.671 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 214.98197523900001, 2023-12-06T18:30:32.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:30:32.671 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.671 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.671 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:32.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:32.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:30:32.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:30:32.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:30:32.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:30:32.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:30:32.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:30:32.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:30:32.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:30:32.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:30:32.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:30:32.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:30:32.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:30:32.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:30:32.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:30:32.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:30:32.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:30:32.674 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.674 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.674 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.674 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:30:32.675 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:32.675 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 214.981403258, 2023-12-06T18:30:32.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:30:32.675 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.675 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.675 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:32.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:32.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:30:32.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:30:32.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:30:32.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:30:32.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:30:32.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:30:32.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.677 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.677 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.677 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:30:32.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:30:32.677 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.677 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.677 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.677 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:30:32.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:30:32.677 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.677 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.677 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.677 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:30:32.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:30:32.677 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:30:32.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:30:32.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:30:32.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:30:32.678 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.678 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.678 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.679 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:30:32.679 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:32.679 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 214.53556778800001, 2023-12-06T18:30:32.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:30:32.679 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.679 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.679 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:32.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:32.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:30:32.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:30:32.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:30:32.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:30:32.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:30:32.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:30:32.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.681 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.681 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:30:32.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:30:32.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.681 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.681 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:30:32.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:30:32.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.681 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.681 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:30:32.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:30:32.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:30:32.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:30:32.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:30:32.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:30:32.682 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.682 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.682 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.683 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:30:32.683 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:32.683 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 214.533967932, 2023-12-06T18:30:32.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:30:32.683 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.683 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.683 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:32.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:32.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:30:32.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:30:32.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:30:32.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:30:32.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:30:32.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:30:32.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:30:32.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:30:32.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:30:32.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:30:32.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:30:32.686 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:30:32.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.686 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.686 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.686 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:30:32.686 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:30:32.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.686 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.686 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.686 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:30:32.686 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:30:32.686 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.686 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.686 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.687 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.687 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:30:32.687 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:32.687 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 214.15476231599999, 2023-12-06T18:30:32.687 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:30:32.687 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.687 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.687 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.687 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.687 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.687 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:32.687 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.687 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.687 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.687 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.687 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:32.688 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.688 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.688 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.688 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.688 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:30:32.688 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:30:32.688 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.688 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.688 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.688 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:30:32.688 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:30:32.688 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.688 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.688 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.688 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:30:32.688 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:30:32.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.689 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:30:32.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:30:32.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.689 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:30:32.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:30:32.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.689 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:30:32.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:30:32.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:30:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:30:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:30:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:30:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.691 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:30:32.691 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:32.691 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 213.917260134, 2023-12-06T18:30:32.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:30:32.691 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.691 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.691 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:32.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:32.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:30:32.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:30:32.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:30:32.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:30:32.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:30:32.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:30:32.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.693 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:30:32.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:30:32.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.693 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:30:32.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:30:32.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.693 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:30:32.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:30:32.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:30:32.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:30:32.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:30:32.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:30:32.694 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.694 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.694 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.694 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:30:32.694 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:32.695 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 213.91653213500001, 2023-12-06T18:30:32.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:30:32.695 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.695 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.695 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:32.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:32.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:30:32.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:30:32.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:30:32.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:30:32.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:30:32.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:30:32.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:30:32.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:30:32.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:30:32.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:30:32.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:30:32.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:30:32.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.698 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:30:32.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:30:32.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.698 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:30:32.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:30:32.698 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.698 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.698 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.698 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:30:32.698 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:32.699 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 213.91151972500001, 2023-12-06T18:30:32.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:30:32.699 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.699 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.699 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.699 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:32.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.699 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:32.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:30:32.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:30:32.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:30:32.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:30:32.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:30:32.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:30:32.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.701 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.701 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.701 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:30:32.701 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:30:32.701 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.701 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.701 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.701 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:30:32.701 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:30:32.701 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.701 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.701 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.701 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:30:32.701 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:30:32.701 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.701 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.702 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.702 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:30:32.702 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:30:32.702 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.702 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.702 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.702 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:30:32.702 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:30:32.702 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.702 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.702 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.702 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.702 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.702 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:30:32.703 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:32.703 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 213.90599264599999, 2023-12-06T18:30:32.703 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:30:32.703 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.703 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.703 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.703 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.703 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.703 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:32.703 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.703 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.703 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.703 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.703 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:32.703 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.703 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:30:32.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:30:32.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:30:32.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:30:32.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:30:32.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:30:32.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:30:32.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:30:32.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:30:32.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:30:32.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:30:32.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:30:32.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.706 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.706 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:30:32.706 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:30:32.706 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.706 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.706 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.706 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:30:32.706 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:30:32.706 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.706 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.706 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.706 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.706 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.706 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:30:32.706 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:32.707 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 213.90548180900001, 2023-12-06T18:30:32.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:30:32.707 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.707 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.707 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.707 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.707 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:32.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.707 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.707 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:32.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:30:32.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:30:32.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:30:32.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:30:32.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:30:32.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:30:32.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:30:32.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:30:32.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:30:32.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:30:32.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:30:32.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:30:32.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.710 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.710 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:30:32.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:30:32.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.710 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.710 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:30:32.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:30:32.710 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.710 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.710 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.710 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:30:32.710 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:32.711 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 213.90502254800001, 2023-12-06T18:30:32.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:30:32.711 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.711 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.711 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.711 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:32.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.711 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:32.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:30:32.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:30:32.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:30:32.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:30:32.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:30:32.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:30:32.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:30:32.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:30:32.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:30:32.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:30:32.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:30:32.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:30:32.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.714 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:30:32.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:30:32.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.714 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.714 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:30:32.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:30:32.714 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.714 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.714 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.714 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:30:32.714 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:32.714 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 207.32510674100001, 2023-12-06T18:30:32.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:30:32.715 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.715 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.715 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:32.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:32.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:30:32.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:30:32.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:30:32.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:30:32.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:30:32.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:30:32.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:30:32.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:30:32.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:30:32.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:30:32.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:30:32.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:30:32.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:30:32.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:30:32.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.718 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.718 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:30:32.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:30:32.718 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.718 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.718 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.718 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:30:32.718 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:32.718 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 207.318868936, 2023-12-06T18:30:32.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:30:32.719 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.719 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.719 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:32.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:32.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:30:32.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:30:32.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:30:32.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:30:32.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:30:32.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:30:32.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:30:32.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:30:32.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:30:32.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:30:32.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:30:32.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:30:32.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:30:32.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:30:32.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:30:32.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:30:32.722 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.722 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.722 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.722 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:30:32.722 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:32.722 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 207.312805857, 2023-12-06T18:30:32.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:30:32.723 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.723 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.723 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.723 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:32.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.723 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:32.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.723 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:30:32.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:30:32.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:30:32.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:30:32.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:30:32.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:30:32.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:30:32.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:30:32.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:30:32.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:30:32.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:30:32.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:30:32.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:30:32.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:30:32.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:30:32.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:30:32.726 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.726 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.726 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.726 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:30:32.726 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:32.726 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 207.175418772, 2023-12-06T18:30:32.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:30:32.726 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.727 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.727 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.727 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.727 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:32.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.727 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.727 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:32.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.727 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.727 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:30:32.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:30:32.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:30:32.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:30:32.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:30:32.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:30:32.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:30:32.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:30:32.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:30:32.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:30:32.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:30:32.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:30:32.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:30:32.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:30:32.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:30:32.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:30:32.730 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.730 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.730 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.730 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:30:32.730 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:32.730 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 207.168748045, 2023-12-06T18:30:32.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:30:32.731 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:32.731 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:32.731 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:32.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:32.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:32.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:32.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:32.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:32.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:32.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:30:32.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:30:32.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:32.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:30:32.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:30:32.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:32.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:30:32.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:30:32.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:32.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:30:32.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:30:32.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:32.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:30:32.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:30:32.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:32.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:30:32.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:30:32.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:32.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:30:32.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:30:32.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:32.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:32.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:32.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:30:32.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:30:32.734 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.734 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.734 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.734 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:32.734 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:32.734 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:32.736 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:30:32.736 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:32.732+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:32.837 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:32.833+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:32.869 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:32.869 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:32.869 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:32.869 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:32.877 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:30:32.938 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:32.934+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:33.005 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:33.005 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:33.005 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:33.005 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:33.005 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:33.015 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:30:33.038 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:33.035+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:33.140 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:33.136+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:33.183 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:33.183 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:33.183 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:33.184 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:30:33.184 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:33.192 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:30:33.276 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:33.273+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:33.341 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:33.338+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:33.363 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:33.363 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:33.363 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:33.363 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:33.373 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:30:33.457 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:33.454+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:33.541 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:33.541 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:33.541 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:33.541 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:33.542 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:33.543 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:33.539+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:33.551 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:30:33.644 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:33.641+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:33.719 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:33.719 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:30:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 12.057629467, 2023-12-06T18:30:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:30:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:30:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:30:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:30:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:33.724 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:33.729 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:30:33.745 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:33.742+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:33.846 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:33.843+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:33.868 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:33.868 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:33.868 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:33.868 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:33.876 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:30:33.949 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:33.945+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:34.026 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:30:34.026 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:30:34.026 INFO:tasks.thrashosds.thrasher:primary_affinity 2023-12-06T18:30:34.026 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 1.000000 2023-12-06T18:30:34.026 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2023-12-06T18:30:34.048 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:34.045+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:34.054 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:34.054 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:34.054 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:34.054 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:34.055 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:34.064 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:30:34.149 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:34.146+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:34.250 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:34.247+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:34.253 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:34.253 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:34.253 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:34.253 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:30:34.253 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:34.262 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:30:34.351 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:34.348+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:34.441 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:34.441 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:34.441 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:34.442 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:34.451 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:30:34.453 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:34.449+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:34.554 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:34.551+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:34.588 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:34.588 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:34.588 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:34.588 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:34.588 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:34.597 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:30:34.654 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:34.651+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:34.793 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:34.793 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:34.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:34.793 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:34.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.793 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:30:34.793 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:34.793 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 219.38029542800001, 2023-12-06T18:30:34.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:30:34.793 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.793 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.793 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.794 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.794 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.794 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:30:34.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.794 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:34.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:34.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:30:34.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:30:34.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:30:34.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:30:34.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:30:34.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:30:34.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:30:34.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:30:34.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:30:34.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:30:34.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:30:34.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:30:34.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:30:34.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:30:34.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:30:34.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:30:34.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:30:34.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:30:34.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:30:34.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:30:34.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:30:34.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:30:34.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:30:34.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:30:34.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:30:34.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:30:34.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:30:34.799 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.799 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.799 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.799 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:34.799 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:34.799 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 217.271722189, 2023-12-06T18:30:34.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:30:34.799 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.799 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.799 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.799 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.800 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.800 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:30:34.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.800 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:34.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:34.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:30:34.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:30:34.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:30:34.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:30:34.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:30:34.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:30:34.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.802 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:30:34.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:30:34.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.802 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:30:34.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:30:34.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.802 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:30:34.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:30:34.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:30:34.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:30:34.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:30:34.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:30:34.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:30:34.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:30:34.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:30:34.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:30:34.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:30:34.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:30:34.804 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.804 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.804 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.805 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:34.805 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:34.805 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 217.27097695699999, 2023-12-06T18:30:34.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:30:34.805 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.805 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.805 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.805 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.805 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.805 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:30:34.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.805 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:34.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:34.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:30:34.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:30:34.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:30:34.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:30:34.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:30:34.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:30:34.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:30:34.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:30:34.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:30:34.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:30:34.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.808 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.808 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:30:34.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:30:34.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.808 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.808 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:30:34.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:30:34.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.808 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.808 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:30:34.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:30:34.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:30:34.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:30:34.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:30:34.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:30:34.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:30:34.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:30:34.809 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.809 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.809 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.810 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:34.810 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:34.810 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 217.270293202, 2023-12-06T18:30:34.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:30:34.810 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.810 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.810 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.810 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.810 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.810 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:30:34.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.810 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:34.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:34.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:30:34.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:30:34.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:30:34.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:30:34.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:30:34.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:30:34.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:30:34.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:30:34.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:30:34.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:30:34.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:30:34.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:30:34.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:30:34.814 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:30:34.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.814 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.814 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:30:34.814 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:30:34.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.814 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.814 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:30:34.814 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:30:34.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.814 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.815 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:30:34.815 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:30:34.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.815 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.815 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.815 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:30:34.815 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:30:34.815 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.815 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.815 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.815 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.815 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:34.815 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:34.815 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 217.264719947, 2023-12-06T18:30:34.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:30:34.816 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.816 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.816 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.816 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.816 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.816 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:30:34.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.816 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:34.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:34.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:30:34.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:30:34.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:30:34.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:30:34.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:30:34.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:30:34.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:30:34.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:30:34.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:30:34.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:30:34.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.819 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.819 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.819 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:30:34.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:30:34.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.819 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.819 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.819 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:30:34.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:30:34.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.819 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.819 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.819 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:30:34.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:30:34.820 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.820 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.820 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.820 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:30:34.820 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:30:34.820 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.820 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.820 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.820 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:30:34.820 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:30:34.820 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.820 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.820 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.821 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:30:34.821 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:30:34.821 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.821 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.821 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.821 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.821 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.821 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:34.821 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:34.821 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 217.264046307, 2023-12-06T18:30:34.821 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:30:34.821 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.821 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.821 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.821 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.821 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.822 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:30:34.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.822 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.822 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.822 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.822 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:34.822 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.822 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.822 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.822 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:34.822 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.822 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.822 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:30:34.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:30:34.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:30:34.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:30:34.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:30:34.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:30:34.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:30:34.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:30:34.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.824 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:30:34.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:30:34.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.824 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:30:34.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:30:34.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.824 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:30:34.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:30:34.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.825 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.825 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.825 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:30:34.825 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:30:34.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.825 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.825 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.825 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:30:34.825 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:30:34.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.825 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.825 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.825 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:30:34.825 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:30:34.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.825 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.826 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.826 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:30:34.826 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:30:34.826 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.826 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.826 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.826 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.826 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.826 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:34.826 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:34.826 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 217.263384108, 2023-12-06T18:30:34.826 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:30:34.826 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.826 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.826 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.827 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.827 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.827 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:30:34.827 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.827 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.827 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.827 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.827 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:34.827 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.827 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.827 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.827 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.827 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:34.827 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.827 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.827 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.827 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.828 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:30:34.828 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:30:34.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.828 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.828 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:30:34.828 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:30:34.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.828 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.828 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:30:34.828 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:30:34.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.828 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:30:34.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:30:34.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.829 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.829 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:30:34.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:30:34.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.829 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.829 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:30:34.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:30:34.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.829 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.829 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.830 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:30:34.830 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:30:34.830 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.830 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.830 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.830 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:30:34.830 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:30:34.830 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.830 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.830 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.830 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:30:34.830 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:30:34.830 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.830 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.830 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.831 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:30:34.831 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:30:34.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.831 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.831 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.831 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:30:34.831 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:30:34.831 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.831 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.831 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.831 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.831 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:34.831 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:34.832 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 217.19067467299999, 2023-12-06T18:30:34.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:30:34.832 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.832 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.832 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.832 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.832 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.832 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:30:34.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.832 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.832 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:34.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.833 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.833 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.833 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:34.833 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.833 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.833 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.833 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:30:34.833 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:30:34.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.833 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.833 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.834 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:30:34.834 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:30:34.834 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.834 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.834 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.834 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:30:34.834 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:30:34.834 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.834 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.834 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.834 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:30:34.834 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:30:34.834 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.834 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.834 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.835 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:30:34.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:30:34.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.835 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.835 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.835 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:30:34.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:30:34.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.835 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.835 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.835 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:30:34.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:30:34.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.835 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.835 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.835 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:30:34.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:30:34.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.836 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.836 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.836 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:30:34.836 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:30:34.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.836 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.836 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.836 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:30:34.836 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:30:34.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.836 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.836 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.836 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:30:34.836 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:30:34.836 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.836 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.837 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.837 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:34.837 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:34.837 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 217.189992964, 2023-12-06T18:30:34.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:30:34.837 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.837 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.837 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.837 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.837 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.837 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:30:34.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.837 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:34.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:34.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:30:34.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:30:34.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.839 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:30:34.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:30:34.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.839 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:30:34.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:30:34.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.839 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:30:34.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:30:34.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:30:34.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:30:34.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.840 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:30:34.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:30:34.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.840 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:30:34.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:30:34.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.840 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:30:34.841 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:30:34.841 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.841 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.841 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.841 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:30:34.841 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:30:34.841 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.841 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.841 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.841 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:30:34.841 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:30:34.841 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.841 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.841 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.841 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:30:34.842 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:30:34.842 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.842 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.842 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.842 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.842 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:34.842 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:34.842 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 216.111188876, 2023-12-06T18:30:34.842 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:30:34.842 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.842 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.842 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.842 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.842 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.843 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:30:34.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.843 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:34.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:34.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:30:34.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:30:34.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:30:34.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:30:34.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:30:34.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:30:34.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:30:34.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:30:34.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.845 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:30:34.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:30:34.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.845 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:30:34.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:30:34.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.845 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:30:34.846 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:30:34.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.846 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.846 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.846 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:30:34.846 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:30:34.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.846 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.846 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.846 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:30:34.846 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:30:34.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.846 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.846 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.847 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:30:34.847 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:30:34.847 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.847 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.847 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.847 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:30:34.847 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:30:34.847 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.847 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.847 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.847 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.847 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.847 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:34.847 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:34.847 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 216.10977230200001, 2023-12-06T18:30:34.847 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:30:34.848 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.848 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.848 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.848 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.848 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.848 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:30:34.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.848 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:34.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:34.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:30:34.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:30:34.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:30:34.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:30:34.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:30:34.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:30:34.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:30:34.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:30:34.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:30:34.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:30:34.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:30:34.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:30:34.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.851 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:30:34.851 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:30:34.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.851 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:30:34.851 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:30:34.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.851 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:30:34.851 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:30:34.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.851 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:30:34.852 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:30:34.852 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.852 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.852 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.852 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:30:34.852 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:30:34.852 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.852 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.852 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.852 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.852 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.852 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:34.852 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:34.852 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 216.10868717700001, 2023-12-06T18:30:34.852 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:30:34.852 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.853 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.853 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.853 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.853 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.853 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:30:34.853 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.853 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.853 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.853 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.853 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:34.853 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.853 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.853 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.853 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.853 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:34.854 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.854 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.854 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.854 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.854 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:30:34.854 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:30:34.854 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.854 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.854 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.854 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:30:34.854 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:30:34.854 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.854 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.854 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.854 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:30:34.855 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:30:34.855 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.855 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.855 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.855 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:30:34.855 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:30:34.855 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.855 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.855 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.855 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:30:34.855 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:30:34.855 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.855 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.855 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.855 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:30:34.855 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:30:34.856 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.856 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.856 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.856 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:30:34.856 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:30:34.856 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.856 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.856 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.856 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:30:34.856 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:30:34.856 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.856 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.856 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.856 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:30:34.857 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:30:34.857 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.857 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.857 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.857 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:30:34.857 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:30:34.857 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.857 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.857 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.857 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:30:34.857 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:30:34.857 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.857 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.857 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.857 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.857 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.858 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:34.858 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:34.858 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 216.07353807499999, 2023-12-06T18:30:34.858 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:30:34.858 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.858 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.858 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.858 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.858 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.858 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:30:34.858 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.858 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.858 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.858 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:34.859 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.859 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:34.859 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.859 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:30:34.859 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:30:34.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.859 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:30:34.860 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:30:34.860 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.860 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.860 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.860 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:30:34.860 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:30:34.860 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.860 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.860 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.860 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:30:34.860 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:30:34.860 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.860 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.860 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.860 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:30:34.861 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:30:34.861 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.861 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.861 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.861 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:30:34.861 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:30:34.861 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.861 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.861 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.861 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:30:34.861 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:30:34.861 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.861 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.861 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.862 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:30:34.862 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:30:34.862 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.862 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.862 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.862 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:30:34.862 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:30:34.862 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.862 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.862 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.862 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:30:34.862 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:30:34.862 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.862 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.862 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.863 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:30:34.863 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:30:34.863 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.863 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.863 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.863 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.863 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.863 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:34.863 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:34.863 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 209.52380754999999, 2023-12-06T18:30:34.863 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:30:34.863 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.863 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.863 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.863 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.864 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.864 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:30:34.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.864 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.864 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.864 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.864 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:34.864 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.864 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.864 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.864 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:34.864 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.864 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.865 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.865 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:30:34.865 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:30:34.865 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.865 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.865 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.865 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:30:34.865 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:30:34.865 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.865 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.865 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.865 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:30:34.865 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:30:34.865 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.865 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.866 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.866 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:30:34.866 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:30:34.866 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.866 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.866 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.866 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:30:34.866 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:30:34.866 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.866 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.866 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.866 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:30:34.866 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:30:34.866 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.866 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.867 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.867 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:30:34.867 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:30:34.867 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.867 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.867 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.867 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:30:34.867 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:30:34.867 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.867 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.867 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.867 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:30:34.867 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:30:34.867 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.867 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.867 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.868 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:30:34.868 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:30:34.868 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.868 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.868 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.868 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:30:34.868 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:30:34.868 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.868 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.868 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.868 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.868 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.868 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:34.868 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:34.868 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 209.52300194200001, 2023-12-06T18:30:34.869 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:30:34.869 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.869 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.869 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.869 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.869 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.869 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:30:34.869 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.869 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.869 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.869 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.869 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:34.869 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.869 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.869 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.869 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.869 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:34.870 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.870 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.870 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.870 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.870 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:30:34.870 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:30:34.870 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.870 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.870 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.870 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:30:34.870 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:30:34.870 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.870 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.870 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.871 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:30:34.871 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:30:34.871 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.871 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.871 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.871 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:30:34.871 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:30:34.871 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.871 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.871 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.871 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:30:34.871 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:30:34.871 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.871 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.871 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.872 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:30:34.872 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:30:34.872 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.872 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.872 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.872 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:30:34.872 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:30:34.872 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.872 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.872 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.872 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:30:34.872 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:30:34.872 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.872 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.872 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.872 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:30:34.873 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:30:34.873 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.873 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.873 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.873 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:30:34.873 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:30:34.873 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.873 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.873 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.873 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:30:34.873 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:30:34.873 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.873 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.873 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.873 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.873 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.873 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:34.874 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:34.874 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 209.52231113299999, 2023-12-06T18:30:34.874 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:30:34.874 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.874 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.874 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.874 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.874 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.874 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:30:34.874 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.874 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.874 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.874 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.875 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:34.875 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.875 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.875 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.875 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.875 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:34.875 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.875 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.875 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.875 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.875 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:30:34.875 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:30:34.875 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.875 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.875 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.875 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:30:34.875 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:30:34.876 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.876 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.876 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.876 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:30:34.876 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:30:34.876 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.876 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.876 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.876 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:30:34.876 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:30:34.876 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.876 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.876 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.876 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:30:34.876 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:30:34.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.877 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.877 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.877 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:30:34.877 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:30:34.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.877 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.877 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.877 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:30:34.877 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:30:34.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.877 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.877 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.877 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:30:34.877 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:30:34.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.878 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.878 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.878 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:30:34.878 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:30:34.878 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.878 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.878 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.878 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:30:34.878 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:30:34.878 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.878 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.878 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.878 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:30:34.878 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:30:34.879 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.879 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.879 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.879 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.879 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.879 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:34.879 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:34.879 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 209.51801362500001, 2023-12-06T18:30:34.879 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:30:34.879 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.879 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.879 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.879 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.879 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.879 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:30:34.879 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.879 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.880 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.880 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.880 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:34.880 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.880 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.880 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.880 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.880 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:34.880 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.880 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.880 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.880 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.880 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:30:34.880 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:30:34.880 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.881 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.881 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.881 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:30:34.881 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:30:34.881 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.881 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.881 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.881 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:30:34.881 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:30:34.881 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.881 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.881 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.881 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:30:34.881 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:30:34.882 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.882 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.882 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.882 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:30:34.882 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:30:34.882 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.882 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.882 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.882 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:30:34.882 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:30:34.882 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.882 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.882 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.882 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:30:34.882 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:30:34.882 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.883 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.883 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.883 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:30:34.883 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:30:34.883 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.883 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.883 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.883 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:30:34.883 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:30:34.883 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.883 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.883 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.883 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:30:34.883 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:30:34.883 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.883 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.884 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.884 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:30:34.884 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:30:34.884 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.884 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.884 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.884 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.884 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.884 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:34.884 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:34.884 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 209.51734594999999, 2023-12-06T18:30:34.884 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:30:34.884 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.884 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.884 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.884 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.884 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.885 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:30:34.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.885 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.885 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.885 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.885 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:34.885 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.885 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.885 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.885 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:34.885 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.885 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.885 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.885 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:30:34.885 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:30:34.886 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.886 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.886 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.886 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:30:34.886 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:30:34.886 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.886 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.886 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.886 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:30:34.886 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:30:34.886 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.886 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.886 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.886 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:30:34.887 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:30:34.887 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.887 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.887 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.887 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:30:34.887 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:30:34.887 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.887 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.887 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.887 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:30:34.887 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:30:34.887 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.887 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.887 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.887 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:30:34.888 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:30:34.888 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.888 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.888 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.888 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:30:34.888 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:30:34.888 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.888 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.888 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.888 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:30:34.888 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:30:34.888 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.888 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.888 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.888 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:30:34.889 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:30:34.889 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.889 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.889 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.889 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:30:34.889 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:30:34.889 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.889 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.889 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.889 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.889 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.889 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:34.889 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:34.889 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 209.516690522, 2023-12-06T18:30:34.890 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:30:34.890 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.890 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.890 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.890 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.890 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.890 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:30:34.890 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.890 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.890 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.890 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.890 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:34.890 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.890 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.891 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.891 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:34.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.891 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.891 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:30:34.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:30:34.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.891 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.891 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:30:34.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:30:34.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.891 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.892 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:30:34.892 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:30:34.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.892 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.892 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:30:34.892 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:30:34.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.892 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.892 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:30:34.892 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:30:34.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.893 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.893 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.893 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:30:34.893 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:30:34.893 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.893 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.893 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.893 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:30:34.893 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:30:34.893 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.893 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.893 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.893 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:30:34.893 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:30:34.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.894 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:30:34.894 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:30:34.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.894 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:30:34.894 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:30:34.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.895 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:30:34.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:30:34.895 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.895 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.895 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.895 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.895 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.895 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:34.895 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:34.895 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 209.476035975, 2023-12-06T18:30:34.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:30:34.895 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:34.895 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:34.895 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:34.895 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:34.896 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:34.896 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:30:34.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.896 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:34.896 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.896 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:34.896 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:34.896 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.896 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.896 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:34.896 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:34.896 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:34.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.896 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.896 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:34.896 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:30:34.897 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:30:34.897 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.897 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.897 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:34.897 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:30:34.897 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:30:34.897 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.897 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.897 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:34.897 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:30:34.897 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:30:34.897 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.897 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.897 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:34.897 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:30:34.897 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:30:34.898 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.898 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.898 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:34.898 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:30:34.898 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:30:34.898 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.898 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.898 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:34.898 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:30:34.898 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:30:34.898 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.898 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.898 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:34.898 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:30:34.898 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:30:34.899 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.899 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.899 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:34.899 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:30:34.899 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:30:34.899 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.899 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.899 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:34.899 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:30:34.899 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:30:34.899 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.899 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.899 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:34.899 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:30:34.899 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:30:34.900 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:34.900 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:34.900 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:34.900 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:30:34.900 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:30:34.900 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.900 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.900 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.900 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:34.900 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:34.900 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:34.901 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:34.790+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:34.902 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:30:34.902 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:34.898+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:35.002 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:34.999+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:35.088 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:35.088 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:35.088 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:35.088 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:35.097 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:30:35.140 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:35.135+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:35.204 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:35.200+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:35.276 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:35.276 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:35.276 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:35.276 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:35.276 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:35.285 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:30:35.326 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:35.323+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:35.405 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:35.402+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:35.466 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:35.466 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:35.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:35.466 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:35.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.466 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:30:35.466 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:35.466 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 193.65006387700001, 2023-12-06T18:30:35.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:30:35.467 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:35.467 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:30:35.467 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:35.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:35.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:35.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:35.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:35.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:35.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:35.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:35.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:30:35.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:30:35.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:35.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:30:35.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:30:35.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:35.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:30:35.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:30:35.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:35.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:30:35.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:30:35.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:35.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:30:35.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:30:35.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:35.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:30:35.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:30:35.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:35.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:30:35.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:30:35.470 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:35.470 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:35.470 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:35.470 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:35.470 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:35.470 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:35.475 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:30:35.518 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:35.513+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:35.607 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:35.603+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:35.653 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:35.653 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:35.653 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:35.653 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:35.663 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:30:35.707 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:35.704+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:35.809 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:35.805+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:35.841 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:35.841 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:35.841 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:35.841 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:35.841 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:35.850 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:30:35.910 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:35.906+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:35.991 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 74.891605745000007, 2023-12-06T18:30:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:30:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:30:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:30:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:30:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:30:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:30:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:30:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:30:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:30:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:30:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:30:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:30:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:30:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:30:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:30:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 14.873348442999999, 2023-12-06T18:30:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:30:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:30:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:30:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:30:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:30:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:30:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:30:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:30:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:30:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:30:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:30:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:30:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:30:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:30:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:35.999 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:36.001 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:30:36.042 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:36.038+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:36.112 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:36.108+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:36.142 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:36.145 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:36.145 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:36.145 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:36.152 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:30:36.213 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:36.210+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:36.280 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:36.280 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:36.280 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:36.281 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:36.281 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:36.290 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:30:36.331 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:36.328+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:36.415 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:36.412+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:36.463 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:36.464 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:36.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:36.464 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:36.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.464 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:36.464 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:36.464 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 134.970429287, 2023-12-06T18:30:36.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:30:36.464 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:36.464 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:36.464 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:36.465 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:36.465 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:36.465 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:30:36.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.465 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:36.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:36.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:36.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:36.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:36.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:36.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:36.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:36.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:30:36.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:30:36.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:36.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:30:36.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:30:36.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:36.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:30:36.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:30:36.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:36.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:30:36.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:30:36.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:36.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:30:36.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:30:36.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:36.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:30:36.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:30:36.468 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:36.468 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:36.468 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:36.468 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:36.468 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:36.468 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:36.472 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:30:36.517 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:36.513+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:36.603 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:36.603 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:36.603 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:36.603 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:36.612 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:30:36.653 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:36.650+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:36.718 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:36.715+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:36.787 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:36.788 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:36.788 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:36.788 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:36.788 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:36.797 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:30:36.819 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:36.816+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:36.921 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:36.917+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:36.966 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:36.966 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:36.966 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:36.966 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:36.967 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.967 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:30:36.967 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:36.967 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 220.35045502299999, 2023-12-06T18:30:36.967 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:30:36.967 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:36.967 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:36.967 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:36.968 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.969 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:36.970 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:36.970 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:36.970 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.970 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.970 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:36.970 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:36.970 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:36.970 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.971 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:36.971 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:30:36.971 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:30:36.971 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.971 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.971 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:36.971 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:30:36.972 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:30:36.972 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.972 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.972 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:36.972 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:30:36.972 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:30:36.972 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.972 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.972 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:36.972 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:30:36.973 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:30:36.973 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.973 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.973 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:36.973 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:30:36.973 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:30:36.973 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.973 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.973 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:36.973 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:30:36.973 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:30:36.973 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.973 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.973 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:36.973 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:30:36.974 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:30:36.974 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.974 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.974 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:36.974 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:30:36.974 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:30:36.974 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:36.974 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:36.974 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:36.974 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.974 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.974 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:30:36.974 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:36.974 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 219.327628014, 2023-12-06T18:30:36.974 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:30:36.975 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:36.975 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:36.975 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:36.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.975 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:36.975 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:36.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:36.975 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.975 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:36.975 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:36.975 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:36.975 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.975 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.975 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:36.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:30:36.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:30:36.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.976 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:36.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:30:36.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:30:36.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.976 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:36.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:30:36.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:30:36.976 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.976 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.976 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:36.976 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:30:36.976 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:30:36.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.977 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:36.977 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:30:36.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:30:36.977 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.977 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.977 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:36.977 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:30:36.977 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:30:36.978 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.978 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:36.978 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:30:36.978 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:30:36.978 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.978 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.978 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:36.978 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:30:36.979 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:30:36.979 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:36.979 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:36.979 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:36.979 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.979 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.979 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:30:36.979 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:36.979 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 219.31558137600001, 2023-12-06T18:30:36.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:30:36.980 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:36.980 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:36.980 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:36.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.980 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:36.980 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:36.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:36.980 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:36.981 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:36.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:36.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:36.981 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:30:36.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:30:36.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:36.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:30:36.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:30:36.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:36.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:30:36.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:30:36.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:36.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:30:36.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:30:36.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.984 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.984 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:36.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:30:36.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:30:36.984 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:36.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:30:36.985 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:30:36.985 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.985 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.985 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:36.985 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:30:36.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:30:36.986 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.986 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.986 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:36.986 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:30:36.986 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:30:36.986 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:36.987 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:36.987 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:36.987 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.987 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.987 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:30:36.987 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:36.988 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 219.29418076900001, 2023-12-06T18:30:36.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:30:36.988 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:36.988 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:36.988 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:36.988 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.988 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:36.988 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:36.988 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:36.988 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:36.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:36.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:36.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:36.989 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:30:36.989 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:30:36.989 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.989 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.989 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:36.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:30:36.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:30:36.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:36.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:30:36.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:30:36.990 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.990 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.990 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:36.990 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:30:36.990 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:30:36.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:36.991 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:30:36.991 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:30:36.991 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.991 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.991 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:36.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:30:36.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:30:36.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:36.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:30:36.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:30:36.992 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.992 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.992 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:36.992 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:30:36.992 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:30:36.992 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:36.992 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:36.992 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:36.993 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.993 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:30:36.993 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:36.993 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 219.29300144600001, 2023-12-06T18:30:36.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:30:36.993 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:36.993 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:36.993 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:36.993 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.993 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:36.993 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:36.993 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:36.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:36.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:36.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:36.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:36.994 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:30:36.994 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:30:36.994 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.994 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.994 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:36.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:30:36.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:30:36.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:36.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:30:36.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:30:36.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:36.995 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:30:36.995 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:30:36.995 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.995 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.995 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:36.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:30:36.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:30:36.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:36.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:30:36.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:30:36.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:36.996 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:30:36.996 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:30:36.996 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.996 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.996 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:36.997 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:30:36.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:30:36.997 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:36.997 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:36.997 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:36.997 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.997 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:30:36.997 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:36.997 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 219.292429465, 2023-12-06T18:30:36.997 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:30:36.997 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:36.997 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:36.997 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:36.997 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.997 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:36.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:36.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:36.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:36.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:36.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:36.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:36.998 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:30:36.998 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:30:36.998 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.998 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.998 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:36.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:30:36.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:30:36.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:36.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:30:36.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:30:36.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:36.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:30:36.999 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:30:36.999 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:36.999 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:36.999 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:36.999 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:30:37.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:30:37.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:37.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:30:37.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:30:37.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:37.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:30:37.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:30:37.000 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.000 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.000 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:37.000 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:30:37.000 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:30:37.001 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.001 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:37.001 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.001 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.001 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:30:37.001 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:37.001 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 218.84659399500001, 2023-12-06T18:30:37.001 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:30:37.001 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:37.001 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:37.001 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:37.001 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:37.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:37.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:37.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:37.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.002 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.002 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:37.002 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:30:37.002 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:30:37.002 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:37.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:30:37.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:30:37.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.003 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:37.003 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:30:37.003 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:30:37.003 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.003 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:37.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:30:37.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:30:37.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:37.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:30:37.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:30:37.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.004 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:37.004 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:30:37.004 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:30:37.004 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.004 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:37.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:30:37.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:30:37.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.005 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:37.005 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:30:37.005 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:30:37.005 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.005 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:37.005 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.005 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.005 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.005 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:30:37.005 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:37.005 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 218.84499413899999, 2023-12-06T18:30:37.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:30:37.006 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:37.006 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:37.006 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:37.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:37.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:37.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.006 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:37.006 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:37.006 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.006 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.006 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:37.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:30:37.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:30:37.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:37.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:30:37.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:30:37.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:37.007 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:30:37.007 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:30:37.007 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.007 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.007 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:37.008 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:30:37.008 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:30:37.008 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.008 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.008 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:37.008 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:30:37.008 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:30:37.008 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.008 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.008 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:37.008 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:30:37.008 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:30:37.008 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.008 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.008 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:37.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:30:37.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:30:37.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.009 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:37.009 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:30:37.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:30:37.009 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.009 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:37.009 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.009 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.009 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.009 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:30:37.009 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:37.009 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 218.46578852299999, 2023-12-06T18:30:37.009 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:30:37.010 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:37.010 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:37.010 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:37.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:37.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:37.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:37.010 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:37.010 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.010 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.010 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.010 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:37.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:30:37.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:30:37.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:37.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:30:37.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:30:37.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:37.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:30:37.011 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:30:37.011 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.011 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.011 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:37.011 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:30:37.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:30:37.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:37.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:30:37.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:30:37.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:37.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:30:37.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:30:37.012 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.012 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.012 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:37.012 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:30:37.012 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:30:37.013 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.013 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.013 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:37.013 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:30:37.013 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:30:37.013 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.013 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:37.013 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.014 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.014 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:30:37.014 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:37.014 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 218.228286341, 2023-12-06T18:30:37.014 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:30:37.014 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:37.014 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:37.014 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:37.014 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.014 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:37.014 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:37.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:37.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:37.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:37.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:30:37.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:30:37.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.015 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.015 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:37.015 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:30:37.015 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:30:37.015 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:37.016 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:30:37.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:30:37.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:37.016 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:30:37.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:30:37.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.016 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.016 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:37.016 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:30:37.016 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:30:37.016 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:37.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:30:37.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:30:37.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:37.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:30:37.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:30:37.017 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.017 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.017 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:37.017 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:30:37.017 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:30:37.017 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.017 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:37.018 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.018 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.018 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:30:37.018 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:37.018 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 218.22755834200001, 2023-12-06T18:30:37.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:30:37.018 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:37.018 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:37.018 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:37.018 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.018 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:37.018 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:37.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:37.019 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:37.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:37.019 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:30:37.019 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:30:37.019 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.019 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.019 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:37.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:30:37.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:30:37.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:37.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:30:37.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:30:37.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.020 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.020 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:37.020 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:30:37.020 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:30:37.020 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.021 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:37.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:30:37.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:30:37.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.021 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:37.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:30:37.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:30:37.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.021 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:37.021 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:30:37.021 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:30:37.021 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.021 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.022 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:37.022 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:30:37.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:30:37.022 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.022 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:37.022 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.022 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.022 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.022 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:30:37.022 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:37.022 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 218.222545932, 2023-12-06T18:30:37.022 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:30:37.022 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:37.022 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:37.023 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:37.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:37.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:37.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:37.023 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:37.023 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.023 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.023 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.023 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:37.024 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:30:37.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:30:37.024 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.024 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:37.024 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:30:37.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:30:37.024 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.024 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:37.024 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:30:37.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:30:37.024 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.024 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.024 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:37.024 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:30:37.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:30:37.025 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.025 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:37.025 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:30:37.025 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:30:37.025 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.025 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.025 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:37.025 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:30:37.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:30:37.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.026 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.026 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:37.026 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:30:37.026 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:30:37.026 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.027 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:37.027 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:30:37.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:30:37.027 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.027 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:37.027 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.027 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.027 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.027 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:30:37.027 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:37.027 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 218.21701885300001, 2023-12-06T18:30:37.027 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:30:37.028 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:37.028 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:37.028 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:37.028 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.028 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:37.028 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:37.028 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.029 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:37.029 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:37.029 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.029 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.029 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.030 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:37.030 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:30:37.030 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:30:37.030 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.030 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:37.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:30:37.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:30:37.031 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.031 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.031 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:37.031 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:30:37.031 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:30:37.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.032 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:37.032 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:30:37.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:30:37.032 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.032 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:37.033 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:30:37.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:30:37.033 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.033 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:37.034 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:30:37.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:30:37.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.034 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:37.034 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:30:37.034 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:30:37.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:37.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:30:37.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:30:37.035 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.035 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:37.036 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.036 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:30:37.036 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:37.036 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 218.21650801600001, 2023-12-06T18:30:37.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:30:37.036 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:37.037 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:37.037 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:37.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:37.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:37.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:37.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:37.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:37.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:30:37.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:30:37.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:37.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:30:37.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:30:37.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.038 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.038 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:37.038 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:30:37.038 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:30:37.038 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:37.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:30:37.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:30:37.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:37.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:30:37.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:30:37.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.039 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:37.039 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:30:37.039 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:30:37.039 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.039 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:37.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:30:37.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:30:37.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.040 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:37.040 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:30:37.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:30:37.040 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.040 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:37.040 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.040 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.040 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:30:37.040 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:37.040 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 218.216048755, 2023-12-06T18:30:37.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:30:37.041 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:37.041 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:37.041 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:37.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:37.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:37.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:37.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:37.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:37.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:30:37.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:30:37.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:37.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:30:37.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:30:37.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:37.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:30:37.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:30:37.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:37.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:30:37.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:30:37.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:37.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:30:37.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:30:37.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:37.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:30:37.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:30:37.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:37.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:30:37.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:30:37.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:37.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:30:37.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:30:37.044 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.044 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:37.044 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.045 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:30:37.045 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:37.045 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 211.63613294800001, 2023-12-06T18:30:37.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:30:37.045 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:37.045 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:37.045 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:37.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:37.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:37.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:37.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:37.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:37.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:30:37.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:30:37.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:37.046 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:30:37.046 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:30:37.046 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.046 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.046 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:37.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:30:37.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:30:37.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:37.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:30:37.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:30:37.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:37.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:30:37.047 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:30:37.047 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.047 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.047 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:37.047 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:30:37.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:30:37.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:37.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:30:37.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:30:37.048 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.048 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.048 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:37.048 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:30:37.048 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:30:37.048 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.049 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:37.049 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.049 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.049 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:30:37.049 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:37.049 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 211.629895143, 2023-12-06T18:30:37.049 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:30:37.049 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:37.049 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:37.049 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:37.049 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.049 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:37.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:37.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:37.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:37.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:37.050 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:30:37.050 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:30:37.050 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.050 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.050 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:37.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:30:37.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:30:37.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:37.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:30:37.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:30:37.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.051 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.051 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:37.051 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:30:37.051 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:30:37.051 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:37.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:30:37.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:30:37.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:37.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:30:37.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:30:37.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.052 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.052 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:37.052 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:30:37.052 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:30:37.052 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.053 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:37.053 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:30:37.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:30:37.053 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.053 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:37.053 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.053 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.053 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:30:37.053 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:37.053 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 211.623832064, 2023-12-06T18:30:37.053 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:30:37.053 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:37.053 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:37.053 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:37.053 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:37.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:37.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.054 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:37.054 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:37.054 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.054 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.054 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:37.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:30:37.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:30:37.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:37.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:30:37.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:30:37.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.055 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:37.055 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:30:37.055 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:30:37.055 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.055 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:37.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:30:37.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:30:37.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:37.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:30:37.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:30:37.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.056 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:37.056 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:30:37.056 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:30:37.056 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.056 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:37.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:30:37.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:30:37.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.057 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:37.057 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:30:37.057 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:30:37.057 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.057 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:37.057 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.057 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.057 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.057 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:30:37.058 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:37.058 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 211.486444979, 2023-12-06T18:30:37.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:30:37.058 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:37.058 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:37.058 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:37.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:37.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:37.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.058 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.058 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.058 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:37.058 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:37.058 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:37.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:30:37.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:30:37.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:37.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:30:37.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:30:37.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.059 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.059 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:37.059 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:30:37.059 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:30:37.059 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:37.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:30:37.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:30:37.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:37.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:30:37.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:30:37.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.060 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:37.060 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:30:37.060 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:30:37.060 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.060 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:37.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:30:37.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:30:37.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.061 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:37.061 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:30:37.061 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:30:37.061 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.061 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:37.061 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.061 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.061 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.061 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:30:37.061 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:37.061 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 211.479774252, 2023-12-06T18:30:37.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:30:37.062 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:37.062 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:37.062 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:37.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:37.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:37.062 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.062 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.062 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.062 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:37.062 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:37.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:37.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:37.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:30:37.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:30:37.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:37.063 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:30:37.063 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:30:37.063 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.063 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.063 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:37.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:30:37.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:30:37.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:37.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:30:37.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:30:37.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:37.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:30:37.064 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:30:37.064 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.064 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.064 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:37.064 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:30:37.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:30:37.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:37.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:30:37.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:30:37.065 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:37.065 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:37.065 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:37.065 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:30:37.065 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:30:37.065 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.065 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:37.065 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.065 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:37.065 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:37.066 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:37.067 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:30:37.068 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:37.064+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:37.168 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:37.165+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:37.196 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:37.197 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:37.197 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:37.197 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:37.206 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:30:37.269 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:37.265+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:37.370 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:37.367+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:37.373 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:37.373 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:37.373 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:37.373 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:37.373 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:37.382 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:30:37.470 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:37.467+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:37.558 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:37.558 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:37.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:37.558 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:30:37.559 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:37.568 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:30:37.571 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:37.567+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:37.672 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:37.668+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:37.699 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:37.700 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:37.700 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:37.700 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:37.710 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:30:37.774 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:37.770+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:37.874 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:37.870+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:37.879 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:37.879 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:37.879 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:37.879 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:37.880 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:37.889 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:30:37.975 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:37.972+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:38.032 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:38.032 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:38.032 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:38.033 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:38.033 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:38.033 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:30:38.033 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:38.033 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 16.370332682000001, 2023-12-06T18:30:38.033 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:30:38.033 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:38.034 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:38.034 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:38.034 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:38.034 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:38.034 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:30:38.034 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:38.034 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:38.034 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:38.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:38.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:38.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:38.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:38.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:38.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:38.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:38.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:38.035 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:38.035 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:38.035 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:38.035 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:30:38.035 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:30:38.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:38.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:38.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:38.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:30:38.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:30:38.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:38.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:38.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:38.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:30:38.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:30:38.036 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:38.036 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:38.036 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:38.036 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:30:38.036 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:30:38.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:38.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:38.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:38.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:30:38.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:30:38.037 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:38.037 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:38.037 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:38.037 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:30:38.037 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:30:38.037 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:38.037 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:38.037 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:38.037 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:38.037 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:38.037 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:38.041 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:30:38.076 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:38.073+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:38.177 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:38.173+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:38.177 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:38.177 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:38.177 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:38.177 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:38.186 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:30:38.278 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:38.274+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:38.339 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:38.340 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:38.340 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:38.340 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:38.340 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:38.348 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:30:38.378 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:38.375+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:38.479 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:38.475+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:38.541 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:38.541 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:38.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:38.541 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:30:38.541 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:38.549 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:30:38.580 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:38.576+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:38.681 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:38.677+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:38.688 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:38.688 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:38.688 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:38.688 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:38.697 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:30:38.781 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:38.778+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:38.879 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:38.880 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:38.880 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:38.880 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:38.880 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:38.882 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:38.879+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:38.889 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:30:38.983 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:38.980+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:39.085 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:39.081+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:39.125 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:39.125 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:39.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:39.125 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:39.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.125 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:30:39.126 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:39.126 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 223.714328596, 2023-12-06T18:30:39.126 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:30:39.126 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.126 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.126 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.126 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.126 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.126 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:30:39.126 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.126 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.126 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.126 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.127 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:39.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.127 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.127 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:39.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.127 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.127 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:30:39.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:30:39.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:30:39.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:30:39.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.128 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:30:39.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:30:39.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.128 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:30:39.129 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:30:39.129 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.129 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.129 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.129 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:30:39.129 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:30:39.129 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.129 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.129 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:30:39.129 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:30:39.129 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:30:39.129 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.129 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.130 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.130 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:30:39.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:30:39.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.130 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.130 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.130 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:30:39.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:30:39.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.130 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.130 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.130 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:30:39.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:30:39.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:30:39.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:30:39.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:30:39.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:30:39.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:30:39.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:30:39.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:30:39.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:30:39.132 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.132 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.132 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.132 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:39.132 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:39.132 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 221.60575535699999, 2023-12-06T18:30:39.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:30:39.132 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.132 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.133 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.133 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.133 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.133 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:30:39.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.133 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:39.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:39.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:30:39.134 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:30:39.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:30:39.134 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:30:39.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:30:39.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:30:39.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.135 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.135 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:30:39.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:30:39.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.135 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.135 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:30:39.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:30:39.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.135 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.135 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:30:39.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:30:39.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:30:39.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:30:39.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:30:39.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:30:39.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:30:39.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:30:39.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.137 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:30:39.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:30:39.137 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.137 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:30:39.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:30:39.137 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.137 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.137 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.137 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.137 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:39.138 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:39.138 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 221.60501012500001, 2023-12-06T18:30:39.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:30:39.138 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.138 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.138 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.138 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.138 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.138 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:30:39.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.138 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:39.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:39.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:30:39.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:30:39.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:30:39.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:30:39.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:30:39.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:30:39.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:30:39.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:30:39.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:30:39.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:30:39.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:30:39.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:30:39.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:30:39.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:30:39.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:30:39.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:30:39.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:30:39.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:30:39.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:30:39.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:30:39.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.143 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:30:39.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:30:39.143 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.143 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.143 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.143 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:39.143 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:39.143 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 221.60432637, 2023-12-06T18:30:39.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:30:39.143 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.143 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.144 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.144 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.144 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.144 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:30:39.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.144 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:39.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:39.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:30:39.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:30:39.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:30:39.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:30:39.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:30:39.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:30:39.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:30:39.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:30:39.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:30:39.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:30:39.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:30:39.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:30:39.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:30:39.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:30:39.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:30:39.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:30:39.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:30:39.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:30:39.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:30:39.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:30:39.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:30:39.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:30:39.148 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.148 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.148 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.149 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:39.149 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:39.149 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 221.59875311499999, 2023-12-06T18:30:39.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:30:39.149 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.149 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.149 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.149 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.149 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.149 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:30:39.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.149 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:39.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:39.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:30:39.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:30:39.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:30:39.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:30:39.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.151 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:30:39.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:30:39.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.151 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:30:39.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:30:39.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.151 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:30:39.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:30:39.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:30:39.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:30:39.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:30:39.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:30:39.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:30:39.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:30:39.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.153 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.153 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:30:39.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:30:39.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.153 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.153 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:30:39.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:30:39.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.153 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:30:39.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:30:39.154 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.154 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.154 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.154 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:39.154 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:39.154 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 221.59807947499999, 2023-12-06T18:30:39.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:30:39.154 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.154 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.154 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.155 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.155 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.155 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:30:39.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.155 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:39.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:39.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:30:39.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:30:39.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:30:39.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:30:39.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:30:39.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:30:39.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:30:39.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:30:39.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:30:39.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:30:39.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:30:39.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:30:39.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.158 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:30:39.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:30:39.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.158 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:30:39.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:30:39.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.158 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:30:39.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:30:39.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.159 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.159 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:30:39.159 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:30:39.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.159 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.159 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:30:39.159 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:30:39.159 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.159 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.159 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.159 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:39.159 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:39.160 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 221.59741727599999, 2023-12-06T18:30:39.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:30:39.160 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.160 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.160 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.160 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.160 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.160 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:30:39.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.160 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:39.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:39.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:30:39.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:30:39.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:30:39.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:30:39.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.162 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.162 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:30:39.162 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:30:39.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.162 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.163 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:30:39.163 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:30:39.163 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.163 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.163 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.163 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:30:39.163 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:30:39.163 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.163 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.163 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.163 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:30:39.163 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:30:39.163 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.163 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.163 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.164 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:30:39.164 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:30:39.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.164 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.164 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.164 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:30:39.164 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:30:39.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.164 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.164 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.164 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:30:39.164 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:30:39.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:30:39.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:30:39.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:30:39.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:30:39.165 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.165 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.165 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.165 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:39.166 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:39.166 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 221.52470784100001, 2023-12-06T18:30:39.166 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:30:39.166 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.166 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.166 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.166 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.166 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.166 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:30:39.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.166 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.166 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.166 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.166 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:39.167 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.167 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.167 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.167 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.167 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:39.167 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.167 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.167 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.167 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.167 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:30:39.167 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:30:39.167 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.167 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.167 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.167 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:30:39.167 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:30:39.168 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.168 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.168 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.168 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:30:39.168 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:30:39.168 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.168 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.168 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.168 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:30:39.168 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:30:39.168 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.168 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.168 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.168 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:30:39.169 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:30:39.169 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.169 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.169 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.169 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:30:39.169 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:30:39.169 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.169 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.169 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.169 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:30:39.169 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:30:39.169 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.169 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.169 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.169 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:30:39.170 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:30:39.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.170 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.170 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.170 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:30:39.170 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:30:39.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.170 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.170 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.170 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:30:39.170 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:30:39.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.170 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.170 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.171 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:30:39.171 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:30:39.171 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.171 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.171 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.171 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.171 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.171 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:39.171 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:39.171 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 221.52402613199999, 2023-12-06T18:30:39.171 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:30:39.171 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.171 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.171 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.171 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.172 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.172 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:30:39.172 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.172 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.172 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.172 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.172 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:39.172 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.172 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.172 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.172 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.172 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:39.172 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.172 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.173 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.173 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.173 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:30:39.173 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:30:39.173 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.173 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.173 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.173 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:30:39.173 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:30:39.173 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.173 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.173 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.173 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:30:39.173 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:30:39.174 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.174 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.174 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.174 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:30:39.174 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:30:39.174 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.174 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.174 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.174 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:30:39.174 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:30:39.174 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.175 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.175 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.175 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:30:39.175 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:30:39.175 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.175 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.175 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.175 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:30:39.175 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:30:39.175 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.175 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.175 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.175 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:30:39.175 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:30:39.175 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.175 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.176 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.176 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:30:39.176 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:30:39.176 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.176 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.176 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.176 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:30:39.176 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:30:39.176 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.176 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.176 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.176 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:30:39.176 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:30:39.176 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.176 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.176 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.176 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.177 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.177 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:39.177 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:39.177 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 220.44522204399999, 2023-12-06T18:30:39.177 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:30:39.177 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.177 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.177 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.177 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.177 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.177 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:30:39.177 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.177 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.177 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.177 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.178 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:39.178 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.178 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.178 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.178 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.178 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:39.178 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.178 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.178 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.178 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.178 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:30:39.178 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:30:39.178 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.178 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.179 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.179 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:30:39.179 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:30:39.179 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.179 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.179 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.179 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:30:39.179 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:30:39.179 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.179 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.179 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.179 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:30:39.179 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:30:39.179 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.179 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.179 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.180 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:30:39.180 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:30:39.180 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.180 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.180 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.180 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:30:39.180 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:30:39.180 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.180 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.180 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.180 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:30:39.180 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:30:39.180 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.180 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.180 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.181 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:30:39.181 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:30:39.181 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.181 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.181 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.181 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:30:39.181 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:30:39.181 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.181 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.181 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.181 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:30:39.182 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:30:39.182 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.182 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.182 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.182 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:30:39.182 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:30:39.182 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.182 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.182 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.182 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.182 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.182 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:39.182 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:39.182 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 220.44380547, 2023-12-06T18:30:39.183 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:30:39.183 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.183 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.183 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.183 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.183 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.183 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:30:39.183 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.183 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.183 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.183 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.183 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:39.183 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.183 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.183 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.183 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.184 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:39.184 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.184 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.184 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.184 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.184 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:30:39.184 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:30:39.184 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.184 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.184 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.184 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:30:39.184 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:30:39.184 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.184 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.184 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.185 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:30:39.185 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:30:39.185 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.185 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.185 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.185 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:30:39.185 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:30:39.185 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.185 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.185 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.185 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:30:39.185 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:30:39.185 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.185 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.186 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.186 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:30:39.186 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:30:39.186 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.186 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.186 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.186 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:30:39.186 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:30:39.186 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.186 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.186 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.186 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:30:39.186 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:30:39.186 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.187 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.187 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.187 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:30:39.187 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:30:39.187 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.187 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.187 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.187 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:30:39.187 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:30:39.187 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.187 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.187 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.187 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:30:39.187 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:30:39.188 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.188 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.188 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.188 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.188 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.188 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:39.188 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:39.188 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 220.442720345, 2023-12-06T18:30:39.188 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:30:39.188 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.188 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.188 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.188 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.188 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.189 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:30:39.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.189 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.189 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.189 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.189 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:39.189 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.189 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.189 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.189 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:39.189 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.189 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.189 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.190 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:30:39.190 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:30:39.190 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.190 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.190 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.190 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:30:39.190 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:30:39.190 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.190 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.190 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.190 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:30:39.190 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:30:39.190 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.190 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.190 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.191 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:30:39.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:30:39.191 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.191 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.191 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.191 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:30:39.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:30:39.191 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.191 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.191 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.191 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:30:39.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:30:39.191 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.191 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.191 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.191 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:30:39.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:30:39.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.192 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.192 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.192 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:30:39.192 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:30:39.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.192 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.192 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.192 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:30:39.192 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:30:39.193 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.193 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.193 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.193 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:30:39.193 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:30:39.193 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.193 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.193 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.193 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:30:39.193 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:30:39.194 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.194 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.194 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.194 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.194 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.194 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:39.194 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:39.194 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 220.40757124300001, 2023-12-06T18:30:39.195 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:30:39.195 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.195 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.195 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.195 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.195 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.195 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:30:39.195 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.195 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.196 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.196 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.196 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:39.196 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.196 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.196 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.197 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.197 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:39.197 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.197 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.197 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.197 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.197 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:30:39.198 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:30:39.198 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.198 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.198 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.198 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:30:39.198 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:30:39.198 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.199 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.199 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.199 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:30:39.199 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:30:39.199 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.199 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.199 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.199 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:30:39.200 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:30:39.200 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.200 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.200 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.200 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:30:39.200 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:30:39.200 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.200 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.200 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.201 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:30:39.201 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:30:39.201 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.201 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.201 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.201 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:30:39.202 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:30:39.202 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.202 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.202 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.202 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:30:39.202 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:30:39.202 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.203 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.203 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.203 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:30:39.203 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:30:39.203 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.203 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.203 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.203 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:30:39.203 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:30:39.204 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.204 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.204 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.204 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:30:39.204 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:30:39.204 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.204 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.204 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.204 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.204 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.204 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:39.204 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:39.204 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 213.85784071800001, 2023-12-06T18:30:39.204 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:30:39.205 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.205 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.205 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.205 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.205 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.205 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:30:39.205 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.205 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.205 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.205 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.205 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:39.205 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.205 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.205 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.205 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.206 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:39.206 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.206 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.206 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.206 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.206 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:30:39.206 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:30:39.206 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.206 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.206 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.206 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:30:39.206 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:30:39.206 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.206 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.207 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.207 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:30:39.207 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:30:39.207 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.207 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.207 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.207 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:30:39.207 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:30:39.207 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.207 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.207 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.207 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:30:39.207 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:30:39.207 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.207 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.208 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.208 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:30:39.208 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:30:39.208 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.208 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.208 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.208 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:30:39.208 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:30:39.208 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.208 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.208 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.208 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:30:39.208 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:30:39.208 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.209 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.209 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.209 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:30:39.209 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:30:39.209 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.209 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.209 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.209 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:30:39.209 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:30:39.209 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.209 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.209 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.210 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:30:39.210 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:30:39.210 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.210 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.210 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.210 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.210 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:39.210 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:39.210 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 213.85703511, 2023-12-06T18:30:39.210 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:30:39.210 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.210 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.211 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.211 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.211 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.211 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:30:39.211 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.211 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.211 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.211 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.211 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:39.211 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.211 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.211 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.211 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.211 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:39.211 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.212 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.212 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.212 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.212 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:30:39.212 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:30:39.212 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.212 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.212 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.212 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:30:39.212 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:30:39.212 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.212 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.212 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.212 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:30:39.212 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:30:39.212 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.213 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.213 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.213 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:30:39.213 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:30:39.213 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.213 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.213 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.213 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:30:39.213 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:30:39.213 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.213 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.213 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.213 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:30:39.213 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:30:39.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.214 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.214 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.214 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:30:39.214 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:30:39.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.214 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.214 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.214 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:30:39.214 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:30:39.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.214 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.214 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.214 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:30:39.215 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:30:39.215 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.215 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.215 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.215 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:30:39.215 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:30:39.215 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.215 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.215 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.215 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:30:39.215 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:30:39.215 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.215 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.215 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.215 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.215 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.216 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:39.216 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:39.216 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 213.85634430100001, 2023-12-06T18:30:39.216 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:30:39.216 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.216 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.216 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.216 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.216 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.216 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:30:39.216 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.216 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.217 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.217 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.217 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:39.217 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.217 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.217 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.217 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.217 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:39.217 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.217 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.217 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.217 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.217 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:30:39.217 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:30:39.217 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.218 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.218 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.218 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:30:39.218 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:30:39.218 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.218 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.218 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.218 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:30:39.218 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:30:39.218 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.218 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.219 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.219 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:30:39.219 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:30:39.219 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.219 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.219 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.219 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:30:39.219 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:30:39.219 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.219 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.219 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.219 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:30:39.219 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:30:39.219 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.219 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.219 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.220 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:30:39.220 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:30:39.220 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.220 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.220 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.220 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:30:39.220 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:30:39.220 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.220 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.220 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.220 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:30:39.220 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:30:39.220 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.220 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.220 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.221 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:30:39.221 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:30:39.221 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.221 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.221 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.221 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:30:39.221 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:30:39.221 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.221 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.221 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.221 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.221 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.221 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:39.221 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:39.222 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 213.852046793, 2023-12-06T18:30:39.222 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:30:39.222 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.222 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.222 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.222 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.222 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.222 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:30:39.222 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.222 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.222 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.222 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.222 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:39.222 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.223 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.223 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.223 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.223 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:39.223 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.223 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.223 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.223 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.223 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:30:39.223 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:30:39.223 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.223 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.223 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.223 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:30:39.223 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:30:39.223 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.224 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.224 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.224 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:30:39.224 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:30:39.224 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.224 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.224 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.224 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:30:39.224 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:30:39.224 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.224 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.224 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.224 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:30:39.224 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:30:39.224 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.224 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.224 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.225 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:30:39.225 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:30:39.225 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.225 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.225 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.225 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:30:39.225 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:30:39.225 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.225 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.225 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.225 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:30:39.225 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:30:39.225 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.225 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.225 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.225 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:30:39.225 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:30:39.226 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.226 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.226 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.226 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:30:39.226 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:30:39.226 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.226 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.226 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.226 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:30:39.226 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:30:39.226 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.226 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.226 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.226 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.226 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.227 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:39.227 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:39.227 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 213.85137911800001, 2023-12-06T18:30:39.227 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:30:39.227 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.227 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.227 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.227 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.227 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.227 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:30:39.227 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.227 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.228 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.228 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.228 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:39.228 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.228 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.228 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.228 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.228 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:39.229 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.229 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.229 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.229 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:30:39.229 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:30:39.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.229 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.229 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.229 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:30:39.229 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:30:39.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.229 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.229 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.229 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:30:39.229 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:30:39.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.230 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.230 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.230 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:30:39.230 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:30:39.230 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.230 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.230 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.230 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:30:39.230 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:30:39.230 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.230 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.230 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.230 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:30:39.230 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:30:39.230 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.231 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.231 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.231 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:30:39.231 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:30:39.231 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.231 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.231 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.231 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:30:39.231 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:30:39.231 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.231 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.231 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.231 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:30:39.231 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:30:39.231 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.232 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.232 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.232 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:30:39.232 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:30:39.232 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.232 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.232 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.232 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:30:39.232 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:30:39.232 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.232 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.232 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.232 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.232 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.233 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:39.233 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:39.233 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 213.85072369, 2023-12-06T18:30:39.233 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:30:39.233 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.233 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.233 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.233 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.233 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.233 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:30:39.233 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.233 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.234 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.234 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.234 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:39.234 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.234 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.234 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.234 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.234 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:39.234 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.234 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.234 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.234 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.234 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:30:39.234 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:30:39.235 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.235 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.235 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.235 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:30:39.235 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:30:39.235 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.235 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.235 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.235 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:30:39.235 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:30:39.235 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.235 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.235 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.235 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:30:39.235 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:30:39.235 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.235 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.236 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.236 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:30:39.236 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:30:39.236 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.236 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.236 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.236 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:30:39.236 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:30:39.236 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.236 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.236 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.236 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:30:39.236 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:30:39.236 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.237 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.237 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.237 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:30:39.237 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:30:39.237 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.237 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.237 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.237 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:30:39.237 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:30:39.237 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.237 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.237 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.237 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:30:39.237 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:30:39.238 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.238 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.238 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.238 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:30:39.238 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:30:39.238 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.238 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.238 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.238 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.238 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.238 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:39.238 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:39.238 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 213.81006914299999, 2023-12-06T18:30:39.238 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:30:39.239 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.239 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:39.239 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:39.239 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:39.239 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:39.239 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:30:39.239 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.239 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.239 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.239 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.239 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:39.239 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.239 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.239 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.239 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.239 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:39.239 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.240 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.240 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.240 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.240 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:30:39.240 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:30:39.240 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.240 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.240 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.240 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:30:39.240 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:30:39.240 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.240 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.240 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.240 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:30:39.240 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:30:39.240 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.241 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.241 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.241 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:30:39.241 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:30:39.241 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.241 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.241 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.241 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:30:39.241 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:30:39.241 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.241 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.241 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.241 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:30:39.241 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:30:39.241 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.242 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.242 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:39.242 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:30:39.242 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:30:39.242 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.242 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.242 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:39.242 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:30:39.242 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:30:39.242 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.242 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.242 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:39.242 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:30:39.242 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:30:39.242 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.243 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.243 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:39.243 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:30:39.243 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:30:39.243 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.243 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.243 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.243 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:30:39.243 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:30:39.243 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.243 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.243 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.243 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.243 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.243 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:39.245 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:30:39.245 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:39.241+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:39.347 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:39.343+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:39.382 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:39.382 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:39.383 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:39.383 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:39.391 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:30:39.447 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:39.444+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:39.529 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:39.529 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:39.529 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:39.529 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:39.529 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:39.538 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:30:39.548 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:39.545+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:39.660 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:39.656+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:39.677 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:39.677 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:39.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:39.677 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:39.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.678 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:30:39.678 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:39.678 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 197.861326437, 2023-12-06T18:30:39.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:30:39.678 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:39.678 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:30:39.678 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:39.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:39.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:39.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:39.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:39.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:39.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:39.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:30:39.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:30:39.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:39.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:30:39.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:30:39.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:39.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:30:39.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:30:39.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:39.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:30:39.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:30:39.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:39.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:30:39.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:30:39.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:39.681 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:30:39.681 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:30:39.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:39.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:39.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:39.681 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:30:39.681 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:30:39.681 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.681 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.681 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.681 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:39.681 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:39.681 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:39.687 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:30:39.750 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:39.746+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:39.851 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:39.847+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:39.865 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:39.865 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:39.865 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:39.865 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:39.874 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:30:39.952 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:39.948+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:40.052 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:40.052 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:40.052 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:40.053 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:40.050+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:40.061 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:30:40.154 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:40.150+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:40.212 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:40.212 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:40.212 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:40.212 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:40.212 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:40.212 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:40.212 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:40.212 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 79.112389183000005, 2023-12-06T18:30:40.212 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:30:40.212 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:40.213 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:40.213 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:40.213 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:40.213 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:40.213 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:30:40.213 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:40.213 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:40.213 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:40.213 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:40.213 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:40.213 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:40.213 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:40.213 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:40.213 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:40.213 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:40.214 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:40.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:40.214 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:40.214 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:40.214 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:30:40.214 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:30:40.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:40.214 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:40.214 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:40.214 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:30:40.214 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:30:40.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:40.214 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:40.214 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:40.214 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:30:40.215 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:30:40.215 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:40.215 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:40.215 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:40.215 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:30:40.215 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:30:40.215 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:40.215 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:40.215 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:40.215 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:30:40.215 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:30:40.215 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:40.215 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:40.215 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:40.215 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:30:40.216 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:30:40.216 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:40.216 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:40.216 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:40.216 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:40.216 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:40.216 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:30:40.216 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:40.216 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 19.094131880999999, 2023-12-06T18:30:40.216 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:30:40.216 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:40.216 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:40.216 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:40.216 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:40.216 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:40.216 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:30:40.217 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:40.217 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:40.217 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:40.217 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:40.217 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:40.217 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:40.217 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:40.217 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:40.217 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:40.217 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:40.217 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:40.217 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:40.217 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:40.217 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:40.217 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:30:40.218 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:30:40.218 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:40.218 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:40.218 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:40.218 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:30:40.218 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:30:40.218 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:40.218 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:40.218 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:40.218 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:30:40.218 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:30:40.218 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:40.218 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:40.219 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:40.219 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:30:40.219 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:30:40.219 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:40.219 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:40.219 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:40.219 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:30:40.219 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:30:40.219 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:40.219 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:40.219 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:40.219 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:30:40.219 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:30:40.219 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:40.219 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:40.219 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:40.220 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:40.220 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:40.220 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:40.221 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:30:40.263 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:40.259+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:40.356 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:40.352+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:40.360 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:40.360 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:40.360 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:40.360 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:40.371 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:30:40.457 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:40.453+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:40.499 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:40.499 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:40.499 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:40.499 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:40.499 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:40.508 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:30:40.557 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:40.554+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:40.638 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:40.638 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:40.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:40.638 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:40.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:40.638 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:40.638 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:40.639 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 139.14469560800001, 2023-12-06T18:30:40.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:30:40.639 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:40.639 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:40.639 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:40.639 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:40.639 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:40.639 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:30:40.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:40.639 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:40.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:40.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:40.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:40.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:40.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:40.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:40.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:40.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:40.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:40.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:40.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:40.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:40.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:30:40.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:30:40.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:40.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:40.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:40.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:30:40.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:30:40.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:40.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:40.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:40.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:30:40.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:30:40.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:40.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:40.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:40.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:30:40.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:30:40.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:40.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:40.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:40.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:30:40.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:30:40.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:40.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:40.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:40.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:30:40.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:30:40.642 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:40.642 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:40.642 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:40.642 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:40.642 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:40.642 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:40.647 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:30:40.658 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:40.655+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:40.759 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:40.756+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:40.778 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:40.778 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:40.778 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:40.779 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:40.788 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:30:40.860 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:40.856+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:40.919 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:40.919 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:40.919 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:40.919 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:40.919 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:40.928 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:30:40.961 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:40.957+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:41.061 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:41.058+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:41.136 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:41.136 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:41.136 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:41.136 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:41.136 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.136 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:30:41.136 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:41.136 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 224.52105544299999, 2023-12-06T18:30:41.136 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:30:41.136 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.136 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.137 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.137 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.137 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.137 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:41.137 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.137 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.137 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.137 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.137 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:41.137 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.137 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.137 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.137 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.137 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:30:41.137 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:30:41.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.138 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:30:41.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:30:41.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.138 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:30:41.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:30:41.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:30:41.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:30:41.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:30:41.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:30:41.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:30:41.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:30:41.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.140 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:30:41.140 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:30:41.140 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.140 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.140 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.140 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:30:41.140 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:30:41.140 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.140 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.140 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.140 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.140 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.140 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:30:41.140 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:41.141 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 223.498228434, 2023-12-06T18:30:41.141 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:30:41.141 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.141 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.141 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.141 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.141 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.142 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:41.142 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.142 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.142 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.142 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.142 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:41.142 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.142 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.142 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.142 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.142 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:30:41.142 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:30:41.143 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.143 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.143 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.143 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:30:41.143 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:30:41.143 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.143 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.143 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.143 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:30:41.143 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:30:41.143 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.143 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.144 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.144 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:30:41.144 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:30:41.144 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.144 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.144 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.144 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:30:41.144 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:30:41.144 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.144 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.144 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.145 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:30:41.145 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:30:41.145 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.145 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.145 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.145 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:30:41.145 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:30:41.145 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.145 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.145 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.146 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:30:41.146 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:30:41.146 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.146 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.146 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.146 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.146 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.146 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:30:41.146 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:41.146 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 223.48618179600001, 2023-12-06T18:30:41.146 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:30:41.146 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.146 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.147 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.147 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.147 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:41.147 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.147 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.147 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.147 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:41.147 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.147 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.148 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:30:41.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:30:41.148 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.148 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.148 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:30:41.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:30:41.148 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.148 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.148 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:30:41.149 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:30:41.149 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.149 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.149 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.149 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:30:41.149 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:30:41.149 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.149 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.149 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.149 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:30:41.149 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:30:41.149 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.149 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.150 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.150 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:30:41.150 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:30:41.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.150 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.150 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:30:41.150 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:30:41.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.150 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.151 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:30:41.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:30:41.151 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.151 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.151 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.151 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.151 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:30:41.151 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:41.151 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 223.46478118900001, 2023-12-06T18:30:41.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:30:41.151 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.152 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.152 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.152 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.152 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.152 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:41.152 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.152 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.152 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.152 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:41.152 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.152 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.153 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.153 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:30:41.153 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:30:41.153 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.153 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.153 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.153 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:30:41.153 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:30:41.153 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.153 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.153 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.153 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:30:41.153 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:30:41.154 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.154 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.154 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.154 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:30:41.154 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:30:41.154 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.154 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.154 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.154 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:30:41.154 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:30:41.154 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.154 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.155 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.155 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:30:41.155 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:30:41.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.155 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.155 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.155 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:30:41.155 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:30:41.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.155 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.155 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.156 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:30:41.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:30:41.156 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.156 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.156 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.156 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.156 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:30:41.156 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:41.156 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 223.463601866, 2023-12-06T18:30:41.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:30:41.156 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.156 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.156 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.157 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.157 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.157 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:41.157 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.157 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.157 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.157 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:41.157 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.157 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.157 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.157 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:30:41.157 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:30:41.158 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.158 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.158 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.158 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:30:41.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:30:41.158 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.158 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.158 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.158 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:30:41.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:30:41.158 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.158 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:30:41.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:30:41.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:30:41.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:30:41.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:30:41.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:30:41.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.160 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.160 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:30:41.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:30:41.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.160 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.160 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:30:41.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:30:41.160 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.161 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.161 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.161 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.161 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:30:41.161 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:41.161 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 223.463029885, 2023-12-06T18:30:41.161 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:30:41.161 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.161 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.161 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.162 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.162 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:41.162 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.162 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.162 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.162 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.162 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:41.162 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.162 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.162 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.162 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.163 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:30:41.163 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:30:41.163 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.163 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.163 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.163 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:30:41.163 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:30:41.163 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.163 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.163 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.163 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:30:41.163 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:30:41.163 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.164 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.164 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.164 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:30:41.164 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:30:41.164 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.164 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.164 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.164 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:30:41.164 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:30:41.164 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.164 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.164 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.164 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:30:41.165 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:30:41.165 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.165 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.165 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.165 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:30:41.165 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:30:41.165 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.165 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.165 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.165 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:30:41.165 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:30:41.165 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.165 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.166 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.166 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.166 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.166 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:30:41.166 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:41.166 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 223.01719441500001, 2023-12-06T18:30:41.166 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:30:41.166 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.166 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.166 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.166 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.166 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.167 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:41.167 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.167 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.167 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.167 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.167 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:41.167 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.167 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.167 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.167 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.167 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:30:41.167 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:30:41.167 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.168 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.168 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.168 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:30:41.168 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:30:41.168 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.168 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.168 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.168 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:30:41.168 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:30:41.168 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.168 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.168 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.168 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:30:41.169 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:30:41.169 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.169 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.169 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.169 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:30:41.169 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:30:41.169 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.169 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.169 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.169 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:30:41.169 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:30:41.169 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.170 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.170 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.170 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:30:41.170 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:30:41.170 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.170 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.170 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.170 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:30:41.170 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:30:41.170 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.170 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.170 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.171 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.171 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.171 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:30:41.171 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:41.171 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 223.01559455899999, 2023-12-06T18:30:41.171 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:30:41.171 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.171 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.171 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.171 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.171 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.171 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:41.171 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.171 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.172 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.172 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.172 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:41.172 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.172 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.172 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.172 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.172 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:30:41.172 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:30:41.172 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.172 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.172 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.173 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:30:41.173 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:30:41.173 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.173 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.173 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.173 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:30:41.173 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:30:41.173 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.173 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.173 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.173 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:30:41.173 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:30:41.173 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.174 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.174 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.174 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:30:41.174 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:30:41.174 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.174 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.174 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.174 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:30:41.174 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:30:41.174 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.174 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.174 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.175 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:30:41.175 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:30:41.175 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.175 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.175 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.175 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:30:41.175 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:30:41.175 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.175 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.175 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.175 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.175 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.176 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:30:41.176 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:41.176 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 222.63638894299999, 2023-12-06T18:30:41.176 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:30:41.176 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.176 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.176 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.176 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.176 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.176 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:41.176 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.176 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.177 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.177 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.177 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:41.177 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.177 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.177 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.177 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.177 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:30:41.177 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:30:41.177 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.177 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.177 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.178 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:30:41.178 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:30:41.178 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.178 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.178 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.178 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:30:41.178 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:30:41.178 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.178 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.178 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.178 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:30:41.179 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:30:41.179 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.179 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.179 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.179 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:30:41.179 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:30:41.179 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.179 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.179 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.179 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:30:41.179 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:30:41.179 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.179 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.180 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.180 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:30:41.180 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:30:41.180 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.180 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.180 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.180 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:30:41.180 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:30:41.180 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.180 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.180 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.180 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.181 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.181 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:30:41.181 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:41.181 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 222.398886761, 2023-12-06T18:30:41.181 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:30:41.181 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.181 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.181 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.181 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.181 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.181 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:41.182 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.182 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.182 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.182 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.182 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:41.182 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.182 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.182 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.182 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.182 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:30:41.182 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:30:41.182 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.182 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.183 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.183 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:30:41.183 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:30:41.183 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.183 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.183 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.183 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:30:41.183 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:30:41.183 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.183 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.183 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.183 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:30:41.184 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:30:41.184 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.184 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.184 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.184 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:30:41.184 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:30:41.184 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.184 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.184 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.184 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:30:41.184 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:30:41.184 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.184 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.185 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.185 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:30:41.185 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:30:41.185 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.185 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.185 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.185 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:30:41.185 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:30:41.185 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.185 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.185 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.185 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.186 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.186 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:30:41.186 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:41.186 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 222.39815876200001, 2023-12-06T18:30:41.186 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:30:41.186 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.186 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.186 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.186 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.186 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.186 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:41.186 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.187 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.187 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.187 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.187 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:41.187 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.187 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.187 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.187 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.187 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:30:41.187 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:30:41.187 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.187 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.188 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.188 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:30:41.188 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:30:41.188 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.188 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.188 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.188 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:30:41.188 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:30:41.188 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.188 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.188 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.188 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:30:41.189 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:30:41.189 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.189 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.189 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.189 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:30:41.189 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:30:41.189 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.189 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.189 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.189 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:30:41.189 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:30:41.189 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.190 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.190 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.190 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:30:41.190 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:30:41.190 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.190 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.190 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.190 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:30:41.190 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:30:41.190 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.190 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.190 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.190 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.191 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.191 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:30:41.191 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:41.191 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 222.393146352, 2023-12-06T18:30:41.191 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:30:41.191 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.191 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.191 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.191 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.191 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.191 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:41.191 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.191 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.192 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.192 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.192 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:41.192 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.192 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.192 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.192 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.192 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:30:41.192 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:30:41.192 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.193 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.193 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.193 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:30:41.193 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:30:41.193 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.193 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.193 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.193 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:30:41.193 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:30:41.193 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.193 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.193 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.194 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:30:41.194 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:30:41.194 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.194 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.194 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.194 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:30:41.194 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:30:41.194 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.194 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.194 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.194 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:30:41.195 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:30:41.195 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.195 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.195 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.195 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:30:41.195 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:30:41.195 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.195 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.195 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.195 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:30:41.195 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:30:41.195 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.195 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.195 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.196 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.196 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.196 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:30:41.196 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:41.196 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 222.38761927300001, 2023-12-06T18:30:41.196 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:30:41.196 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.196 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.196 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.196 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.196 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.196 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:41.196 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.196 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.197 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.197 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.197 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:41.197 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.197 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.197 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.197 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.197 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:30:41.197 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:30:41.197 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.197 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.197 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.197 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:30:41.197 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:30:41.198 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.198 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.198 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.198 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:30:41.198 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:30:41.198 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.198 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.198 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.198 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:30:41.198 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:30:41.198 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.199 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.199 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.199 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:30:41.199 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:30:41.199 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.199 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.199 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.199 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:30:41.199 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:30:41.199 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.199 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.199 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.200 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:30:41.200 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:30:41.200 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.200 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.200 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.200 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:30:41.200 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:30:41.200 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.200 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.200 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.200 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.200 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.201 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:30:41.201 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:41.201 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 222.38710843600001, 2023-12-06T18:30:41.201 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:30:41.201 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.201 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.201 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.201 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.201 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.201 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:41.201 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.201 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.202 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.202 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.202 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:41.202 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.202 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.202 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.202 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.202 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:30:41.202 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:30:41.202 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.202 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.202 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.203 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:30:41.203 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:30:41.203 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.203 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.203 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.203 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:30:41.203 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:30:41.203 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.203 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.203 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.203 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:30:41.203 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:30:41.203 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.203 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.204 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.204 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:30:41.204 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:30:41.204 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.204 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.204 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.204 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:30:41.204 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:30:41.204 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.204 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.204 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.204 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:30:41.204 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:30:41.205 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.205 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.205 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.205 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:30:41.205 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:30:41.205 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.205 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.205 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.205 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.205 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.205 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:30:41.205 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:41.205 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 222.386649175, 2023-12-06T18:30:41.205 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:30:41.206 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.206 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.206 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.206 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.206 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.206 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:41.206 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.206 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.206 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.206 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.206 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:41.206 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.206 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.207 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.207 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.207 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:30:41.207 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:30:41.207 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.207 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.207 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.207 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:30:41.207 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:30:41.207 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.207 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.207 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.207 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:30:41.208 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:30:41.208 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.208 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.208 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.208 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:30:41.208 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:30:41.208 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.208 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.208 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.208 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:30:41.208 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:30:41.208 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.209 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.209 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.209 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:30:41.209 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:30:41.209 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.209 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.209 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.209 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:30:41.209 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:30:41.209 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.209 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.209 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.210 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:30:41.210 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:30:41.210 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.210 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.210 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.210 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.210 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.210 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:30:41.210 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:41.210 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 215.80673336800001, 2023-12-06T18:30:41.210 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:30:41.211 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.211 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.211 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.211 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.211 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.211 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:41.211 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.211 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.211 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.211 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.211 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:41.211 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.211 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.211 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.212 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.212 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:30:41.212 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:30:41.212 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.212 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.212 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.212 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:30:41.212 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:30:41.212 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.212 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.212 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.212 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:30:41.213 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:30:41.213 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.213 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.213 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.213 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:30:41.213 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:30:41.213 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.213 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.213 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.213 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:30:41.213 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:30:41.213 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.213 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.213 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.214 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:30:41.214 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:30:41.214 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.214 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.214 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.214 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:30:41.214 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:30:41.214 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.214 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.214 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.214 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:30:41.214 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:30:41.215 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.215 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.215 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.215 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.215 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.215 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:30:41.215 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:41.215 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 215.800495563, 2023-12-06T18:30:41.215 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:30:41.215 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.215 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.215 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.215 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.216 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.216 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:41.216 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.216 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.216 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.216 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.216 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:41.216 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.216 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.216 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.216 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.216 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:30:41.216 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:30:41.217 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.217 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.217 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.217 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:30:41.217 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:30:41.217 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.217 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.217 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.217 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:30:41.217 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:30:41.217 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.217 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.217 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.217 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:30:41.218 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:30:41.218 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.218 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.218 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.218 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:30:41.218 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:30:41.218 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.218 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.218 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.218 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:30:41.218 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:30:41.218 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.218 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.219 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.219 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:30:41.219 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:30:41.219 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.219 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.219 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.219 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:30:41.219 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:30:41.219 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.219 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.219 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.219 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.219 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.220 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:30:41.220 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:41.220 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 215.794432484, 2023-12-06T18:30:41.220 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:30:41.220 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.220 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.220 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.220 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.220 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.220 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:41.220 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.220 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.220 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.221 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.221 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:41.221 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.221 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.221 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.221 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.221 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:30:41.221 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:30:41.221 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.221 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.221 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.221 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:30:41.221 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:30:41.222 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.222 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.222 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.222 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:30:41.222 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:30:41.222 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.222 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.222 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.222 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:30:41.223 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:30:41.223 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.223 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.223 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.223 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:30:41.223 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:30:41.223 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.223 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.223 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.223 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:30:41.224 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:30:41.224 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.224 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.224 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.224 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:30:41.224 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:30:41.224 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.224 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.224 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.224 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:30:41.224 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:30:41.224 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.224 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.224 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.225 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.225 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.225 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:30:41.225 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:41.225 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 215.657045399, 2023-12-06T18:30:41.225 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:30:41.225 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.225 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.225 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.225 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.225 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.225 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:41.226 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.226 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.226 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.226 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.226 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:41.226 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.226 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.226 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.226 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.226 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:30:41.226 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:30:41.226 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.227 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.227 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.227 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:30:41.227 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:30:41.227 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.227 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.227 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.227 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:30:41.227 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:30:41.227 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.227 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.228 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.228 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:30:41.228 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:30:41.228 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.228 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.228 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.228 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:30:41.228 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:30:41.228 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.228 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.228 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.228 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:30:41.229 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:30:41.229 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.229 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.229 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.229 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:30:41.229 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:30:41.229 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.229 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.229 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.229 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:30:41.229 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:30:41.229 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.230 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.230 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.230 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.230 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.230 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:30:41.230 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:41.230 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 215.650374672, 2023-12-06T18:30:41.230 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:30:41.230 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:41.230 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:41.230 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:41.231 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.231 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:41.231 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:41.231 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.231 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.231 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.231 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:41.231 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:41.231 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:41.231 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.232 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.232 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:41.232 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:30:41.232 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:30:41.232 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.232 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.232 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:41.232 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:30:41.232 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:30:41.232 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.232 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.232 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:41.232 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:30:41.233 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:30:41.233 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.233 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.233 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:41.233 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:30:41.233 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:30:41.233 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.233 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.233 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:41.233 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:30:41.233 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:30:41.233 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.233 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.234 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:41.234 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:30:41.234 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:30:41.234 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.234 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.234 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:41.234 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:30:41.234 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:30:41.234 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:41.234 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:41.234 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:41.234 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:30:41.235 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:30:41.235 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.235 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.235 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.235 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:41.235 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:41.235 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:41.237 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:30:41.237 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:41.233+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:41.337 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:41.333+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:41.407 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:41.407 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:41.407 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:41.407 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:41.416 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:30:41.437 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:41.434+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:41.538 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:41.535+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:41.543 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:41.543 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:41.543 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:41.543 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:41.543 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:41.553 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:30:41.639 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:41.635+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:41.680 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:41.681 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:41.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:41.681 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:30:41.681 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:41.689 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:30:41.740 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:41.736+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:41.818 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:41.818 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:41.818 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:41.818 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:41.827 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:30:41.841 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:41.837+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:41.942 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:41.939+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:41.957 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:41.957 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:41.957 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:41.957 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:41.957 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:41.966 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:30:42.043 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:42.039+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:42.136 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:30:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 20.474673333999998, 2023-12-06T18:30:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:30:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:30:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:30:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:30:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:30:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:30:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:30:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:30:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:30:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:30:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:30:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:30:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:30:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:30:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:42.141 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:42.141 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:42.144 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:42.141+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:42.145 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:30:42.245 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:42.242+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:42.299 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:42.299 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:42.299 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:42.299 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:42.308 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:30:42.352 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:42.348+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:42.446 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:42.443+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:42.466 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:42.466 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:42.466 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:42.466 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:42.466 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:42.476 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:30:42.547 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:42.544+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:42.649 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:42.645+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:42.678 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:42.678 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:42.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:42.678 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:30:42.678 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:42.688 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:30:42.772 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:42.768+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:42.851 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:42.847+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:42.898 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:42.898 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:42.898 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:42.898 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:42.907 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:30:42.992 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:42.988+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:43.052 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:43.048+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:43.112 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:43.112 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:43.112 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:43.112 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:43.112 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:43.121 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:30:43.162 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:43.159+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:43.254 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:43.251+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:43.355 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:43.352+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:43.363 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:43.363 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:43.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:43.363 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:43.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.363 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:30:43.363 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:43.364 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 227.94861017700001, 2023-12-06T18:30:43.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:30:43.364 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.364 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.364 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.364 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.364 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.364 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:30:43.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.364 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:43.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.365 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:43.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.365 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.365 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:30:43.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:30:43.365 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.365 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:30:43.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:30:43.365 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.366 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.366 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.366 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:30:43.366 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:30:43.366 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.366 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.366 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.366 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:30:43.366 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:30:43.366 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.366 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.366 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.366 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:30:43.366 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:30:43.366 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.367 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.367 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:30:43.367 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:30:43.367 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:30:43.367 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.367 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.367 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.367 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:30:43.367 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:30:43.367 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.367 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.367 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.367 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:30:43.367 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:30:43.367 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.367 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.368 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:30:43.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:30:43.368 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.368 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.368 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:30:43.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:30:43.368 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.368 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.368 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:30:43.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:30:43.368 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.368 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.369 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:30:43.369 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:30:43.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.369 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.369 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.369 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:30:43.369 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:30:43.369 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.369 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.369 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.369 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.369 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:43.369 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:43.370 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 225.840036938, 2023-12-06T18:30:43.370 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:30:43.370 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.370 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.370 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.370 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.370 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.370 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:30:43.370 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.370 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.370 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.370 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.370 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:43.370 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.370 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.371 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.371 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.371 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:43.371 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.371 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.371 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.371 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.371 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:30:43.371 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:30:43.371 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.371 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.371 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.371 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:30:43.371 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:30:43.372 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.372 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.372 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.372 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:30:43.372 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:30:43.372 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.372 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.372 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.372 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:30:43.372 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:30:43.372 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.372 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.372 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.372 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:30:43.372 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:30:43.372 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.373 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.373 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:30:43.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:30:43.373 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.373 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.373 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:30:43.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:30:43.373 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.373 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.373 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:30:43.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:30:43.373 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:30:43.374 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:30:43.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:30:43.374 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:30:43.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:30:43.374 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:30:43.375 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.375 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.375 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.375 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:43.375 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:43.375 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 225.83929170600001, 2023-12-06T18:30:43.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:30:43.375 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.375 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.375 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.375 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.375 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.375 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:30:43.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.376 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:43.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:43.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:30:43.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:30:43.377 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.377 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.377 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:30:43.377 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:30:43.377 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.377 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.377 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:30:43.377 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:30:43.377 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.377 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.377 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:30:43.377 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:30:43.378 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.378 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.378 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.378 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:30:43.378 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:30:43.378 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.378 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.378 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.378 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:30:43.378 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:30:43.378 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.378 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.378 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.378 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:30:43.378 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:30:43.379 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.379 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.379 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.379 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:30:43.379 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:30:43.379 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.379 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.379 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.379 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:30:43.379 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:30:43.379 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.379 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.379 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.379 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:30:43.380 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:30:43.380 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.380 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.380 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.380 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:30:43.380 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:30:43.380 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.380 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.380 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.380 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.380 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.380 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:43.380 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:43.380 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 225.838607951, 2023-12-06T18:30:43.380 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:30:43.381 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.381 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.381 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.381 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.381 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.381 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:30:43.381 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.381 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.381 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.381 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.381 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:43.381 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.381 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.381 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.381 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.381 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:43.382 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.382 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.382 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.382 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.382 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:30:43.382 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:30:43.382 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.382 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.382 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.382 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:30:43.382 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:30:43.382 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.382 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.382 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.382 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:30:43.383 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:30:43.383 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.383 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.383 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.383 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:30:43.383 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:30:43.383 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.383 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.383 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.383 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:30:43.383 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:30:43.383 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.383 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.383 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.383 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:30:43.384 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:30:43.384 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.384 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.384 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.384 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:30:43.384 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:30:43.384 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.384 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.384 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.384 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:30:43.384 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:30:43.384 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.384 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.384 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.384 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:30:43.385 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:30:43.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.385 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.385 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.385 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:30:43.385 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:30:43.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.385 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.385 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.385 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:30:43.385 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:30:43.385 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.385 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.385 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.385 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.386 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:43.386 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:43.386 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 225.833034696, 2023-12-06T18:30:43.386 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:30:43.386 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.386 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.386 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.386 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.386 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.386 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:30:43.386 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.386 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.386 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.386 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.386 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:43.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.387 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.387 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.387 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.387 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:43.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.387 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.387 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.387 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.387 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:30:43.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:30:43.387 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.387 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.387 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.387 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:30:43.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:30:43.388 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.388 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.388 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.388 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:30:43.388 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:30:43.388 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.388 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.388 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.388 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:30:43.388 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:30:43.388 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.388 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.388 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.388 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:30:43.388 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:30:43.389 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.389 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.389 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.389 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:30:43.389 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:30:43.389 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.389 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.389 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.389 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:30:43.389 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:30:43.389 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.389 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.389 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.389 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:30:43.389 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:30:43.390 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.390 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.390 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.390 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:30:43.390 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:30:43.390 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.390 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.390 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.390 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:30:43.390 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:30:43.390 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.390 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.390 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.390 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:30:43.391 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:30:43.391 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.391 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.391 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.391 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.391 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.391 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:43.391 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:43.391 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 225.832361056, 2023-12-06T18:30:43.391 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:30:43.391 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.391 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.391 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.391 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.392 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.392 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:30:43.392 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.392 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.392 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.392 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.392 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:43.392 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.392 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.392 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.392 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.392 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:43.392 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.392 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.392 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.392 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.393 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:30:43.393 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:30:43.393 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.393 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.393 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.393 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:30:43.393 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:30:43.393 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.393 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.393 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.393 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:30:43.393 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:30:43.393 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.393 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.393 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.394 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:30:43.394 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:30:43.394 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.394 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.394 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.394 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:30:43.394 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:30:43.394 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.394 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.395 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.395 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:30:43.395 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:30:43.395 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.395 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.395 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.395 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:30:43.395 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:30:43.395 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.395 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.395 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.395 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:30:43.395 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:30:43.395 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.395 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.396 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.396 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:30:43.396 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:30:43.396 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.396 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.396 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.396 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:30:43.396 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:30:43.396 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.396 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.396 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.396 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:30:43.396 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:30:43.396 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.396 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.396 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.397 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.397 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.397 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:43.397 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:43.397 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 225.83169885699999, 2023-12-06T18:30:43.397 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:30:43.397 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.397 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.397 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.397 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.397 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.397 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:30:43.397 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.397 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.397 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.398 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.398 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:43.398 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.398 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.398 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.398 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.398 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:43.398 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.398 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.398 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.398 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.399 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:30:43.399 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:30:43.399 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.399 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.399 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.399 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:30:43.399 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:30:43.399 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.399 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.399 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.399 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:30:43.399 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:30:43.399 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.399 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.399 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.400 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:30:43.400 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:30:43.400 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.400 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.400 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.400 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:30:43.400 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:30:43.400 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.400 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.400 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.400 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:30:43.400 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:30:43.400 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.400 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.400 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.401 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:30:43.401 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:30:43.401 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.401 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.401 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.401 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:30:43.401 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:30:43.401 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.401 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.401 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.401 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:30:43.401 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:30:43.401 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.401 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.401 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.402 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:30:43.402 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:30:43.402 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.402 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.402 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.402 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:30:43.402 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:30:43.402 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.402 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.402 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.402 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.402 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.402 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:43.402 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:43.402 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 225.75898942200001, 2023-12-06T18:30:43.403 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:30:43.403 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.403 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.403 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.403 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.403 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.403 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:30:43.403 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.403 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.403 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.403 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:43.403 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.403 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.404 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:43.404 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.404 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.404 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:30:43.404 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:30:43.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.404 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.404 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:30:43.404 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:30:43.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.404 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.405 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.405 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:30:43.405 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:30:43.405 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.405 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.405 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.405 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:30:43.405 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:30:43.405 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.405 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.405 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.405 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:30:43.405 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:30:43.405 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.405 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.405 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.406 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:30:43.406 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:30:43.406 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.406 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.406 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.406 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:30:43.406 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:30:43.406 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.406 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.406 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.406 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:30:43.406 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:30:43.406 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.406 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.406 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:30:43.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:30:43.407 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.407 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:30:43.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:30:43.407 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.407 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:30:43.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:30:43.407 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.407 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.407 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.408 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:43.408 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:43.408 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 225.75830771299999, 2023-12-06T18:30:43.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:30:43.408 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.408 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.408 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.408 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.408 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.408 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:30:43.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.408 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.409 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.409 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:43.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.409 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.409 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.409 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:43.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.409 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.409 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.409 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:30:43.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:30:43.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.409 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.409 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.410 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:30:43.410 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:30:43.410 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.410 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.410 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.410 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:30:43.410 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:30:43.410 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.410 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.410 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.410 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:30:43.410 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:30:43.410 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.410 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.410 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.411 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:30:43.411 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:30:43.411 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.411 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.411 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.411 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:30:43.411 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:30:43.411 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.411 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.411 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.411 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:30:43.411 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:30:43.411 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.411 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.411 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:30:43.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:30:43.412 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:30:43.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:30:43.412 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:30:43.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:30:43.412 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:30:43.413 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:30:43.413 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.413 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.413 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.413 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.413 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.413 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:43.413 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:43.413 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 224.679503625, 2023-12-06T18:30:43.413 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:30:43.413 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.413 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.413 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.413 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.413 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.413 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:30:43.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.414 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.414 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.414 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.414 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:43.414 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.414 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.414 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.414 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:43.414 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.414 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.414 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.415 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:30:43.415 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:30:43.415 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.415 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.415 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.415 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:30:43.415 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:30:43.415 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.415 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.415 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.415 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:30:43.415 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:30:43.415 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.415 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.415 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.416 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:30:43.416 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:30:43.416 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.416 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.416 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.416 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:30:43.416 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:30:43.416 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.416 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.416 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.416 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:30:43.416 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:30:43.416 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.416 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.416 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.416 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:30:43.417 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:30:43.417 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.417 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.417 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.417 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:30:43.417 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:30:43.417 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.417 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.417 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.417 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:30:43.417 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:30:43.417 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.417 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.417 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.417 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:30:43.418 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:30:43.418 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.418 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.418 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.418 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:30:43.418 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:30:43.418 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.418 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.418 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.418 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.418 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.418 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:43.418 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:43.418 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 224.67808705100001, 2023-12-06T18:30:43.418 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:30:43.419 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.419 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.419 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.419 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.419 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.419 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:30:43.419 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.419 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.419 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.419 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.419 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:43.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.419 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.419 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.419 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:43.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:30:43.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:30:43.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:30:43.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:30:43.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:30:43.421 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:30:43.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.421 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.421 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.421 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:30:43.421 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:30:43.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.421 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.421 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.421 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:30:43.421 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:30:43.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.421 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.421 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.421 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:30:43.421 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:30:43.422 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.422 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.422 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.422 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:30:43.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:30:43.422 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.422 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.422 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.422 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:30:43.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:30:43.422 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.422 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.422 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.422 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:30:43.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:30:43.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.423 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.423 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:30:43.423 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:30:43.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.423 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.423 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:30:43.423 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:30:43.423 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.423 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.423 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.424 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:43.424 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:43.424 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 224.677001926, 2023-12-06T18:30:43.424 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:30:43.424 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.424 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.424 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.424 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.424 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.424 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:30:43.424 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.424 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.424 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.424 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:43.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:43.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:30:43.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:30:43.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:30:43.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:30:43.426 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.426 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.426 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.426 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:30:43.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:30:43.426 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.426 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.426 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.426 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:30:43.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:30:43.426 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.426 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.426 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.426 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:30:43.427 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:30:43.427 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.427 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.427 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.427 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:30:43.427 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:30:43.427 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.427 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.427 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.427 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:30:43.427 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:30:43.427 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.427 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.427 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.428 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:30:43.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:30:43.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.428 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.428 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.428 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:30:43.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:30:43.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.428 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.428 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.428 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:30:43.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:30:43.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.428 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.428 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.429 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:30:43.429 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:30:43.429 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.429 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.429 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.429 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.429 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.429 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:43.429 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:43.429 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 224.64185282400001, 2023-12-06T18:30:43.429 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:30:43.429 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.429 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.429 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.429 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.429 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.430 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:30:43.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.430 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.430 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.430 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.430 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:43.430 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.430 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.430 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.430 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:43.430 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.430 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.430 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.431 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:30:43.431 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:30:43.431 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.431 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.431 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.431 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:30:43.431 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:30:43.431 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.431 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.431 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.431 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:30:43.431 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:30:43.431 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.431 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.431 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.431 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:30:43.432 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:30:43.432 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.432 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.432 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.432 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:30:43.432 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:30:43.432 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.432 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.432 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.432 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:30:43.432 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:30:43.432 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.432 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.432 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:30:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:30:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:30:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:30:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:30:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:30:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.433 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:30:43.434 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:30:43.434 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.434 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.434 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.434 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:30:43.434 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:30:43.434 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.434 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.434 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.434 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.434 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.434 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:43.434 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:43.434 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 218.09212229900001, 2023-12-06T18:30:43.434 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:30:43.435 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.435 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.435 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.435 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.435 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.435 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:30:43.435 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.435 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.435 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.435 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.435 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:43.435 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.435 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.435 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.435 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.435 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:43.435 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.436 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.436 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.436 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:30:43.436 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:30:43.436 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.436 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.436 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:30:43.436 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:30:43.436 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.436 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.436 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:30:43.437 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:30:43.437 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.437 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.437 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.437 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:30:43.437 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:30:43.437 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.437 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.437 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.437 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:30:43.437 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:30:43.437 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.437 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.437 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.437 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:30:43.437 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:30:43.438 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.438 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.438 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.438 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:30:43.438 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:30:43.438 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.438 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.438 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.438 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:30:43.438 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:30:43.438 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.438 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.438 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.438 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:30:43.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:30:43.439 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.439 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.439 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.439 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:30:43.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:30:43.439 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.439 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.439 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.439 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:30:43.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:30:43.439 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.439 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.439 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.439 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.440 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.440 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:43.440 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:43.440 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 218.091316691, 2023-12-06T18:30:43.440 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:30:43.440 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.440 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.440 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.440 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.440 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.440 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:30:43.440 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.440 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.440 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.440 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:43.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.441 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:43.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.441 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:30:43.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:30:43.441 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:30:43.442 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:30:43.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.442 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.442 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.442 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:30:43.442 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:30:43.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.442 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.442 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.442 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:30:43.442 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:30:43.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.442 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.442 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.443 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:30:43.443 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:30:43.443 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.443 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.443 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.443 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:30:43.443 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:30:43.443 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.443 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.443 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.443 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:30:43.443 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:30:43.443 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.443 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.443 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.443 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:30:43.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:30:43.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.444 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.444 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:30:43.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:30:43.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.444 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.444 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:30:43.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:30:43.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.444 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.444 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:30:43.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:30:43.445 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.445 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.445 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.445 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.445 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.445 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:43.445 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:43.445 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 218.09062588200001, 2023-12-06T18:30:43.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:30:43.445 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.445 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.445 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.445 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.446 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.446 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:30:43.446 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.446 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.446 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.446 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:43.446 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.446 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.446 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.446 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:43.446 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.446 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.447 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.447 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:30:43.447 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:30:43.447 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.447 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.447 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.447 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:30:43.447 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:30:43.447 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.447 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.447 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.447 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:30:43.447 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:30:43.447 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.447 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.448 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.448 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:30:43.448 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:30:43.448 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.448 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.448 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.448 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:30:43.448 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:30:43.448 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.448 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.448 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.448 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:30:43.448 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:30:43.448 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.448 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.448 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:30:43.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:30:43.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:30:43.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:30:43.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:30:43.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:30:43.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:30:43.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:30:43.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.450 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.450 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:30:43.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:30:43.450 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.450 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.450 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.450 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:43.450 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:43.450 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 218.086328374, 2023-12-06T18:30:43.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:30:43.451 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.451 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.451 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.451 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.451 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.451 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:30:43.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.451 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.451 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.451 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:43.451 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.451 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:43.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:30:43.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:30:43.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:30:43.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:30:43.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.453 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:30:43.453 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:30:43.453 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.453 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.453 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.453 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:30:43.453 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:30:43.453 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.453 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.453 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.453 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:30:43.453 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:30:43.453 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.453 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.453 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.453 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:30:43.454 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:30:43.454 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.454 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.454 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.454 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:30:43.454 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:30:43.454 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.454 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.454 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.454 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:30:43.454 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:30:43.454 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.454 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.454 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.454 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:30:43.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:30:43.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:30:43.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:30:43.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:30:43.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:30:43.455 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.455 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.455 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.456 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:43.456 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:43.456 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 218.08566069899999, 2023-12-06T18:30:43.456 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:30:43.456 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.456 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.456 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.456 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.456 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.456 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:30:43.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.456 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.456 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:43.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:43.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:30:43.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:30:43.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:30:43.458 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:30:43.458 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.458 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.458 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.458 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:30:43.458 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:30:43.458 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.458 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.458 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.458 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:30:43.458 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:30:43.458 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.458 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.458 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.458 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:30:43.459 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:30:43.459 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.459 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.459 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.459 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:30:43.459 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:30:43.459 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.459 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.459 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.459 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:30:43.459 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:30:43.459 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.459 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.459 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.459 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:30:43.459 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:30:43.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:30:43.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:30:43.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:30:43.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:30:43.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:30:43.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:30:43.461 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.461 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.461 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.461 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:43.461 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:43.461 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 218.085005271, 2023-12-06T18:30:43.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:30:43.461 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.461 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.461 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.461 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.461 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.461 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:30:43.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.462 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:43.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:43.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:30:43.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:30:43.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:30:43.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:30:43.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:30:43.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:30:43.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:30:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:30:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:30:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:30:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:30:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:30:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:30:43.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:30:43.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:30:43.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:30:43.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:30:43.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:30:43.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:30:43.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:30:43.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:30:43.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:30:43.466 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.466 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.466 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.466 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:43.466 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:43.466 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 218.044350724, 2023-12-06T18:30:43.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:30:43.467 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.467 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:43.467 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:43.467 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:43.467 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:43.467 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:30:43.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.467 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:43.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:43.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:30:43.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:30:43.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:30:43.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:30:43.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:30:43.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:30:43.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:30:43.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:30:43.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:30:43.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:30:43.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:30:43.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:30:43.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:43.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:30:43.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:30:43.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:43.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:30:43.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:30:43.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:43.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:30:43.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:30:43.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:43.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:30:43.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:30:43.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:30:43.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:30:43.471 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.471 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.471 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.471 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.472 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.472 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:43.473 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:30:43.474 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:43.470+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:43.574 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:43.571+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:43.611 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:43.611 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:43.611 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:43.611 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:43.620 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:30:43.675 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:43.672+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:43.776 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:43.773+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:43.795 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:43.795 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:43.805 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:30:43.890 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:43.886+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:43.979 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:43.975+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:43.987 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:43.987 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:43.987 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:43.987 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:43.987 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.987 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:30:43.987 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:43.987 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 202.17132184600001, 2023-12-06T18:30:43.987 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:30:43.987 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:43.988 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:30:43.988 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:43.988 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.988 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:43.988 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:43.988 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.988 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.988 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.988 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:43.988 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:43.988 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:43.988 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.988 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.988 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:43.988 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:30:43.989 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:30:43.989 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.989 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.989 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:43.989 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:30:43.989 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:30:43.989 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.989 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.989 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:43.989 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:30:43.989 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:30:43.989 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.989 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.989 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:43.989 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:30:43.990 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:30:43.990 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.990 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.990 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:43.990 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:30:43.990 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:30:43.990 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.990 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.990 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:43.990 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:30:43.990 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:30:43.990 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:43.990 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:43.991 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:43.991 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:30:43.991 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:30:43.991 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.991 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.991 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.991 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:43.991 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:43.991 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:43.996 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:30:44.080 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:44.076+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:44.175 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:44.176 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:44.176 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:44.176 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:44.180 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:44.177+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:44.184 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:30:44.281 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:44.278+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:44.362 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:44.362 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:44.362 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:44.362 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:44.362 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:44.371 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:30:44.382 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:44.379+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:44.483 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:44.480+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:44.511 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 83.411971284000003, 2023-12-06T18:30:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:30:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:30:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:30:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:30:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:30:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:30:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:30:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:30:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:30:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:30:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:30:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:30:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:30:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:30:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:30:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 23.393713982000001, 2023-12-06T18:30:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:30:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:30:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:30:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:30:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:30:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:30:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:30:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:30:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:30:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:30:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:30:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:30:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:30:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:30:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:44.519 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:44.520 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:30:44.585 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:44.581+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:44.660 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:44.661 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:44.661 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:44.661 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:44.670 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:30:44.714 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:44.710+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:44.787 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:44.783+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:44.798 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:44.798 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:44.798 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:44.798 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:44.799 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:44.808 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:30:44.887 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:44.884+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:44.980 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:44.980 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:44.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:44.980 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:44.980 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:44.980 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:44.980 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:44.980 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 143.486531058, 2023-12-06T18:30:44.980 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:30:44.980 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:44.980 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:44.980 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:44.981 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:44.981 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:44.981 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:30:44.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:44.981 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:44.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:44.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:44.981 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:44.981 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:44.981 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:44.981 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:44.981 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:44.981 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:44.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:44.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:44.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:44.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:44.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:30:44.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:30:44.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:44.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:44.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:44.982 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:30:44.982 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:30:44.982 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:44.982 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:44.982 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:44.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:30:44.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:30:44.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:44.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:44.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:44.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:30:44.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:30:44.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:44.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:44.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:44.983 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:30:44.983 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:30:44.983 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:44.983 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:44.983 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:44.984 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:30:44.984 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:30:44.984 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:44.984 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:44.984 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:44.984 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:44.984 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:44.984 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:44.989 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:30:44.989 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:44.985+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:45.090 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:45.086+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:45.118 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:45.118 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:45.118 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:45.118 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:45.127 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:30:45.190 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:45.187+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:45.291 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:45.288+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:45.294 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:45.294 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:45.294 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:45.294 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:45.294 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:45.303 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:30:45.394 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:45.389+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:45.493 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:45.489+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:45.513 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:45.513 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:45.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:45.513 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:45.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.513 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:30:45.513 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:45.513 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 228.89599940700001, 2023-12-06T18:30:45.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:30:45.514 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.514 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.514 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:45.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:45.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:30:45.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:30:45.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:30:45.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:30:45.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:30:45.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:30:45.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:30:45.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:30:45.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:30:45.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:30:45.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:30:45.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:30:45.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:30:45.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:30:45.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:30:45.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:30:45.517 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.517 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.517 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.517 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:30:45.518 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:45.518 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 227.87317239800001, 2023-12-06T18:30:45.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:30:45.518 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.518 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.518 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:45.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:45.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:30:45.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:30:45.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:30:45.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:30:45.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:30:45.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:30:45.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:30:45.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:30:45.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:30:45.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:30:45.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:30:45.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:30:45.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:30:45.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:30:45.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:30:45.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:30:45.521 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.521 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.522 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.522 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:30:45.522 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:45.522 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 227.86112575999999, 2023-12-06T18:30:45.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:30:45.522 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.522 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.522 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.522 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.522 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:45.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:45.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:30:45.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:30:45.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:30:45.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:30:45.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:30:45.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:30:45.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:30:45.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:30:45.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:30:45.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:30:45.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:30:45.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:30:45.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:30:45.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:30:45.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:30:45.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:30:45.526 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.526 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.526 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.526 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:30:45.526 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:45.526 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 227.83972515299999, 2023-12-06T18:30:45.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:30:45.526 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.526 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.526 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:45.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:45.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:30:45.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:30:45.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:30:45.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:30:45.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:30:45.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:30:45.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:30:45.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:30:45.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:30:45.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:30:45.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:30:45.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:30:45.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:30:45.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:30:45.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.530 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:30:45.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:30:45.530 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.530 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.530 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.530 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:30:45.530 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:45.530 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 227.83854582999999, 2023-12-06T18:30:45.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:30:45.531 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.531 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.531 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:45.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:45.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:30:45.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:30:45.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:30:45.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:30:45.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:30:45.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:30:45.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.533 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.533 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:30:45.533 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:30:45.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.533 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.533 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:30:45.533 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:30:45.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.533 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.533 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:30:45.533 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:30:45.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.534 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.534 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.534 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:30:45.534 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:30:45.534 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.534 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.534 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.534 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:30:45.534 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:30:45.534 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.534 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.534 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.535 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.535 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.535 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:30:45.535 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:45.535 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 227.83797384900001, 2023-12-06T18:30:45.535 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:30:45.535 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.535 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.535 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.535 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.535 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.535 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:45.535 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.535 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:45.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:30:45.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:30:45.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:30:45.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:30:45.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.537 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:30:45.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:30:45.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.537 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:30:45.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:30:45.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.537 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:30:45.538 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:30:45.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.538 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.538 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.538 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:30:45.538 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:30:45.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.538 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.538 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.538 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:30:45.538 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:30:45.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.538 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.538 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.539 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:30:45.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:30:45.539 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.539 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.539 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.539 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.539 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:30:45.539 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:45.539 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 227.39213837899999, 2023-12-06T18:30:45.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:30:45.539 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.539 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.539 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:45.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:45.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:30:45.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:30:45.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:30:45.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:30:45.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:30:45.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:30:45.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:30:45.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:30:45.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.542 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.542 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.542 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:30:45.542 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:30:45.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.542 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.542 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.542 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:30:45.542 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:30:45.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.542 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.542 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.542 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:30:45.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:30:45.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.543 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.543 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:30:45.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:30:45.543 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.543 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.543 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.543 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:30:45.543 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:45.543 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 227.390538523, 2023-12-06T18:30:45.544 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:30:45.544 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.544 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.544 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.544 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.544 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.544 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:45.544 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.544 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.544 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.544 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:45.544 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.545 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:30:45.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:30:45.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.545 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:30:45.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:30:45.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.545 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:30:45.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:30:45.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.546 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.546 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:30:45.546 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:30:45.546 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.546 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.546 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.546 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:30:45.546 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:30:45.546 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.546 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.546 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.546 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:30:45.546 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:30:45.546 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.547 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.547 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:30:45.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:30:45.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.547 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.547 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:30:45.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:30:45.547 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.547 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.547 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.547 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:30:45.548 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:45.548 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 227.011332907, 2023-12-06T18:30:45.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:30:45.548 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.548 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.548 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.548 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:45.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.548 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:45.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:30:45.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:30:45.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:30:45.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:30:45.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:30:45.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:30:45.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.550 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.550 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.550 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:30:45.550 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:30:45.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.550 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.550 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.550 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:30:45.550 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:30:45.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.551 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.551 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.551 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:30:45.551 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:30:45.551 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.551 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.551 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.551 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:30:45.551 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:30:45.551 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.551 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.551 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.551 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:30:45.551 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:30:45.551 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.552 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.552 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.552 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:30:45.552 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:45.552 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 226.77383072500001, 2023-12-06T18:30:45.552 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:30:45.552 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.552 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.552 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.552 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.552 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:45.552 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:45.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:30:45.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:30:45.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:30:45.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:30:45.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.554 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.554 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:30:45.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:30:45.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.554 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.554 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:30:45.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:30:45.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.554 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.554 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:30:45.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:30:45.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.555 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.555 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.555 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:30:45.555 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:30:45.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.555 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.555 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.555 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:30:45.555 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:30:45.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.555 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.555 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.555 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:30:45.555 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:30:45.555 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.556 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.556 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.556 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.556 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:30:45.556 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:45.556 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 226.77310272599999, 2023-12-06T18:30:45.556 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:30:45.556 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.556 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.556 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.556 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.556 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.556 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:45.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.557 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:45.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.557 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:30:45.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:30:45.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.557 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:30:45.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:30:45.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.558 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.558 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.558 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:30:45.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:30:45.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.558 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.558 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.558 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:30:45.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:30:45.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.559 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.559 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.559 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:30:45.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:30:45.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.559 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.559 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.559 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:30:45.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:30:45.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.559 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.559 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.559 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:30:45.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:30:45.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.560 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.560 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.560 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:30:45.560 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:30:45.560 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.560 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.560 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.560 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.560 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.560 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:30:45.560 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:45.560 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 226.76809031600001, 2023-12-06T18:30:45.560 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:30:45.560 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.560 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.560 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.561 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.561 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.561 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:45.561 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.561 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.561 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.561 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.561 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:45.561 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.561 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.561 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.561 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.561 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:30:45.561 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:30:45.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.562 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.562 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.562 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:30:45.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:30:45.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.562 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.562 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.562 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:30:45.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:30:45.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.562 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.562 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.562 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:30:45.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:30:45.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.563 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.563 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.563 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:30:45.563 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:30:45.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.563 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.563 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.563 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:30:45.563 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:30:45.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.563 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.563 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.563 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:30:45.563 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:30:45.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.563 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.563 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.564 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:30:45.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:30:45.564 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.564 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.564 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.564 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:30:45.564 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:45.564 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 226.76256323699999, 2023-12-06T18:30:45.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:30:45.564 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.564 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.564 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.564 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.564 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:45.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.565 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:45.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.565 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:30:45.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:30:45.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.565 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:30:45.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:30:45.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:30:45.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:30:45.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:30:45.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:30:45.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:30:45.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:30:45.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:30:45.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:30:45.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:30:45.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:30:45.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.568 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.568 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:30:45.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:30:45.568 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.568 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.568 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.568 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:30:45.568 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:45.568 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 226.76205239999999, 2023-12-06T18:30:45.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:30:45.568 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.568 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.568 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:45.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:45.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:30:45.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:30:45.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:30:45.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:30:45.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:30:45.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:30:45.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:30:45.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:30:45.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.571 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:30:45.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:30:45.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.571 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:30:45.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:30:45.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.571 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:30:45.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:30:45.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.572 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.572 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:30:45.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:30:45.572 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.572 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.572 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.572 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:30:45.572 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:45.572 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 226.76159313900001, 2023-12-06T18:30:45.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:30:45.572 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.572 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.573 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.573 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.573 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.573 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:45.573 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.573 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.573 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.573 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:45.573 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.573 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.573 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.573 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:30:45.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:30:45.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:30:45.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:30:45.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:30:45.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:30:45.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:30:45.575 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:30:45.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.575 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.575 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.575 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:30:45.575 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:30:45.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.575 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.575 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.575 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:30:45.575 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:30:45.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.576 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:30:45.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:30:45.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.576 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.576 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:30:45.576 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:30:45.576 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.576 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.576 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.576 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.576 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:30:45.577 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:45.577 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 220.18167733199999, 2023-12-06T18:30:45.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:30:45.577 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.577 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.577 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.577 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.577 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:45.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.577 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.577 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:45.577 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.578 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.578 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:30:45.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:30:45.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.578 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.578 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.578 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:30:45.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:30:45.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.578 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.578 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.578 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:30:45.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:30:45.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.578 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.578 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.579 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:30:45.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:30:45.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.579 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.579 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:30:45.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:30:45.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.579 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.579 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:30:45.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:30:45.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.579 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:30:45.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:30:45.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:30:45.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:30:45.580 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.580 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.580 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.580 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:30:45.580 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:45.581 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 220.17543952700001, 2023-12-06T18:30:45.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:30:45.581 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.581 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.581 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:45.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:45.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:30:45.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:30:45.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:30:45.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:30:45.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:30:45.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:30:45.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.583 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.583 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:30:45.583 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:30:45.583 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.583 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.583 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.583 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:30:45.583 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:30:45.583 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.583 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.583 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.583 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:30:45.583 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:30:45.583 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.583 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.584 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.584 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:30:45.584 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:30:45.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.584 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.584 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.584 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:30:45.584 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:30:45.584 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.584 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.584 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.584 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.584 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:30:45.584 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:45.585 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 220.16937644800001, 2023-12-06T18:30:45.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:30:45.585 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.585 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.585 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:45.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:45.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.586 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.586 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.586 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:30:45.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:30:45.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.586 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.586 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.586 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:30:45.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:30:45.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.586 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.586 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.586 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:30:45.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:30:45.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.587 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.587 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.587 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:30:45.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:30:45.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.587 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.587 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.587 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:30:45.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:30:45.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.587 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.587 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.587 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:30:45.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:30:45.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.588 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.588 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.588 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:30:45.588 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:30:45.588 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.588 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.588 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.588 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:30:45.588 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:30:45.588 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.588 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.588 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.588 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.588 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.588 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:30:45.588 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:45.588 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 220.03198936300001, 2023-12-06T18:30:45.588 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:30:45.589 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.589 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.589 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.589 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.589 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.589 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:45.589 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.589 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.589 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.589 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:45.589 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.589 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.589 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.589 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:30:45.589 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:30:45.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.590 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.590 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.590 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:30:45.590 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:30:45.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.590 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.590 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.590 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:30:45.590 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:30:45.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.590 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.591 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.591 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:30:45.591 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:30:45.591 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.591 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.591 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.591 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:30:45.591 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:30:45.591 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.591 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.591 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.591 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:30:45.591 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:30:45.591 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.591 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.591 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.591 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:30:45.592 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:30:45.592 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.592 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.592 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.592 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:30:45.592 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:30:45.592 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.592 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.592 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.592 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.592 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.592 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:30:45.592 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:45.592 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 220.02531863600001, 2023-12-06T18:30:45.592 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:30:45.592 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:45.592 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:45.593 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:45.593 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.593 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:45.593 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:45.593 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.593 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.593 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.593 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:45.593 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:45.593 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:45.593 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.593 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.593 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:45.593 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:30:45.593 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:30:45.593 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.593 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.594 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:45.594 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:30:45.594 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:30:45.594 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.594 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.594 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:45.594 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:30:45.594 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:30:45.594 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.594 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.594 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:45.594 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:30:45.594 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:30:45.594 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.594 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.595 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:45.595 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:30:45.595 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:30:45.595 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.595 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.595 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:45.595 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:30:45.595 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:30:45.595 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.595 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.595 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:45.595 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:30:45.595 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:30:45.595 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:45.595 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:45.595 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:45.595 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:30:45.595 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:30:45.596 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.596 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.596 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.596 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:45.596 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:45.596 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:45.597 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:30:45.598 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:45.594+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:45.698 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:45.695+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:45.766 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:45.766 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:45.766 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:45.766 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:45.776 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:30:45.820 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:45.816+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:45.900 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:45.897+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:45.906 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:45.906 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:45.906 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:45.906 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:45.906 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:45.915 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:30:46.006 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:46.001+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:46.086 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:46.086 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:46.087 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:46.087 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:30:46.087 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:46.097 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:30:46.102 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:46.099+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:46.203 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:46.199+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:46.226 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:46.226 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:46.226 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:46.226 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:46.235 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:30:46.304 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:46.301+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:46.364 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:46.364 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:46.364 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:46.364 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:46.364 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:46.373 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:30:46.418 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:46.414+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:46.503 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:46.504 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:46.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:46.504 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:46.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:46.504 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:30:46.504 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:46.504 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 24.841960341, 2023-12-06T18:30:46.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:30:46.504 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:46.504 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:46.504 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:46.504 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:46.504 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:46.505 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:30:46.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:46.505 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:46.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:46.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:46.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:46.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:46.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:46.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:46.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:46.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:46.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:46.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:46.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:46.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:46.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:30:46.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:30:46.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:46.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:46.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:46.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:30:46.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:30:46.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:46.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:46.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:46.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:30:46.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:30:46.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:46.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:46.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:46.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:30:46.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:30:46.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:46.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:46.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:46.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:30:46.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:30:46.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:46.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:46.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:46.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:30:46.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:30:46.507 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:46.508 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:46.508 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:46.508 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:46.508 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:46.508 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:46.509 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:46.506+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:46.513 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:30:46.610 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:46.607+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:46.650 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:46.650 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:46.650 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:46.650 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:46.658 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:30:46.710 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:46.707+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:46.795 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:46.795 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:46.795 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:46.795 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:46.795 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:46.804 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:30:46.811 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:46.808+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:46.912 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:46.908+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:46.941 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:46.941 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:46.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:46.941 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:30:46.941 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:46.951 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:30:47.035 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:47.031+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:47.114 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:47.110+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:47.128 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:47.128 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:47.128 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:47.128 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:47.137 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:30:47.215 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:47.212+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:47.313 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:47.313 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:47.313 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:47.314 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:47.314 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:47.315 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:47.311+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:47.323 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:30:47.416 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:47.412+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:47.531 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:47.531 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:47.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:47.531 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:47.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.531 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:30:47.531 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:47.531 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 232.11370097700001, 2023-12-06T18:30:47.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:30:47.531 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.531 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.531 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.531 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.532 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.532 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:30:47.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.532 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:47.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:47.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:30:47.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:30:47.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:30:47.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:30:47.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:30:47.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:30:47.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:30:47.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:30:47.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:30:47.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:30:47.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:30:47.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:30:47.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:30:47.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:30:47.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:30:47.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:30:47.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:30:47.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:30:47.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:30:47.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:30:47.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:30:47.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:30:47.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:30:47.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:30:47.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:30:47.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:30:47.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:30:47.537 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.537 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.537 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.537 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:47.537 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:47.537 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 230.005127738, 2023-12-06T18:30:47.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:30:47.537 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.537 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.537 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.537 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.537 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.537 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:30:47.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.538 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.538 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:47.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.538 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:47.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:30:47.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:30:47.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:30:47.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:30:47.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:30:47.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:30:47.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:30:47.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:30:47.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:30:47.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:30:47.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:30:47.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:30:47.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:30:47.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:30:47.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:30:47.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:30:47.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:30:47.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:30:47.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:30:47.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:30:47.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:30:47.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:30:47.542 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.542 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.542 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.542 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:47.542 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:47.542 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 230.00438250600001, 2023-12-06T18:30:47.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:30:47.543 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.543 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.543 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.543 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.543 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.543 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:30:47.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.543 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:47.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:47.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:30:47.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:30:47.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:30:47.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:30:47.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:30:47.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:30:47.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:30:47.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:30:47.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:30:47.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:30:47.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:30:47.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:30:47.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:30:47.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:30:47.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:30:47.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:30:47.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:30:47.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:30:47.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:30:47.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:30:47.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:30:47.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:30:47.547 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.548 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.548 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.548 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:47.548 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:47.548 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 230.003698751, 2023-12-06T18:30:47.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:30:47.548 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.548 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.548 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.548 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.548 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.548 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:30:47.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.549 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:47.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:47.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:30:47.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:30:47.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:30:47.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:30:47.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:30:47.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:30:47.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:30:47.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:30:47.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:30:47.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:30:47.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:30:47.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:30:47.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:30:47.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:30:47.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:30:47.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:30:47.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:30:47.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:30:47.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:30:47.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:30:47.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:30:47.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:30:47.553 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.553 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.553 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.553 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:47.553 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:47.553 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 229.998125496, 2023-12-06T18:30:47.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:30:47.553 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.553 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.554 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.554 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.554 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.554 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:30:47.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.554 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:47.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:47.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:30:47.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:30:47.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:30:47.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:30:47.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.556 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:30:47.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:30:47.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.556 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.556 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:30:47.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:30:47.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.556 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.556 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:30:47.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:30:47.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:30:47.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:30:47.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:30:47.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:30:47.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:30:47.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:30:47.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:30:47.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:30:47.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:30:47.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:30:47.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:30:47.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:30:47.559 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.559 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.559 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.559 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:47.559 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:47.559 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 229.997451856, 2023-12-06T18:30:47.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:30:47.559 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.559 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.559 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.559 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.559 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.560 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:30:47.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.560 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:47.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:47.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:30:47.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:30:47.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:30:47.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:30:47.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:30:47.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:30:47.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:30:47.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:30:47.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:30:47.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:30:47.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:30:47.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:30:47.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:30:47.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:30:47.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:30:47.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:30:47.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:30:47.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:30:47.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:30:47.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:30:47.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:30:47.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:30:47.564 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.564 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.564 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.564 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:47.564 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:47.564 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 229.99678965699999, 2023-12-06T18:30:47.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:30:47.565 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.565 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.565 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.565 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.565 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.565 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:30:47.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.565 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:47.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:47.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:30:47.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:30:47.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:30:47.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:30:47.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.567 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:30:47.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:30:47.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.567 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.567 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:30:47.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:30:47.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.567 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.567 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:30:47.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:30:47.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.567 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.568 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:30:47.568 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:30:47.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.568 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.568 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:30:47.568 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:30:47.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.568 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.568 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:30:47.568 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:30:47.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.568 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:30:47.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:30:47.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:30:47.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:30:47.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:30:47.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:30:47.569 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.569 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.569 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.570 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:47.570 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:47.570 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 229.92408022199999, 2023-12-06T18:30:47.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:30:47.570 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.570 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.570 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.570 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.570 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.570 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:30:47.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.570 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:47.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:47.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:30:47.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:30:47.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.572 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:30:47.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:30:47.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.572 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:30:47.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:30:47.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.572 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:30:47.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:30:47.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.573 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.573 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:30:47.573 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:30:47.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.573 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.573 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:30:47.573 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:30:47.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.573 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.573 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:30:47.573 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:30:47.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:30:47.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:30:47.574 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:30:47.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:30:47.574 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:30:47.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:30:47.574 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.575 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.575 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:30:47.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:30:47.575 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.575 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.575 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.575 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:47.575 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:47.575 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 229.923398513, 2023-12-06T18:30:47.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:30:47.575 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.575 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.575 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.576 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.576 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.576 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:30:47.576 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.576 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.576 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.576 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.576 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:47.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.576 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.576 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.576 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.576 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:47.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.576 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.577 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.577 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:30:47.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:30:47.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.577 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.577 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:30:47.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:30:47.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.577 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.577 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:30:47.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:30:47.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:30:47.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:30:47.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:30:47.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:30:47.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:30:47.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:30:47.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.579 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.579 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.579 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:30:47.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:30:47.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.579 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.579 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.579 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:30:47.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:30:47.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.579 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.579 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.579 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:30:47.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:30:47.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.580 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.580 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:30:47.580 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:30:47.580 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.580 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.580 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:30:47.580 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:30:47.580 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.580 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.580 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.580 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.580 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:47.581 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:47.581 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 228.844594425, 2023-12-06T18:30:47.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:30:47.581 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.581 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.581 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.581 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.581 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.581 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:30:47.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.581 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.581 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.581 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:47.582 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.582 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.582 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:47.582 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.582 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.582 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:30:47.582 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:30:47.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.582 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.583 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:30:47.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:30:47.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.583 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:30:47.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:30:47.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.583 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:30:47.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:30:47.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.584 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:30:47.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:30:47.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.584 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:30:47.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:30:47.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.584 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:30:47.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:30:47.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.585 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.585 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:30:47.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:30:47.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.585 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.585 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:30:47.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:30:47.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.585 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.585 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:30:47.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:30:47.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:30:47.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:30:47.586 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.586 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.586 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.586 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:47.586 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:47.586 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 228.84317785100001, 2023-12-06T18:30:47.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:30:47.586 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.587 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.587 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.587 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.587 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.587 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:30:47.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.587 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:47.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.588 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.588 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:47.588 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.588 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.588 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.588 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.588 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:30:47.588 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:30:47.588 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.588 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.588 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.588 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:30:47.588 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:30:47.588 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.588 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.589 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:30:47.589 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:30:47.589 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.589 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.589 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:30:47.589 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:30:47.589 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.589 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.589 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:30:47.589 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:30:47.589 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.589 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.590 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:30:47.590 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:30:47.590 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.590 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.590 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.590 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:30:47.590 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:30:47.590 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.590 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.590 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.590 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:30:47.590 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:30:47.590 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.590 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.591 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.591 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:30:47.591 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:30:47.591 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.591 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.591 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.591 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:30:47.591 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:30:47.591 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.591 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.591 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.591 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:30:47.591 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:30:47.591 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.591 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.591 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.592 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.592 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.592 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:47.592 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:47.592 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 228.842092726, 2023-12-06T18:30:47.592 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:30:47.592 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.592 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.592 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.592 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.592 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.592 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:30:47.592 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.592 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.592 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.592 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:47.593 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.593 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.593 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.593 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:47.593 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.593 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.593 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.593 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:30:47.593 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:30:47.593 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.593 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.593 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:30:47.594 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:30:47.594 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.594 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.594 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.594 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:30:47.594 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:30:47.594 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.594 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.594 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.594 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:30:47.594 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:30:47.594 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.594 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.594 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.595 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:30:47.595 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:30:47.595 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.595 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.595 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.595 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:30:47.595 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:30:47.595 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.595 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.595 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.595 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:30:47.595 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:30:47.595 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.595 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.595 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.595 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:30:47.596 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:30:47.596 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.596 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.596 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.596 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:30:47.596 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:30:47.596 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.596 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.596 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.596 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:30:47.596 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:30:47.596 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.596 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.596 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.596 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:30:47.597 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:30:47.597 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.597 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.597 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.597 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.597 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.597 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:47.597 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:47.597 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 228.80694362400001, 2023-12-06T18:30:47.597 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:30:47.597 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.597 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.597 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.597 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.597 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.597 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:30:47.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.598 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.598 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:47.598 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.598 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:47.598 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:30:47.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:30:47.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.599 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:30:47.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:30:47.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.599 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:30:47.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:30:47.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.599 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:30:47.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:30:47.600 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.600 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.600 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.600 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:30:47.600 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:30:47.600 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.600 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.600 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.600 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:30:47.600 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:30:47.600 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.600 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.600 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.600 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:30:47.600 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:30:47.600 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.600 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.601 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.601 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:30:47.601 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:30:47.601 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.601 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.601 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.601 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:30:47.601 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:30:47.601 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.601 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.601 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.601 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:30:47.601 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:30:47.601 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.601 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.601 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.601 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:30:47.602 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:30:47.602 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.602 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.602 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.602 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.602 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.602 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:47.602 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:47.602 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 222.25721309900001, 2023-12-06T18:30:47.602 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:30:47.602 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.602 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.602 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.602 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.603 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.603 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:30:47.603 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.603 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.603 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.603 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.603 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:47.603 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.603 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.603 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.603 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.603 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:47.603 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.603 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.603 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.603 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.604 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:30:47.604 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:30:47.604 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.604 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.604 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.604 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:30:47.604 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:30:47.604 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.604 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.604 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.604 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:30:47.604 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:30:47.604 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.604 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.604 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.604 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:30:47.605 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:30:47.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.605 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.605 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.605 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:30:47.605 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:30:47.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.605 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.605 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.605 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:30:47.605 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:30:47.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.605 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.605 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.605 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:30:47.606 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:30:47.606 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.606 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.606 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.606 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:30:47.606 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:30:47.606 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.606 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.606 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.606 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:30:47.606 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:30:47.606 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.606 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.606 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.607 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:30:47.607 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:30:47.607 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.607 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.607 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.607 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:30:47.607 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:30:47.607 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.607 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.607 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.607 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.607 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.607 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:47.607 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:47.608 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 222.256407491, 2023-12-06T18:30:47.608 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:30:47.608 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.608 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.608 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.608 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.608 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.608 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:30:47.608 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.608 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.608 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.608 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.608 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:47.608 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.608 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.609 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.609 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.609 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:47.609 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.609 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.609 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.609 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:30:47.609 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:30:47.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.609 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.609 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.609 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:30:47.609 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:30:47.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.609 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.610 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.610 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:30:47.610 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:30:47.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.610 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.610 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.610 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:30:47.610 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:30:47.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.610 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.610 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.610 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:30:47.610 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:30:47.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.611 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.611 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.611 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:30:47.611 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:30:47.611 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.611 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.611 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.611 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:30:47.611 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:30:47.611 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.611 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.611 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.611 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:30:47.611 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:30:47.611 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.612 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.612 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.612 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:30:47.612 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:30:47.612 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.612 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.612 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.612 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:30:47.612 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:30:47.612 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.612 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.612 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.612 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:30:47.612 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:30:47.612 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.612 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.613 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.613 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.613 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:47.613 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:47.613 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 222.25571668200001, 2023-12-06T18:30:47.613 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:30:47.613 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.613 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.613 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.613 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.613 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.613 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:30:47.613 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.613 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.614 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:47.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.614 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:47.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.614 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.614 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.614 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:30:47.614 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:30:47.614 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.615 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:30:47.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:30:47.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.615 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:30:47.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:30:47.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.615 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.615 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:30:47.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:30:47.615 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:30:47.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:30:47.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:30:47.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:30:47.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:30:47.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:30:47.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:30:47.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:30:47.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:30:47.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:30:47.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:30:47.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:30:47.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:30:47.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:30:47.618 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.618 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.618 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.618 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:47.618 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:47.618 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 222.25141917400001, 2023-12-06T18:30:47.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:30:47.618 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.618 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.619 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.619 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.619 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.619 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:30:47.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.619 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.619 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.619 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:47.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.619 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.619 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.619 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:47.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.620 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.620 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.620 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.620 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:30:47.620 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:30:47.620 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.620 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.620 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.620 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:30:47.620 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:30:47.620 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.620 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.620 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.620 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:30:47.621 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:30:47.621 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.621 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.621 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.621 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:30:47.621 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:30:47.621 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.621 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.621 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.621 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:30:47.621 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:30:47.621 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.621 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.621 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.622 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:30:47.622 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:30:47.622 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.622 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.622 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.622 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:30:47.622 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:30:47.622 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.622 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.622 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.622 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:30:47.622 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:30:47.622 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.622 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.623 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.623 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:30:47.623 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:30:47.623 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.623 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.623 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.623 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:30:47.623 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:30:47.623 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.623 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.623 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.623 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:30:47.623 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:30:47.623 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.623 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.623 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.624 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.624 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.624 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:47.624 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:47.624 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 222.25075149899999, 2023-12-06T18:30:47.624 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:30:47.624 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.624 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.624 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.624 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.624 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.624 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:30:47.624 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.624 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.624 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.624 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:47.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:47.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.625 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:30:47.625 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:30:47.625 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.625 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.625 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.626 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:30:47.626 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:30:47.626 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.626 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.626 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.626 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:30:47.626 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:30:47.626 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.626 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.626 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.626 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:30:47.626 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:30:47.626 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.626 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.626 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.627 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:30:47.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:30:47.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.627 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:30:47.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:30:47.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.627 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:30:47.627 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:30:47.627 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.627 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.627 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:30:47.628 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:30:47.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.628 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.628 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:30:47.628 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:30:47.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.628 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.628 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:30:47.628 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:30:47.628 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.628 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.628 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.628 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:30:47.629 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:30:47.629 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.629 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.629 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.629 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.629 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.629 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:47.629 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:47.629 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 222.250096071, 2023-12-06T18:30:47.629 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:30:47.629 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.629 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.629 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.629 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.629 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.630 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:30:47.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.630 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.630 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.630 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.630 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:47.630 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.630 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.630 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.630 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:47.630 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.630 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.630 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.631 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.631 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:30:47.631 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:30:47.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.631 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.631 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.631 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:30:47.631 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:30:47.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.631 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.631 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.631 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:30:47.631 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:30:47.631 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.631 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.631 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.632 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:30:47.632 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:30:47.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.632 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.632 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:30:47.632 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:30:47.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.632 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.632 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:30:47.632 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:30:47.632 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.632 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.632 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.633 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:30:47.633 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:30:47.633 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.633 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.633 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.633 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:30:47.633 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:30:47.633 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.633 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.633 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.633 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:30:47.633 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:30:47.633 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.633 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.634 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.634 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:30:47.634 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:30:47.634 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.634 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.634 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.634 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:30:47.634 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:30:47.634 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.634 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.634 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.634 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.634 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.634 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:47.634 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:47.635 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 222.209441524, 2023-12-06T18:30:47.635 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:30:47.635 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:47.635 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:47.635 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:47.635 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:47.635 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:47.635 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:30:47.635 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.635 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:47.635 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.635 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:47.635 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:47.635 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.635 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:47.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:47.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:47.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:47.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:30:47.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:30:47.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.636 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.636 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:47.636 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:30:47.636 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:30:47.636 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:47.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:30:47.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:30:47.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:47.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:30:47.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:30:47.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.637 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:47.637 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:30:47.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:30:47.637 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:47.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:30:47.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:30:47.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:47.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:30:47.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:30:47.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.638 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.638 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:47.638 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:30:47.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:30:47.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:47.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:30:47.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:30:47.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:47.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:30:47.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:30:47.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:47.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:47.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:47.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:30:47.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:30:47.639 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.640 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.640 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.640 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:47.640 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:47.640 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:47.640 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:47.527+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:47.641 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:30:47.643 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:47.639+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:47.744 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:47.740+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:47.845 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:47.841+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:47.850 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:47.850 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:47.850 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:47.850 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:47.861 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:30:47.945 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:47.941+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:48.081 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:48.077+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:48.083 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:48.083 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:48.083 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:48.083 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:48.083 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:48.093 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:30:48.159 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:05.344 106652== 2023-12-06T18:30:48.159 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:05.344 106652== Exit program on first error (--exit-on-first-error=yes) 2023-12-06T18:30:48.159 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:48.156+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:48.171 INFO:tasks.ceph.mon.c.smithi061.stderr:daemon-helper: command failed with exit status 42 2023-12-06T18:30:48.248 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:48.244+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:48.277 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:48.277 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:48.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:48.277 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:48.277 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.277 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:30:48.277 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:48.277 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 206.46120933700001, 2023-12-06T18:30:48.278 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:30:48.278 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:48.278 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:30:48.278 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:48.278 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.278 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:48.278 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:48.278 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:48.278 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.278 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.278 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:48.278 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:48.278 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:48.278 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.279 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:48.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:30:48.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:30:48.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.279 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:48.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:30:48.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:30:48.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.279 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:48.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:30:48.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:30:48.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.279 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:48.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:30:48.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:30:48.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:48.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:30:48.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:30:48.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:48.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:30:48.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:30:48.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:48.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:30:48.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:30:48.281 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:48.281 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:48.281 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:48.281 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:48.281 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:48.281 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:48.287 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:30:48.349 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:48.345+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:48.450 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:48.446+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:48.466 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:48.466 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:48.466 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:48.466 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:48.476 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:30:48.551 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:48.548+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:48.613 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:48.613 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:48.614 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:48.614 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:48.614 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:48.623 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:30:48.667 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:48.663+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:48.753 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:48.749+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:48.764 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 87.664220929999999, 2023-12-06T18:30:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:30:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:48.764 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:30:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:30:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:30:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:30:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:30:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:30:48.766 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:30:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:30:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:30:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:30:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:30:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:30:48.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:30:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:30:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 27.645963628000001, 2023-12-06T18:30:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:30:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:48.768 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:30:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:48.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:30:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:30:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:30:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:30:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:30:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:30:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:30:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:30:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:30:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:30:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:30:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:30:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:48.771 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:48.772 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:48.772 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:48.773 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:30:48.854 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:48.850+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:48.910 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:48.910 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:48.910 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:48.910 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:48.920 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:30:48.954 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:48.951+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:49.055 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:49.052+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:49.094 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:49.095 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:49.095 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:49.095 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:49.095 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:49.106 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:30:49.156 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:49.153+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:49.250 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:49.251 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:49.251 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:49.251 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:49.251 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.251 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:49.251 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:49.251 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 147.75724095300001, 2023-12-06T18:30:49.251 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:30:49.252 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.252 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:49.252 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:49.252 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:49.252 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:49.252 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:30:49.252 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.252 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.253 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.253 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.253 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:49.253 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.253 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.253 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.253 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.253 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:49.253 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:30:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:30:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.254 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:30:49.255 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:30:49.255 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.255 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.255 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.255 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:30:49.255 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:30:49.255 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.255 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.255 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.256 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:30:49.256 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:30:49.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.256 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.256 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.256 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:30:49.256 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:30:49.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.256 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.257 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.257 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:30:49.257 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:30:49.257 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.257 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.257 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.257 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.257 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.257 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:49.259 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:30:49.260 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:49.255+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:49.361 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:49.357+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:49.388 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:49.388 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:49.388 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:49.388 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:49.398 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:30:49.461 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:49.458+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:49.525 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:49.525 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:49.525 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:49.525 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:49.525 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:49.534 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:30:49.579 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:49.575+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:49.702 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:49.702 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:49.702 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:49.702 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:49.702 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.702 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:30:49.702 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:49.702 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 233.08650225900001, 2023-12-06T18:30:49.702 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:30:49.702 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.702 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:30:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:30:49.703 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:30:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:30:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:30:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:30:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:30:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:30:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:30:49.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:30:49.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:30:49.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:30:49.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:30:49.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:30:49.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.706 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.706 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:30:49.706 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:30:49.706 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.706 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.706 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.706 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.706 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.706 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:30:49.706 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:49.706 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 232.06367524999999, 2023-12-06T18:30:49.706 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:30:49.706 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.706 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.706 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.707 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.707 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:49.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.707 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.707 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:49.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.707 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.707 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:30:49.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:30:49.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:30:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:30:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:30:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:30:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:30:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:30:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:30:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:30:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:30:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:30:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:30:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:30:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.710 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:30:49.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:30:49.710 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.710 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.710 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.710 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:30:49.710 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:49.710 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 232.051628612, 2023-12-06T18:30:49.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:30:49.710 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.710 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.710 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.711 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:49.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.711 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:49.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:30:49.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:30:49.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:30:49.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:30:49.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:30:49.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:30:49.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:30:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:30:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:30:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:30:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:30:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:30:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.714 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.714 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:30:49.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:30:49.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.714 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.714 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:30:49.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:30:49.714 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.714 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.714 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:30:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 232.030228005, 2023-12-06T18:30:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:30:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:49.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:30:49.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:30:49.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:30:49.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:30:49.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:30:49.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:30:49.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:30:49.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:30:49.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:30:49.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:30:49.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.718 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:30:49.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:30:49.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.718 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.718 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:30:49.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:30:49.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.718 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.718 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:30:49.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:30:49.718 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.718 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.719 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.719 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:30:49.719 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:49.719 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 232.029048682, 2023-12-06T18:30:49.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:30:49.719 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.719 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.719 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:49.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:49.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:30:49.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:30:49.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:30:49.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:30:49.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:30:49.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:30:49.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:30:49.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:30:49.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:30:49.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:30:49.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:30:49.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:30:49.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:30:49.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:30:49.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:30:49.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:30:49.722 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.722 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.722 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.723 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:30:49.723 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:49.723 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 232.02847670099999, 2023-12-06T18:30:49.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:30:49.723 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.723 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.723 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.723 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:49.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.723 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:49.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:30:49.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:30:49.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:30:49.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:30:49.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:30:49.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:30:49.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:30:49.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:30:49.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:30:49.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:30:49.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:30:49.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:30:49.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:30:49.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:30:49.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:30:49.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:30:49.726 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.726 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.726 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.727 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:30:49.727 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:49.727 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 231.582641231, 2023-12-06T18:30:49.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:30:49.727 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.727 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.727 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.727 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.727 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:49.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.727 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:49.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:30:49.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:30:49.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:30:49.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:30:49.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:30:49.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:30:49.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:30:49.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:30:49.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:30:49.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:30:49.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:30:49.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:30:49.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:30:49.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:30:49.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:30:49.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:30:49.730 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.730 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.731 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.731 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:30:49.731 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:49.731 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 231.58104137500001, 2023-12-06T18:30:49.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:30:49.731 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.731 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.731 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:49.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:49.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:30:49.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:30:49.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:30:49.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:30:49.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:30:49.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:30:49.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:30:49.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:30:49.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:30:49.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:30:49.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:30:49.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:30:49.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:30:49.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:30:49.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:30:49.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:30:49.734 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.735 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.735 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.735 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:30:49.735 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:49.735 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 231.20183575900001, 2023-12-06T18:30:49.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:30:49.735 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.735 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.735 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:49.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:49.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:30:49.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:30:49.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:30:49.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:30:49.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:30:49.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:30:49.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:30:49.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:30:49.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:30:49.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:30:49.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:30:49.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:30:49.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:30:49.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:30:49.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:30:49.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:30:49.739 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.739 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.739 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.739 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:30:49.739 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:49.739 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 230.96433357699999, 2023-12-06T18:30:49.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:30:49.739 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.739 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.739 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:49.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:49.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:30:49.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:30:49.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:30:49.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:30:49.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:30:49.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:30:49.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:30:49.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:30:49.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:30:49.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:30:49.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:30:49.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:30:49.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:30:49.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:30:49.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:30:49.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:30:49.743 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.743 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.743 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.743 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:30:49.743 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:49.743 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 230.963605578, 2023-12-06T18:30:49.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:30:49.743 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.743 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.743 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:49.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:49.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:30:49.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:30:49.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:30:49.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:30:49.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:30:49.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:30:49.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:30:49.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:30:49.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:30:49.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:30:49.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:30:49.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:30:49.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:30:49.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:30:49.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:30:49.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:30:49.747 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.747 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.747 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.747 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:30:49.747 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:49.747 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 230.95859316799999, 2023-12-06T18:30:49.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:30:49.748 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.748 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.748 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:49.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:49.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:30:49.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:30:49.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:30:49.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:30:49.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:30:49.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:30:49.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:30:49.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:30:49.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.750 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:30:49.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:30:49.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.750 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:30:49.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:30:49.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:30:49.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:30:49.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:30:49.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:30:49.751 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.751 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.751 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.751 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:30:49.751 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:49.751 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 230.953066089, 2023-12-06T18:30:49.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:30:49.752 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.752 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.752 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:49.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:49.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:30:49.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:30:49.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:30:49.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:30:49.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:30:49.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:30:49.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:30:49.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:30:49.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:30:49.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:30:49.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:30:49.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:30:49.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:30:49.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:30:49.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:30:49.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:30:49.755 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.755 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.755 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.756 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:30:49.756 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:49.756 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 230.952555252, 2023-12-06T18:30:49.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:30:49.756 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.756 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.756 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:49.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:49.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:30:49.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:30:49.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:30:49.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:30:49.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:30:49.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:30:49.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:30:49.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:30:49.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:30:49.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:30:49.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:30:49.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:30:49.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:30:49.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:30:49.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:30:49.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:30:49.759 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.760 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.760 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.760 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:30:49.760 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:49.760 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 230.95209599099999, 2023-12-06T18:30:49.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:30:49.760 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.760 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.760 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:49.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:49.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:30:49.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:30:49.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:30:49.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:30:49.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:30:49.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:30:49.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:30:49.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:30:49.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:30:49.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:30:49.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:30:49.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:30:49.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:30:49.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:30:49.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:30:49.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:30:49.764 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.764 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.764 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.764 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:30:49.764 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:49.764 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 224.372180184, 2023-12-06T18:30:49.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:30:49.764 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.764 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.764 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:49.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:49.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:30:49.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:30:49.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:30:49.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:30:49.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:30:49.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:30:49.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:30:49.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:30:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:30:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:30:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:30:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:30:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:30:49.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:30:49.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.768 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.768 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:30:49.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:30:49.768 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.768 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.768 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.768 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:30:49.768 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:49.768 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 224.36594237899999, 2023-12-06T18:30:49.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:30:49.769 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.769 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.769 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:49.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:49.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:30:49.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:30:49.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:30:49.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:30:49.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:30:49.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:30:49.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:30:49.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:30:49.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:30:49.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:30:49.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:30:49.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:30:49.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:30:49.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:30:49.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:30:49.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:30:49.772 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.772 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.772 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.772 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:30:49.773 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:49.773 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 224.35987929999999, 2023-12-06T18:30:49.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:30:49.773 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.773 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.773 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:49.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:49.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:30:49.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:30:49.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:30:49.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:30:49.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:30:49.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:30:49.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:30:49.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:30:49.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:30:49.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:30:49.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:30:49.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:30:49.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:30:49.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:30:49.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:30:49.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:30:49.776 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.776 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.776 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.777 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:30:49.777 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:49.777 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 224.22249221499999, 2023-12-06T18:30:49.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:30:49.777 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.777 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.777 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:49.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:49.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:30:49.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:30:49.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:30:49.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:30:49.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:30:49.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:30:49.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:30:49.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:30:49.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:30:49.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:30:49.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:30:49.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:30:49.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:30:49.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:30:49.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:30:49.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:30:49.780 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.780 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.780 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.781 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:30:49.781 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:49.781 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 224.21582148799999, 2023-12-06T18:30:49.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:30:49.781 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:49.781 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:49.781 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:49.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:49.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:49.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:49.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:49.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:49.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:49.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:30:49.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:30:49.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:49.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:30:49.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:30:49.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:49.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:30:49.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:30:49.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:49.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:30:49.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:30:49.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:49.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:30:49.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:30:49.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:49.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:30:49.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:30:49.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:49.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:30:49.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:30:49.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:49.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:49.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:49.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:30:49.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:30:49.784 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.784 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.784 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.785 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:49.785 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:49.785 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:49.785 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:49.698+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:49.786 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:30:49.787 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-12-06T18:30:49.788 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:49.784+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:49.889 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:49.885+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:49.955 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:49.955 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:49.955 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:49.955 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:49.964 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:30:49.990 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:49.986+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:50.091 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:50.087+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:50.130 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:50.131 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:50.131 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:50.131 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:50.131 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:50.139 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:30:50.191 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:50.188+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:50.267 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:50.268 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:50.268 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:50.268 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:30:50.268 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:50.277 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:30:50.293 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:50.289+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:50.394 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:50.390+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:50.447 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:50.447 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:50.447 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:50.447 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:50.456 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:30:50.495 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:50.491+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:50.614 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:50.610+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:50.627 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:50.628 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:50.628 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:50.628 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:50.628 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:50.637 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:30:50.722 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:50.718+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:50.798 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:50.794+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:50.806 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:50.806 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:50.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:50.806 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:50.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:50.807 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:30:50.807 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:50.807 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 29.14480446, 2023-12-06T18:30:50.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:30:50.807 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:50.807 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:50.807 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:50.807 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:50.807 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:50.807 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:30:50.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:50.807 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:50.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:50.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:50.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:50.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:50.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:50.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:50.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:50.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:50.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:50.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:50.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:50.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:50.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:30:50.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:30:50.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:50.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:50.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:50.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:30:50.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:30:50.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:50.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:50.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:50.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:30:50.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:30:50.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:50.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:50.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:50.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:30:50.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:30:50.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:50.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:50.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:50.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:30:50.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:30:50.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:50.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:50.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:50.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:30:50.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:30:50.810 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:50.810 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:50.810 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:50.811 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:50.811 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:50.811 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:50.815 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:30:50.899 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:50.895+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:50.967 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:50.967 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:50.967 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:50.968 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:50.977 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:30:51.000 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:50.997+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:51.100 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:51.097+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:51.153 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:51.153 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:51.153 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:51.153 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:51.154 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:51.162 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:30:51.207 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:51.203+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:51.302 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:51.302 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:51.302 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:51.302 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:30:51.302 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:51.303 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:51.300+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:51.311 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:30:51.403 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:51.400+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:51.487 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:51.488 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:51.488 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:51.488 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:51.497 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:30:51.538 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:51.534+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:51.606 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:51.602+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:51.672 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:51.672 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:51.672 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:51.672 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:51.672 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:51.681 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:30:51.706 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:51.703+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:51.808 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:51.805+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:51.876 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:51.876 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:51.876 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:51.876 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:51.876 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.876 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:30:51.876 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:51.876 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 236.465507874, 2023-12-06T18:30:51.877 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:30:51.877 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.877 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.877 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.877 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.877 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.877 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:30:51.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.877 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.877 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.877 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.877 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:51.877 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.877 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.878 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.878 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:51.878 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.878 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.878 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.878 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.878 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:30:51.878 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:30:51.878 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.878 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.878 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.878 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:30:51.878 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:30:51.878 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.878 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.879 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.879 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:30:51.879 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:30:51.879 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.879 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.879 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.879 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:30:51.879 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:30:51.879 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.879 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.879 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.879 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:30:51.879 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:30:51.879 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.879 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.880 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:30:51.880 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:30:51.880 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:30:51.880 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.880 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.880 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.880 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:30:51.880 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:30:51.880 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.880 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.880 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.880 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:30:51.880 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:30:51.880 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.880 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.881 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.881 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:30:51.881 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:30:51.881 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.881 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.881 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.881 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:30:51.881 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:30:51.881 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.881 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.881 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.881 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:30:51.881 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:30:51.881 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.881 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.882 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.882 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:30:51.882 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:30:51.882 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.882 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.882 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.882 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:30:51.882 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:30:51.882 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.882 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.882 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.882 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.882 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.882 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:51.883 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:51.883 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 234.35693463499999, 2023-12-06T18:30:51.883 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:30:51.883 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.883 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.883 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.883 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.883 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.883 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:30:51.883 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.883 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.883 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.883 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.883 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:51.883 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.883 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.884 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.884 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.884 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:51.884 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.884 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.884 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.884 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.884 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:30:51.884 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:30:51.884 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.884 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.884 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.884 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:30:51.884 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:30:51.884 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.885 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.885 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.885 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:30:51.885 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:30:51.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.885 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.885 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.885 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:30:51.885 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:30:51.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.885 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.885 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.885 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:30:51.885 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:30:51.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.886 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.886 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.886 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:30:51.886 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:30:51.886 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.886 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.886 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.886 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:30:51.886 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:30:51.886 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.886 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.886 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.886 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:30:51.886 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:30:51.886 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.887 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.887 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.887 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:30:51.887 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:30:51.887 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.887 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.887 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.887 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:30:51.887 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:30:51.887 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.887 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.887 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.887 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:30:51.887 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:30:51.887 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.888 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.888 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.888 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.888 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.888 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:51.888 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:51.888 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 234.356189403, 2023-12-06T18:30:51.888 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:30:51.888 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.888 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.888 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.888 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.888 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.888 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:30:51.888 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.889 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.889 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.889 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.889 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:51.889 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.889 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.889 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.889 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.889 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:51.889 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.889 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.889 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.889 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.889 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:30:51.889 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:30:51.890 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.890 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.890 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.890 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:30:51.890 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:30:51.890 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.890 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.890 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.890 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:30:51.890 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:30:51.890 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.890 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.890 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.890 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:30:51.890 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:30:51.890 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.891 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.891 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:30:51.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:30:51.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.891 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.891 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:30:51.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:30:51.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.891 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.891 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:30:51.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:30:51.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.891 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.892 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.892 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:30:51.892 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:30:51.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.892 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.892 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:30:51.892 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:30:51.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.892 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.892 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:30:51.892 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:30:51.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.893 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.893 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:30:51.893 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:30:51.893 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.893 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.893 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.893 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.893 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.893 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:51.893 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:51.893 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 234.35550564799999, 2023-12-06T18:30:51.893 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:30:51.893 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.893 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.893 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.893 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.893 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.894 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:30:51.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.894 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.894 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:51.894 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.894 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:51.894 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.895 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:30:51.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:30:51.895 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.895 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.895 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.895 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:30:51.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:30:51.895 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.895 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.895 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.895 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:30:51.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:30:51.895 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.895 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.895 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.895 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:30:51.896 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:30:51.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.896 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.896 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.896 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:30:51.896 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:30:51.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.896 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.896 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.896 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:30:51.896 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:30:51.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.896 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.896 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.896 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:30:51.896 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:30:51.897 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.897 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.897 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.897 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:30:51.897 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:30:51.897 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.897 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.897 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.897 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:30:51.897 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:30:51.897 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.897 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.897 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.897 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:30:51.897 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:30:51.898 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.898 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.898 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.898 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:30:51.898 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:30:51.898 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.898 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.898 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.898 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.898 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.898 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:51.898 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:51.898 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 234.34993239299999, 2023-12-06T18:30:51.898 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:30:51.898 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.899 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.899 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.899 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.899 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.899 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:30:51.899 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.899 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.899 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.899 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.899 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:51.899 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.899 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.899 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.899 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.899 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:51.900 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.900 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.900 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.900 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.900 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:30:51.900 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:30:51.900 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.900 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.900 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.900 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:30:51.900 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:30:51.900 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.900 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.900 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.900 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:30:51.900 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:30:51.901 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.901 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.901 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.901 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:30:51.901 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:30:51.901 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.901 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.901 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.901 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:30:51.901 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:30:51.901 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.901 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.901 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.901 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:30:51.901 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:30:51.902 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.902 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.902 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.902 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:30:51.902 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:30:51.902 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.902 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.902 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.902 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:30:51.902 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:30:51.902 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.902 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.902 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.902 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:30:51.902 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:30:51.903 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.903 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.903 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.903 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:30:51.903 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:30:51.903 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.903 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.903 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.903 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:30:51.903 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:30:51.903 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.903 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.903 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.903 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.903 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.904 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:51.904 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:51.904 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 234.34925875299999, 2023-12-06T18:30:51.904 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:30:51.904 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.904 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.904 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.904 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.904 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.904 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:30:51.904 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.904 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.904 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.904 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.904 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:51.904 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.904 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.905 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.905 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.905 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:51.905 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.905 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.905 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.905 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.905 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:30:51.905 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:30:51.905 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.905 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.905 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.905 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:30:51.905 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:30:51.905 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.905 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.906 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.906 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:30:51.906 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:30:51.906 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.906 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.906 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.906 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:30:51.906 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:30:51.906 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.906 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.906 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.906 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:30:51.906 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:30:51.906 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.906 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.907 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.907 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:30:51.907 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:30:51.907 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.907 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.907 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.907 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:30:51.907 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:30:51.907 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.907 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.907 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.907 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:30:51.907 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:30:51.907 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.907 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.908 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.908 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:30:51.908 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:30:51.908 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.908 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.908 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.908 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:30:51.908 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:30:51.908 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.908 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.908 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.908 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:30:51.908 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:30:51.908 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.908 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.909 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.909 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.909 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.909 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:51.909 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:51.909 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 234.34859655400001, 2023-12-06T18:30:51.909 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:30:51.909 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.909 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.909 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.909 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.909 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.909 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:30:51.909 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.909 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.909 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.909 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.910 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:51.910 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.910 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.910 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.910 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.910 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:51.910 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.910 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.910 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.910 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.910 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:30:51.910 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:30:51.910 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.910 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.910 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.911 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:30:51.911 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:30:51.911 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.911 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.911 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.911 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:30:51.911 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:30:51.911 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.911 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.911 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.911 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:30:51.911 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:30:51.911 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.912 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.912 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.912 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:30:51.912 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:30:51.912 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.912 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.912 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.912 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:30:51.912 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:30:51.912 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.912 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.912 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.912 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:30:51.912 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:30:51.912 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.913 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.913 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.913 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:30:51.913 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:30:51.913 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.913 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.913 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.913 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:30:51.913 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:30:51.913 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.913 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.913 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.913 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:30:51.913 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:30:51.913 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.913 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.913 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.914 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:30:51.914 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:30:51.914 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.914 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.914 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.914 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.914 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.914 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:51.914 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:51.914 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 234.275887119, 2023-12-06T18:30:51.914 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:30:51.914 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.914 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.914 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.914 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.915 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.915 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:30:51.915 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.915 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.915 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.915 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.915 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:51.915 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.915 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.915 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.915 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.915 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:51.915 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.915 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.916 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.916 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.916 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:30:51.916 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:30:51.916 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.916 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.916 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.916 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:30:51.916 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:30:51.916 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.916 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.916 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.916 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:30:51.916 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:30:51.916 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.916 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.917 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.917 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:30:51.917 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:30:51.917 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.917 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.917 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.917 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:30:51.917 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:30:51.917 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.917 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.917 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.917 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:30:51.917 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:30:51.917 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.917 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.917 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.918 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:30:51.918 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:30:51.918 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.918 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.918 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.918 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:30:51.918 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:30:51.918 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.918 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.918 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.918 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:30:51.918 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:30:51.918 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.918 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.918 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.919 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:30:51.919 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:30:51.919 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.919 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.919 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.919 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:30:51.919 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:30:51.919 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.919 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.919 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.919 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.919 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.919 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:51.919 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:51.919 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 234.27520541000001, 2023-12-06T18:30:51.920 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:30:51.920 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.920 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.920 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.920 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.920 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.920 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:30:51.920 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.920 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.920 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.920 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.920 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:51.920 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.920 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.921 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.921 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.921 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:51.921 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.921 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.921 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.921 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.921 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:30:51.921 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:30:51.921 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.921 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.921 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.921 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:30:51.921 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:30:51.921 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.922 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.922 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.922 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:30:51.922 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:30:51.922 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.922 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.922 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.922 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:30:51.922 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:30:51.922 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.922 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.922 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.922 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:30:51.922 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:30:51.922 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.923 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.923 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.923 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:30:51.923 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:30:51.923 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.923 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.923 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.923 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:30:51.923 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:30:51.923 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.923 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.923 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.923 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:30:51.923 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:30:51.923 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.923 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.924 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.924 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:30:51.924 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:30:51.924 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.924 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.924 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.924 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:30:51.924 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:30:51.924 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.924 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.924 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.924 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:30:51.924 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:30:51.924 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.924 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.924 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.925 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.925 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.925 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:51.925 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:51.925 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 233.19640132200001, 2023-12-06T18:30:51.925 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:30:51.925 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.925 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.925 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.925 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.925 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.925 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:30:51.925 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.925 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.925 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.925 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.925 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:51.926 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.926 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.926 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.926 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.926 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:51.926 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.926 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.926 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.926 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.926 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:30:51.926 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:30:51.926 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.926 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.926 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.927 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:30:51.927 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:30:51.927 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.927 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.927 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.927 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:30:51.927 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:30:51.927 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.927 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.927 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.927 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:30:51.927 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:30:51.927 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.927 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.927 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.927 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:30:51.927 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:30:51.928 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.928 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.928 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.928 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:30:51.928 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:30:51.928 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.928 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.928 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.928 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:30:51.928 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:30:51.928 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.928 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.928 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.928 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:30:51.928 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:30:51.928 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.928 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.929 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.929 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:30:51.929 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:30:51.929 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.929 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.929 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.929 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:30:51.929 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:30:51.929 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.929 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.929 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.929 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:30:51.929 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:30:51.929 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.929 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.929 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.929 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.929 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.930 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:51.930 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:51.930 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 233.194984748, 2023-12-06T18:30:51.930 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:30:51.930 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.930 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.930 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.930 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.930 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.930 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:30:51.930 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.930 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.930 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.931 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.931 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:51.931 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.931 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.931 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.931 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.931 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:51.931 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.931 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.931 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.931 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.931 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:30:51.931 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:30:51.931 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.931 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.932 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:30:51.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:30:51.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.932 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:30:51.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:30:51.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.932 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:30:51.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:30:51.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.933 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.933 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:30:51.933 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:30:51.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.933 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.933 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:30:51.933 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:30:51.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.933 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.933 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:30:51.933 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:30:51.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.933 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.934 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:30:51.934 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:30:51.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.934 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.934 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.934 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:30:51.934 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:30:51.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.934 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.934 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.934 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:30:51.934 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:30:51.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.934 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.934 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.935 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:30:51.935 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:30:51.935 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.935 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.935 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.935 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.935 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:51.935 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:51.935 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 233.19389962299999, 2023-12-06T18:30:51.935 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:30:51.935 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.935 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.935 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.935 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.935 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.935 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:30:51.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.936 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.936 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.936 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:51.936 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.936 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.936 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:51.936 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.936 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.936 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:30:51.937 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:30:51.937 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.937 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.937 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.937 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:30:51.937 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:30:51.937 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.937 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.937 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.937 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:30:51.937 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:30:51.937 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.937 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.937 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.937 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:30:51.937 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:30:51.938 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.938 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.938 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.938 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:30:51.938 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:30:51.938 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.938 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.938 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.938 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:30:51.938 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:30:51.938 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.938 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.938 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.938 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:30:51.939 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:30:51.939 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.939 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.939 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.939 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:30:51.939 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:30:51.939 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.939 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.939 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.939 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:30:51.939 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:30:51.939 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.939 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.939 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.939 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:30:51.939 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:30:51.939 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.940 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.940 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.940 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:30:51.940 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:30:51.940 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.940 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.940 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.940 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.940 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.940 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:51.940 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:51.940 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 233.158750521, 2023-12-06T18:30:51.940 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:30:51.940 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.940 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.941 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.941 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.941 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.941 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:30:51.941 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.941 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.941 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.941 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:51.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.941 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.941 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.941 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:51.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:30:51.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:30:51.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:30:51.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:30:51.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:30:51.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:30:51.943 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.943 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:30:51.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:30:51.943 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.943 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:30:51.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:30:51.943 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.943 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:30:51.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:30:51.943 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.944 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.944 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.944 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:30:51.944 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:30:51.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.944 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.944 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.944 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:30:51.944 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:30:51.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.944 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.944 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.944 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:30:51.944 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:30:51.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.944 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.944 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.944 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:30:51.945 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:30:51.945 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.945 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.945 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.945 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:30:51.945 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:30:51.945 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.945 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.945 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.945 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.945 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.945 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:51.945 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:51.945 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 226.609019996, 2023-12-06T18:30:51.945 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:30:51.945 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.946 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.946 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.946 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.946 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.946 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:30:51.946 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.946 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.946 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.946 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.946 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:51.946 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.946 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.946 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.946 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.946 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:51.947 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.947 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.947 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.947 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.947 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:30:51.947 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:30:51.947 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.947 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.947 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.947 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:30:51.947 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:30:51.947 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.947 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.947 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.947 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:30:51.948 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:30:51.948 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.948 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.948 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.948 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:30:51.948 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:30:51.948 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.948 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.948 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.948 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:30:51.948 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:30:51.948 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.948 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.948 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.948 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:30:51.948 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:30:51.949 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.949 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.949 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.949 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:30:51.949 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:30:51.949 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.949 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.949 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.949 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:30:51.949 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:30:51.949 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.949 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.949 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.949 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:30:51.949 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:30:51.949 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.949 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.950 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.950 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:30:51.950 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:30:51.950 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.950 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.950 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.950 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:30:51.950 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:30:51.950 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.950 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.950 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.950 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.950 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.950 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:51.951 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:51.951 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 226.60821438799999, 2023-12-06T18:30:51.951 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:30:51.951 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.951 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.951 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.951 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.951 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.951 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:30:51.951 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.951 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.951 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.951 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.951 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:51.951 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.952 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.952 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.952 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.952 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:51.952 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.952 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.952 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.952 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.952 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:30:51.952 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:30:51.952 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.952 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.952 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.952 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:30:51.953 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:30:51.953 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.953 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.953 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.953 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:30:51.953 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:30:51.953 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.953 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.953 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.953 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:30:51.953 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:30:51.953 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.953 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.953 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.953 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:30:51.953 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:30:51.954 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.954 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.954 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.954 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:30:51.954 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:30:51.954 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.954 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.954 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.954 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:30:51.954 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:30:51.954 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.954 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.954 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.954 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:30:51.954 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:30:51.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.955 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.955 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.955 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:30:51.955 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:30:51.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.955 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.955 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.955 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:30:51.955 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:30:51.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.955 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.955 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.955 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:30:51.955 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:30:51.955 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.956 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.956 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.956 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.956 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.956 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:51.956 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:51.956 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 226.607523579, 2023-12-06T18:30:51.956 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:30:51.956 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.956 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.956 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.956 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.956 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.956 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:30:51.956 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.956 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.957 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.957 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.957 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:51.957 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.957 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.957 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.957 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.957 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:51.957 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.957 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.957 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.957 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.957 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:30:51.957 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:30:51.958 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.958 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.958 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.958 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:30:51.958 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:30:51.958 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.958 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.958 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.958 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:30:51.958 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:30:51.958 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.958 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.958 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.958 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:30:51.958 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:30:51.959 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.959 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.959 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.959 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:30:51.959 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:30:51.959 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.959 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.959 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.959 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:30:51.959 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:30:51.959 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.959 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.959 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.959 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:30:51.959 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:30:51.960 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.960 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.960 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.960 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:30:51.960 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:30:51.960 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.960 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.960 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.960 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:30:51.960 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:30:51.960 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.960 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.960 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.960 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:30:51.960 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:30:51.961 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.961 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.961 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.961 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:30:51.961 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:30:51.961 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.961 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.961 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.961 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.961 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.961 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:51.961 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:51.961 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 226.60322607099999, 2023-12-06T18:30:51.961 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:30:51.961 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.962 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.962 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.962 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.962 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.962 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:30:51.962 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.962 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.962 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.962 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.962 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:51.962 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.962 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.962 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.962 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.963 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:51.963 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.963 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.963 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.963 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.963 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:30:51.963 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:30:51.963 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.963 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.963 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.963 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:30:51.963 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:30:51.963 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.963 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.963 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.964 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:30:51.964 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:30:51.964 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.964 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.964 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.964 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:30:51.964 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:30:51.964 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.965 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.965 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.965 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:30:51.965 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:30:51.965 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.965 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.965 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.965 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:30:51.965 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:30:51.965 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.965 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.966 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.966 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:30:51.966 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:30:51.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.966 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.966 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.966 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:30:51.966 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:30:51.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.966 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.966 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.966 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:30:51.966 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:30:51.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.967 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.967 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.967 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:30:51.967 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:30:51.967 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.967 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.967 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.967 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:30:51.967 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:30:51.967 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.967 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.967 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.967 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.967 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.967 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:51.967 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:51.968 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 226.60255839600001, 2023-12-06T18:30:51.968 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:30:51.968 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.968 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.968 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.968 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.968 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:30:51.968 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.968 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.968 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.968 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.968 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:51.968 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.968 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.968 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.969 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.969 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:51.969 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.969 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.969 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.969 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:30:51.969 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:30:51.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.969 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.969 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.969 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:30:51.969 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:30:51.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.969 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.969 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.970 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:30:51.970 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:30:51.970 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.970 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.970 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.970 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:30:51.970 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:30:51.970 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.970 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.970 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.970 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:30:51.970 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:30:51.970 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.970 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.970 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.971 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:30:51.971 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:30:51.971 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.971 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.971 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.971 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:30:51.971 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:30:51.971 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.971 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.971 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.971 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:30:51.971 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:30:51.971 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.971 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.971 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:30:51.972 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:30:51.972 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.972 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.972 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:30:51.972 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:30:51.972 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.972 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.972 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:30:51.972 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:30:51.972 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.972 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.972 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.973 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.973 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.973 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:51.973 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:51.973 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 226.60190296799999, 2023-12-06T18:30:51.973 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:30:51.973 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.973 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.973 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.973 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.973 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.973 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:30:51.973 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.973 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.973 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.973 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.973 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:51.974 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.974 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.974 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.974 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.974 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:51.974 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.974 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.974 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.974 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.974 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:30:51.974 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:30:51.974 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.974 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.974 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.974 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:30:51.975 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:30:51.975 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.975 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.975 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.975 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:30:51.975 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:30:51.975 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.975 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.975 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.975 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:30:51.975 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:30:51.975 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.975 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.975 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.975 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:30:51.976 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:30:51.976 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.976 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.976 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.976 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:30:51.976 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:30:51.976 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.976 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.976 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.976 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:30:51.976 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:30:51.976 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.976 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.976 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.977 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:30:51.977 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:30:51.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.977 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.977 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.977 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:30:51.977 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:30:51.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.977 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.977 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.977 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:30:51.977 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:30:51.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.977 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.978 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.978 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:30:51.978 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:30:51.978 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.978 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.978 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.978 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.978 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.978 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:51.978 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:51.978 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 226.56124842099999, 2023-12-06T18:30:51.978 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:30:51.978 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:51.978 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:51.978 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:51.978 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:51.979 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:51.979 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:30:51.979 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.979 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:51.979 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.979 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:51.979 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:51.979 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.979 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.979 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.979 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:51.979 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:51.979 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:51.979 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.979 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.979 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:51.979 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:30:51.980 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:30:51.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.980 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:51.980 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:30:51.980 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:30:51.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.980 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:51.980 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:30:51.980 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:30:51.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.980 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:51.980 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:30:51.980 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:30:51.981 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.981 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.981 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:51.981 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:30:51.981 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:30:51.981 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.981 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.981 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:51.981 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:30:51.981 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:30:51.981 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.981 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.981 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:51.981 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:30:51.981 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:30:51.981 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.982 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.982 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:51.982 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:30:51.982 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:30:51.982 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.982 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.982 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:51.982 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:30:51.982 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:30:51.982 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.982 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.982 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:51.982 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:30:51.982 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:30:51.982 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:51.982 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:51.983 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:51.983 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:30:51.983 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:30:51.983 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.983 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.983 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.983 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:51.983 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:51.983 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:51.984 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:30:51.987 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:51.983+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:52.085 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:52.082+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:52.124 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:52.124 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:52.124 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:52.124 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:52.134 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:30:52.186 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:52.183+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:52.287 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:52.283+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:52.312 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:52.312 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:52.312 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:52.312 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:52.312 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:52.325 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:30:52.388 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:52.384+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:52.455 INFO:tasks.ceph.mon.a.smithi061.stderr:2023-12-06T18:30:52.444+0000 d75a640 -1 mon.a@0(probing) e1 get_health_metrics reporting 6 slow ops, oldest is auth(proto 0 30 bytes epoch 0) 2023-12-06T18:30:52.489 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:52.486+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:52.520 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:52.521 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:52.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:52.521 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:52.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:52.521 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:30:52.521 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:52.521 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 210.70430751000001, 2023-12-06T18:30:52.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:30:52.521 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:52.521 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:30:52.521 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:52.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:52.522 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:52.522 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:52.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:52.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:52.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:52.522 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:52.522 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:52.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:52.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:52.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:52.522 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:52.522 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:30:52.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:30:52.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:52.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:52.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:52.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:30:52.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:30:52.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:52.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:52.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:52.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:30:52.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:30:52.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:52.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:52.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:52.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:30:52.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:30:52.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:52.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:52.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:52.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:30:52.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:30:52.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:52.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:52.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:52.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:30:52.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:30:52.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:52.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:52.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:52.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:30:52.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:30:52.525 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:52.525 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:52.525 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:52.525 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:52.525 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:52.525 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:52.530 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:30:52.590 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:52.587+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:52.673 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:52.674 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:52.674 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:52.674 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:52.683 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:30:52.725 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:52.721+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:52.810 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:52.807+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:52.876 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:52.877 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:52.877 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:52.877 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:52.877 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:52.886 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:30:52.931 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:52.928+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:52.994 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:52.991+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:53.041 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:53.042 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:53.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:53.042 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:53.042 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:53.042 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:53.042 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:53.042 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 91.942146820000005, 2023-12-06T18:30:53.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:30:53.042 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:53.042 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:53.042 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:53.042 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:53.042 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:53.042 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:30:53.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:53.043 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:53.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:53.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:53.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:53.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:53.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:53.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:53.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:53.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:53.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:53.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:53.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:53.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:53.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:30:53.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:30:53.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:53.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:53.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:53.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:30:53.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:30:53.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:53.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:53.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:53.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:30:53.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:30:53.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:53.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:53.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:53.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:30:53.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:30:53.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:53.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:53.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:53.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:30:53.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:30:53.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:53.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:53.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:53.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:30:53.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:30:53.045 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:53.045 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:53.045 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:53.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:53.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:53.045 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:30:53.046 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:53.046 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 31.923889517999999, 2023-12-06T18:30:53.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:30:53.046 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:53.046 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:53.046 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:53.046 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:53.046 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:53.046 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:30:53.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:53.046 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:53.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:53.046 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:53.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:53.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:53.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:53.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:53.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:53.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:53.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:53.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:53.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:53.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:53.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:30:53.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:30:53.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:53.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:53.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:53.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:30:53.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:30:53.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:53.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:53.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:53.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:30:53.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:30:53.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:53.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:53.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:53.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:30:53.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:30:53.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:53.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:53.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:53.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:30:53.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:30:53.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:53.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:53.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:53.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:30:53.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:30:53.049 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:53.049 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:53.049 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:53.049 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:53.049 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:53.049 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:53.052 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:30:53.097 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:53.093+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:53.180 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:53.180 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:53.180 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:53.181 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:53.189 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:30:53.231 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:53.227+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:53.298 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:53.294+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:53.358 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:53.358 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:53.359 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:53.359 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:53.359 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:53.367 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:30:53.398 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:53.395+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:53.499 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:53.496+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:53.537 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:53.537 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:53.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:53.537 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:53.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:53.538 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:53.538 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:53.538 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 152.04410672, 2023-12-06T18:30:53.538 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:30:53.538 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:53.538 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:53.538 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:53.538 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:53.538 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:53.538 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:30:53.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:53.538 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:53.538 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:53.538 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:53.538 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:53.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:53.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:53.539 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:53.539 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:53.539 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:53.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:53.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:53.539 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:53.539 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:53.539 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:30:53.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:30:53.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:53.539 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:53.539 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:53.539 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:30:53.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:30:53.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:53.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:53.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:53.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:30:53.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:30:53.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:53.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:53.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:53.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:30:53.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:30:53.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:53.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:53.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:53.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:30:53.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:30:53.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:53.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:53.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:53.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:30:53.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:30:53.541 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:53.541 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:53.541 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:53.541 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:53.541 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:53.541 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:53.546 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:30:53.631 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:53.627+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:53.702 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:53.698+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:53.714 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:53.714 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:53.714 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:53.714 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:53.723 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:30:53.808 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:53.804+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:53.891 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:53.892 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:53.892 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:53.892 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:53.892 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:53.901 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:30:53.903 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:53.899+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:54.004 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:54.000+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:54.040 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-12-06T18:30:54.040 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi061 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c" 2023-12-06T18:30:54.041 INFO:tasks.ceph.mon.c:Stopped 2023-12-06T18:30:54.041 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-12-06T18:30:54.041 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:30:54.056 INFO:tasks.ceph.mon.a.smithi061.stderr:2023-12-06T18:30:54.042+0000 9f54640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 107948) UID: 0 2023-12-06T18:30:54.057 INFO:tasks.ceph.mon.a.smithi061.stderr:2023-12-06T18:30:54.044+0000 9f54640 -1 mon.a@0(probing) e1 *** Got Signal Terminated *** 2023-12-06T18:30:54.110 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:54.110 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:54.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:54.110 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:54.110 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.111 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:30:54.111 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:54.111 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 237.493263896, 2023-12-06T18:30:54.111 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:30:54.111 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.111 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.111 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.111 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.111 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.111 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:54.111 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.111 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.111 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.111 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.111 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:54.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.112 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.112 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.112 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:30:54.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:30:54.112 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.112 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.112 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:30:54.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:30:54.112 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.112 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:30:54.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:30:54.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:30:54.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:30:54.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:30:54.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:30:54.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:30:54.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:30:54.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.114 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:30:54.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:30:54.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.114 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:30:54.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:30:54.115 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.115 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.115 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.115 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.115 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:30:54.115 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:54.115 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 236.47043688700001, 2023-12-06T18:30:54.115 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:30:54.115 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.115 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.115 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.115 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.115 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.116 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:54.116 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.116 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.116 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.116 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.116 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:54.116 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.116 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.116 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.116 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.116 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:30:54.117 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:30:54.117 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.117 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.117 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:30:54.117 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:30:54.117 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.117 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.117 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:30:54.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:30:54.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:30:54.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:30:54.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:30:54.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:30:54.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:30:54.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:30:54.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:30:54.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:30:54.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.119 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.119 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:30:54.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:30:54.120 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.120 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.120 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.120 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.120 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:30:54.120 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:54.120 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 236.45839024899999, 2023-12-06T18:30:54.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:30:54.120 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.120 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.120 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.120 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.120 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:54.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:54.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:30:54.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:30:54.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:30:54.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:30:54.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.122 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.122 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:30:54.122 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:30:54.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.122 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.122 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:30:54.122 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:30:54.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.122 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.122 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:30:54.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:30:54.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:30:54.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:30:54.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:30:54.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:30:54.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:30:54.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:30:54.124 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.124 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.124 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.124 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.124 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.124 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:30:54.124 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:54.124 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 236.43698964199999, 2023-12-06T18:30:54.124 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:30:54.124 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.124 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.124 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.124 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.124 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.125 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:54.125 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.125 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.125 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.125 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:54.125 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.125 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.125 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.125 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:30:54.125 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:30:54.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.125 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.125 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.126 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:30:54.126 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:30:54.126 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.126 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.126 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.126 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:30:54.126 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:30:54.126 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.126 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.127 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.127 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:30:54.127 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:30:54.127 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.127 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.127 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.127 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:30:54.127 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:30:54.127 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.127 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.127 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.127 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:30:54.127 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:30:54.127 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.128 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.128 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.128 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:30:54.128 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:30:54.128 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.128 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.128 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.128 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:30:54.128 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:30:54.128 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.128 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.128 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.128 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.128 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.128 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:30:54.129 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:54.129 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 236.43581031900001, 2023-12-06T18:30:54.129 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:30:54.129 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.129 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.129 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.129 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.129 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.129 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:54.129 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.129 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.129 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.129 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.129 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:54.129 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.130 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.130 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.130 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.130 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:30:54.130 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:30:54.130 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.130 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.130 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.130 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:30:54.130 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:30:54.130 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.130 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.130 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.131 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:30:54.131 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:30:54.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.131 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.131 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.131 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:30:54.131 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:30:54.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.131 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.131 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.131 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:30:54.131 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:30:54.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.131 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.131 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.131 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:30:54.132 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:30:54.132 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.132 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.132 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.132 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:30:54.132 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:30:54.132 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.132 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.132 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.132 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:30:54.132 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:30:54.132 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.132 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.132 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.133 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.133 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.133 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:30:54.133 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:54.133 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 236.435238338, 2023-12-06T18:30:54.133 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:30:54.133 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.133 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.133 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.133 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.133 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.133 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:54.133 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.133 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.133 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.134 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.134 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:54.134 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.134 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.134 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.134 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.134 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:30:54.134 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:30:54.134 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.134 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.134 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.134 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:30:54.134 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:30:54.134 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.135 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.135 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.135 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:30:54.135 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:30:54.135 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.135 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.135 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.135 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:30:54.135 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:30:54.135 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.135 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.135 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.135 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:30:54.135 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:30:54.135 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.136 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.136 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.136 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:30:54.136 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:30:54.136 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.136 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.136 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.136 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:30:54.136 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:30:54.136 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.136 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.136 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.136 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:30:54.136 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:30:54.136 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.137 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.137 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.137 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.137 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.137 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:30:54.137 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:54.137 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 235.98940286800001, 2023-12-06T18:30:54.137 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:30:54.137 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.137 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.137 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.137 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.137 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.137 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:54.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.138 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:54.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.138 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:30:54.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:30:54.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:30:54.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:30:54.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:30:54.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:30:54.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:30:54.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:30:54.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.140 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:30:54.140 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:30:54.140 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.140 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.140 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.140 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:30:54.140 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:30:54.140 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.140 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.140 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.140 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:30:54.140 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:30:54.140 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.140 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.140 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.141 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:30:54.141 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:30:54.141 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.141 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.141 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.141 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.141 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.141 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:30:54.141 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:54.141 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 235.987803012, 2023-12-06T18:30:54.141 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:30:54.141 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.141 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.141 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.142 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.142 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.142 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:54.142 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.142 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.142 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.142 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.142 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:54.142 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.142 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.142 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.142 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.142 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:30:54.142 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:30:54.143 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.143 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.143 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.143 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:30:54.143 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:30:54.143 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.143 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.143 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.143 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:30:54.143 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:30:54.143 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.143 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.143 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.143 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:30:54.143 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:30:54.144 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.144 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.144 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.144 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:30:54.144 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:30:54.144 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.144 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.144 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.144 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:30:54.144 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:30:54.144 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.144 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.144 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.144 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:30:54.144 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:30:54.145 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.145 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.145 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.145 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:30:54.145 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:30:54.145 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.145 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.145 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.145 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.145 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.145 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:30:54.145 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:54.145 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 235.60859739599999, 2023-12-06T18:30:54.145 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:30:54.145 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.146 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.146 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.146 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.146 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.146 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:54.146 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.146 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.146 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.146 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.146 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:54.146 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.146 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.146 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.146 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.146 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:30:54.147 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:30:54.147 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.147 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.147 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:30:54.147 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:30:54.147 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.147 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.147 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:30:54.147 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:30:54.147 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.147 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.147 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:30:54.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:30:54.148 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.148 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.148 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:30:54.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:30:54.148 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.148 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.148 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:30:54.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:30:54.148 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.148 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.148 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:30:54.149 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:30:54.149 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.149 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.149 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.149 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:30:54.149 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:30:54.149 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.149 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.149 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.149 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.149 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.149 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:30:54.149 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:54.149 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 235.37109521400001, 2023-12-06T18:30:54.149 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:30:54.149 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.150 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.150 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.150 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.150 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:54.150 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.150 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.150 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:54.150 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.150 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.150 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:30:54.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:30:54.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.151 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.151 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.151 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:30:54.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:30:54.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.151 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.151 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.151 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:30:54.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:30:54.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.151 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.151 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.151 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:30:54.152 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:30:54.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.152 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.152 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.152 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:30:54.152 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:30:54.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.152 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.152 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.152 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:30:54.152 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:30:54.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.152 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.152 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.152 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:30:54.153 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:30:54.153 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.153 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.153 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.153 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:30:54.153 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:30:54.153 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.153 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.153 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.153 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.153 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.153 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:30:54.153 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:54.153 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 235.37036721499999, 2023-12-06T18:30:54.153 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:30:54.154 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.154 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.154 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.154 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.154 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.154 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:54.154 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.154 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.154 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.154 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.154 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:54.154 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.154 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.154 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.155 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.155 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:30:54.155 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:30:54.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.155 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.155 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.155 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:30:54.155 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:30:54.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.155 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.155 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.155 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:30:54.155 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:30:54.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.155 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.156 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.156 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:30:54.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:30:54.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.156 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.156 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.156 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:30:54.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:30:54.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.156 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.156 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.156 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:30:54.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:30:54.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.156 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.157 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.157 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:30:54.157 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:30:54.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.157 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.157 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.157 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:30:54.157 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:30:54.157 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.157 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.157 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.157 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.157 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:30:54.157 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:54.157 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 235.36535480500001, 2023-12-06T18:30:54.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:30:54.158 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.158 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.158 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.158 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.158 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.158 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:54.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.158 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.158 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.158 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.158 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:54.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.158 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:30:54.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:30:54.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:30:54.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:30:54.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:30:54.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:30:54.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.160 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.160 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:30:54.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:30:54.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.160 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.160 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:30:54.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:30:54.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.160 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.160 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:30:54.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:30:54.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.161 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.161 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:30:54.161 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:30:54.161 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.161 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.161 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:30:54.161 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:30:54.161 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.161 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.161 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.161 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.161 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:30:54.161 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:54.162 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 235.35982772599999, 2023-12-06T18:30:54.162 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:30:54.162 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.162 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.162 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.162 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.162 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.162 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:54.162 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.162 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.162 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.162 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.162 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:54.162 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.163 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.163 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.163 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.163 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:30:54.163 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:30:54.163 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.163 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.163 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.163 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:30:54.163 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:30:54.163 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.163 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.163 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.163 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:30:54.163 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:30:54.163 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.164 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.164 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.164 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:30:54.164 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:30:54.164 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.164 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.164 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.164 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:30:54.164 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:30:54.164 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.164 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.164 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.164 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:30:54.164 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:30:54.165 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.165 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.165 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.165 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:30:54.165 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:30:54.165 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.165 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.165 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.165 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:30:54.165 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:30:54.165 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.165 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.165 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.165 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.165 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.166 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:30:54.166 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:54.166 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 235.35931688900001, 2023-12-06T18:30:54.166 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:30:54.166 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.166 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.166 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.166 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.166 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.166 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:54.166 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.166 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.166 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.166 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.167 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:54.167 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.167 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.167 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.167 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.167 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:30:54.167 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:30:54.167 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.167 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.167 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.167 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:30:54.167 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:30:54.167 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.167 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.167 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.168 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:30:54.168 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:30:54.168 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.168 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.168 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.168 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:30:54.168 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:30:54.168 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.168 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.168 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.168 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:30:54.168 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:30:54.168 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.168 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.168 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.169 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:30:54.169 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:30:54.169 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.169 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.169 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.169 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:30:54.169 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:30:54.169 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.169 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.169 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.169 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:30:54.169 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:30:54.169 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.169 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.169 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.170 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.170 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.170 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:30:54.170 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:54.170 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 235.35885762800001, 2023-12-06T18:30:54.170 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:30:54.170 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.170 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.170 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.170 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.170 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.170 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:54.170 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.170 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.171 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.171 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.171 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:54.171 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.171 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.171 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.171 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.171 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:30:54.171 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:30:54.171 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.171 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.171 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.171 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:30:54.171 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:30:54.171 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.172 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.172 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.172 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:30:54.172 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:30:54.172 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.172 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.172 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.172 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:30:54.172 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:30:54.172 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.172 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.172 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.172 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:30:54.172 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:30:54.172 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.173 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.173 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.173 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:30:54.173 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:30:54.173 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.173 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.173 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.173 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:30:54.173 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:30:54.173 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.173 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.173 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.173 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:30:54.173 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:30:54.173 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.173 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.174 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.174 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.174 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.174 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:30:54.174 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:54.174 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 228.77894182099999, 2023-12-06T18:30:54.174 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:30:54.174 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.174 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.174 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.174 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.174 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.174 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:54.174 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.175 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.175 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.175 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.175 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:54.175 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.175 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.175 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.175 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.175 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:30:54.175 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:30:54.175 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.175 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.175 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.175 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:30:54.175 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:30:54.176 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.176 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.176 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.176 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:30:54.176 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:30:54.176 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.176 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.176 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.176 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:30:54.176 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:30:54.176 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.176 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.176 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.176 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:30:54.177 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:30:54.177 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.177 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.177 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.177 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:30:54.177 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:30:54.177 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.177 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.177 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.177 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:30:54.177 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:30:54.177 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.177 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.177 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.177 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:30:54.178 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:30:54.178 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.178 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.178 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.178 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.178 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.178 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:30:54.178 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:54.178 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 228.77270401600001, 2023-12-06T18:30:54.178 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:30:54.178 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.178 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.178 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.178 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.179 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.179 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:54.179 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.179 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.179 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.179 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.179 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:54.179 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.179 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.179 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.179 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.179 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:30:54.179 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:30:54.179 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.179 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.180 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.180 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:30:54.180 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:30:54.180 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.180 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.180 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.180 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:30:54.180 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:30:54.180 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.180 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.180 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.180 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:30:54.180 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:30:54.180 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.180 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.181 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.181 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:30:54.181 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:30:54.181 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.181 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.181 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.181 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:30:54.181 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:30:54.181 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.181 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.181 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.181 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:30:54.181 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:30:54.181 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.181 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.182 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.182 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:30:54.182 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:30:54.182 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.182 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.182 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.182 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.182 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.182 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:30:54.182 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:54.182 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 228.76664093700001, 2023-12-06T18:30:54.182 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:30:54.182 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.182 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.183 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.183 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.183 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.183 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:54.183 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.183 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.183 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.183 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.183 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:54.183 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.183 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.183 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.183 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.183 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:30:54.183 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:30:54.183 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.184 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.184 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.184 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:30:54.184 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:30:54.184 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.184 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.184 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.184 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:30:54.184 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:30:54.184 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.184 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.184 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.184 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:30:54.184 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:30:54.184 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.185 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.185 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.185 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:30:54.185 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:30:54.185 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.185 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.185 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.185 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:30:54.185 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:30:54.185 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.185 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.185 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.185 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:30:54.185 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:30:54.186 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.186 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.186 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.186 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:30:54.186 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:30:54.186 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.186 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.186 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.186 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.186 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.186 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:30:54.186 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:54.186 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 228.62925385200001, 2023-12-06T18:30:54.186 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:30:54.186 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.187 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.187 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.187 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.187 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.187 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:54.187 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.187 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.187 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.187 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.187 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:54.187 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.187 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.187 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.187 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.188 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:30:54.188 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:30:54.188 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.188 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.188 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.188 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:30:54.188 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:30:54.188 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.188 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.188 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.188 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:30:54.188 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:30:54.188 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.188 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.188 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.188 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:30:54.189 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:30:54.189 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.189 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.189 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.189 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:30:54.189 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:30:54.189 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.189 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.189 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.189 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:30:54.189 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:30:54.189 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.189 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.189 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.189 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:30:54.190 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:30:54.190 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.190 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.190 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.190 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:30:54.190 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:30:54.190 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.190 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.190 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.190 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.190 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.190 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:30:54.190 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:54.190 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 228.62258312500001, 2023-12-06T18:30:54.191 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:30:54.191 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:54.191 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:54.191 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:54.191 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.191 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:54.191 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:54.191 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.191 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.191 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.191 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:54.191 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:54.191 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:54.191 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.191 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.191 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:54.192 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:30:54.192 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:30:54.192 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.192 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.192 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:54.192 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:30:54.192 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:30:54.192 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.192 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.192 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:54.192 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:30:54.192 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:30:54.192 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.192 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.192 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:54.193 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:30:54.193 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:30:54.193 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.193 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.193 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:54.193 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:30:54.193 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:30:54.193 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.193 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.193 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:54.193 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:30:54.193 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:30:54.193 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.193 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.193 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:54.193 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:30:54.194 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:30:54.194 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:54.194 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:54.194 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:54.194 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:30:54.194 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:30:54.194 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.194 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.194 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.194 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:54.194 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:54.194 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:54.195 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:54.106+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:54.196 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:30:54.238 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:54.234+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:54.306 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:54.303+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:54.367 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:54.367 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:54.367 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:54.367 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:54.375 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:30:54.416 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:54.413+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:54.508 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:54.505+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:54.546 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:54.546 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:54.546 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:54.546 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:54.546 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:54.556 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:30:54.599 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:53.030 107953== 2023-12-06T18:30:54.599 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:53.030 107953== Exit program on first error (--exit-on-first-error=yes) 2023-12-06T18:30:54.609 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:54.605+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:54.609 INFO:tasks.ceph.mon.a.smithi061.stderr:daemon-helper: command failed with exit status 42 2023-12-06T18:30:54.710 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:54.706+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:54.725 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:54.725 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:54.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:54.726 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:30:54.726 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:54.735 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:30:54.811 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:54.807+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:54.863 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:54.864 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:54.864 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:54.864 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:54.873 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:30:54.917 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:54.913+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:55.002 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:55.002 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:55.002 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:55.002 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:55.003 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:55.013 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:30:55.021 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:55.015+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:55.114 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:55.111+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:55.158 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:55.158 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:55.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:55.158 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:55.158 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:55.158 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:30:55.158 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:55.158 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 33.495975928999997, 2023-12-06T18:30:55.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:30:55.158 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:55.158 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:55.159 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:55.159 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:55.159 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:55.159 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:30:55.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:55.159 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:55.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:55.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:55.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:55.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:55.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:55.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:55.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:55.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:55.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:55.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:55.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:55.160 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:55.160 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:30:55.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:30:55.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:55.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:55.160 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:55.160 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:30:55.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:30:55.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:55.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:55.160 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:55.160 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:30:55.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:30:55.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:55.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:55.161 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:55.161 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:30:55.161 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:30:55.161 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:55.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:55.161 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:55.161 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:30:55.161 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:30:55.161 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:55.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:55.161 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:55.161 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:30:55.161 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:30:55.161 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:55.161 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:55.162 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:55.162 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:55.162 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:55.162 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:55.167 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:30:55.215 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:55.212+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:55.310 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:55.310 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:55.310 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:55.310 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:55.317 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:55.313+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:55.319 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:30:55.418 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:55.415+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:55.498 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:55.498 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:55.498 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:55.498 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:55.498 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:55.507 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:30:55.518 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:55.515+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:55.620 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:55.616+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:55.645 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:55.645 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:55.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:55.645 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:30:55.645 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:55.654 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:30:55.739 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:55.735+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:55.792 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-12-06T18:30:55.792 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-12-06T18:30:55.822 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:55.819+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:55.836 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:55.836 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:55.836 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:55.836 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:55.845 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:30:55.930 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:55.926+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:56.023 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:56.023 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:56.023 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:56.023 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:56.023 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:56.023 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:56.020+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:56.032 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:30:56.124 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:56.121+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:56.267 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:56.267 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:56.267 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:56.267 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:56.267 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.267 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:30:56.267 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:56.268 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 240.85681263999999, 2023-12-06T18:30:56.268 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:30:56.268 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.268 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.268 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.268 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.268 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.268 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:30:56.268 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.268 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.268 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.268 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.268 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:56.268 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.268 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.269 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.269 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:30:56.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.269 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.269 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:30:56.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:30:56.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.269 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.269 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:30:56.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:30:56.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.270 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.270 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:30:56.270 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:30:56.270 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.270 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.270 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.270 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:30:56.270 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:30:56.270 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.270 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.270 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.270 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:30:56.270 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:30:56.270 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.271 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.271 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:30:56.271 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:30:56.271 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:30:56.271 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.271 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.271 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.271 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:30:56.271 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:30:56.271 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.271 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.271 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.271 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:30:56.271 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:30:56.271 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.272 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.272 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.272 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:30:56.272 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:30:56.272 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.272 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.272 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.272 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:30:56.272 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:30:56.272 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.272 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.273 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.273 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:30:56.273 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:30:56.273 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.273 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.273 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.273 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:30:56.273 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:30:56.273 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.273 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.273 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.273 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:30:56.273 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:30:56.273 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.273 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.274 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.274 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.274 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.274 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:56.274 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:56.274 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 238.74823940100001, 2023-12-06T18:30:56.274 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:30:56.274 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.274 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.274 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.274 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.275 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.275 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:30:56.275 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.275 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.275 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.275 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:56.275 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.275 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.275 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.275 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:30:56.275 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.275 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.276 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:30:56.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:30:56.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.276 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:30:56.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:30:56.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.276 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:30:56.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:30:56.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.277 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.277 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.277 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:30:56.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:30:56.277 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.277 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.277 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.277 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:30:56.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:30:56.277 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.277 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.277 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.277 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:30:56.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:30:56.277 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.277 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.278 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.278 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:30:56.278 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:30:56.278 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.278 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.278 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.278 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:30:56.278 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:30:56.278 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.278 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.278 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.278 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:30:56.278 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:30:56.278 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.278 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:30:56.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:30:56.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.279 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:30:56.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:30:56.279 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.279 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.279 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.279 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.279 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:56.279 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:56.280 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 238.74749416899999, 2023-12-06T18:30:56.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:30:56.280 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.280 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.280 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.280 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.280 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.280 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:30:56.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.280 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:56.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.281 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:30:56.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.281 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:30:56.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:30:56.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.281 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:30:56.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:30:56.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:30:56.282 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:30:56.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:30:56.282 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:30:56.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:30:56.283 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:30:56.283 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.283 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.283 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.283 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:30:56.283 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:30:56.283 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.283 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.283 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.283 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:30:56.283 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:30:56.283 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.283 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.283 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.284 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:30:56.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:30:56.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.284 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.284 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:30:56.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:30:56.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.284 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.284 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:30:56.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:30:56.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.284 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.284 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:30:56.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:30:56.285 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.285 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.285 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.285 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:56.285 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:56.285 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 238.74681041400001, 2023-12-06T18:30:56.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:30:56.285 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.285 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.285 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.285 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.285 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.286 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:30:56.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.286 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.286 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:56.286 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.286 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:30:56.286 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:30:56.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:30:56.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:30:56.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:30:56.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:30:56.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:30:56.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.288 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:30:56.288 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:30:56.288 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.288 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.288 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.288 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:30:56.288 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:30:56.288 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.288 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.288 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.288 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:30:56.288 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:30:56.288 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.288 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.288 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.288 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:30:56.289 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:30:56.289 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.289 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.289 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.289 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:30:56.289 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:30:56.289 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.289 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.289 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.289 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:30:56.289 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:30:56.289 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.289 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.289 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.289 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:30:56.290 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:30:56.290 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.290 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.290 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:30:56.290 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:30:56.290 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.290 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.290 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.290 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.290 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:56.290 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:56.290 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 238.74123715900001, 2023-12-06T18:30:56.290 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:30:56.291 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.291 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.291 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.291 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.291 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.291 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:30:56.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.291 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.291 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.291 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.291 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:56.291 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.291 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.292 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.292 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:30:56.292 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.292 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.292 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:30:56.292 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:30:56.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.292 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.292 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:30:56.292 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:30:56.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.293 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.293 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:30:56.293 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:30:56.293 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.293 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.293 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.293 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:30:56.293 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:30:56.293 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.293 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.293 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.293 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:30:56.293 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:30:56.293 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.294 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.294 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.294 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:30:56.294 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:30:56.294 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.294 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.294 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.294 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:30:56.294 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:30:56.294 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.294 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.294 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.294 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:30:56.294 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:30:56.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.295 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.295 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.295 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:30:56.295 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:30:56.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.295 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.295 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.295 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:30:56.295 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:30:56.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.295 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.295 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.295 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:30:56.296 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:30:56.296 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.296 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.296 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.296 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.296 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.296 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:56.296 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:56.296 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 238.74056351900001, 2023-12-06T18:30:56.296 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:30:56.296 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.296 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.296 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.296 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.296 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.297 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:30:56.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.297 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.297 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.297 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.297 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:56.297 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.297 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.297 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.297 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:30:56.297 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.297 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.298 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.298 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:30:56.298 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:30:56.298 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.298 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.298 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.298 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:30:56.298 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:30:56.298 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.298 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.298 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.298 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:30:56.298 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:30:56.299 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.299 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.299 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.299 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:30:56.299 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:30:56.299 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.299 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.299 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.299 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:30:56.299 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:30:56.299 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.299 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.299 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.299 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:30:56.299 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:30:56.300 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.300 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.300 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.300 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:30:56.300 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:30:56.300 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.300 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.300 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.300 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:30:56.300 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:30:56.300 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.300 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.300 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.300 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:30:56.300 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:30:56.301 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.301 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.301 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.301 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:30:56.301 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:30:56.301 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.301 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.301 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.301 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:30:56.301 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:30:56.301 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.301 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.301 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.301 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.301 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.302 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:56.302 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:56.302 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 238.73990132, 2023-12-06T18:30:56.302 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:30:56.302 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.302 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.302 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.302 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.302 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.302 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:30:56.302 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.302 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.302 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.302 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:56.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.303 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:30:56.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.303 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:30:56.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:30:56.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.303 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:30:56.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:30:56.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:30:56.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:30:56.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:30:56.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:30:56.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:30:56.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:30:56.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:30:56.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:30:56.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:30:56.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:30:56.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:30:56.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:30:56.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:30:56.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:30:56.306 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.306 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:30:56.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:30:56.306 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.306 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:30:56.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:30:56.307 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.307 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.307 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.307 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.307 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:56.307 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:56.307 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 238.66719188499999, 2023-12-06T18:30:56.307 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:30:56.307 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.307 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.307 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.307 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.307 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.307 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:30:56.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.308 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.308 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.308 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.308 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:56.308 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.308 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.308 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.308 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.308 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:30:56.308 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.308 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.308 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.308 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.308 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:30:56.308 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:30:56.308 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.308 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.309 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.309 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:30:56.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:30:56.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.309 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.309 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:30:56.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:30:56.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.309 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.309 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:30:56.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:30:56.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.310 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.310 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:30:56.310 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:30:56.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.310 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.310 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.310 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:30:56.310 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:30:56.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.310 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.310 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.310 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:30:56.310 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:30:56.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.310 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.311 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:30:56.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:30:56.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.311 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:30:56.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:30:56.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.311 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:30:56.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:30:56.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:30:56.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:30:56.312 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.312 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.312 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.312 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:30:56.312 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:56.312 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 238.666510176, 2023-12-06T18:30:56.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:30:56.312 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.312 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.313 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.313 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.313 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.313 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:30:56.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.313 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.313 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:56.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.313 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:30:56.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:30:56.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:30:56.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:30:56.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:30:56.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:30:56.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:30:56.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.315 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.315 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:30:56.315 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:30:56.315 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.315 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.315 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.315 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:30:56.315 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:30:56.315 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.315 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.315 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.315 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:30:56.315 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:30:56.315 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.315 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.316 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.316 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:30:56.316 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:30:56.316 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.316 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.316 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.316 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:30:56.316 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:30:56.316 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.316 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.316 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.316 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:30:56.316 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:30:56.316 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.316 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.317 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.317 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:30:56.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:30:56.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.317 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.317 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:30:56.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:30:56.317 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.317 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.317 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.317 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:56.317 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:56.318 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 237.587706088, 2023-12-06T18:30:56.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:30:56.318 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.318 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.318 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.318 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.318 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.318 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:30:56.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.318 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:56.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:30:56.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:30:56.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:30:56.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:30:56.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:30:56.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:30:56.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:30:56.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:30:56.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:30:56.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:30:56.321 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:30:56.321 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.321 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.321 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.321 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:30:56.321 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:30:56.321 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.321 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.321 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.321 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:30:56.321 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:30:56.321 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.321 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.321 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.321 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:30:56.322 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:30:56.322 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.322 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.322 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.322 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:30:56.322 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:30:56.322 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.322 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.322 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.322 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:30:56.322 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:30:56.322 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.322 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.322 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.323 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:30:56.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:30:56.323 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.323 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.323 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.323 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.323 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:56.323 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:56.323 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 237.58628951399999, 2023-12-06T18:30:56.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:30:56.323 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.323 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.323 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.323 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.324 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.324 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:30:56.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.324 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.324 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:56.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.324 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:30:56.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:30:56.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:30:56.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:30:56.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:30:56.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:30:56.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:30:56.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.326 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:30:56.326 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:30:56.326 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.326 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.326 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.326 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:30:56.326 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:30:56.326 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.326 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.326 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.326 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:30:56.326 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:30:56.326 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.326 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.326 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.327 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:30:56.327 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:30:56.327 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.327 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.327 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.327 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:30:56.327 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:30:56.327 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.327 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.327 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.327 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:30:56.327 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:30:56.327 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.327 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.327 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.328 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:30:56.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:30:56.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.328 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.328 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.328 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:30:56.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:30:56.328 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.328 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.328 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.328 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.328 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:56.328 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:56.328 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 237.58520438900001, 2023-12-06T18:30:56.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:30:56.329 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.329 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.329 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.329 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.329 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.329 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:30:56.329 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.329 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.329 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:56.329 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.329 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:30:56.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:30:56.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:30:56.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:30:56.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:30:56.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.331 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:30:56.331 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:30:56.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.331 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.331 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.331 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:30:56.331 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:30:56.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.331 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.331 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.331 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:30:56.331 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:30:56.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.331 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.331 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.332 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:30:56.332 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:30:56.332 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.332 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.332 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.332 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:30:56.332 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:30:56.332 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.332 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.332 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.332 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:30:56.332 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:30:56.332 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.332 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.332 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.333 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:30:56.333 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:30:56.333 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.333 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.333 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.333 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:30:56.333 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:30:56.333 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.333 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.333 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.333 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:30:56.333 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:30:56.333 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.333 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.333 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.334 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.334 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.334 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:30:56.334 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:56.334 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 237.55005528699999, 2023-12-06T18:30:56.334 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:30:56.334 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.334 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.334 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.334 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.334 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.334 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:30:56.334 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.334 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.334 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.334 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.335 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:56.335 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.335 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.335 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.335 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.335 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:30:56.335 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.335 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.335 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.335 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.335 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:30:56.335 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:30:56.335 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.335 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.335 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.336 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:30:56.336 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:30:56.336 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.336 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.336 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.336 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:30:56.336 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:30:56.336 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.336 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.336 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.336 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:30:56.336 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:30:56.336 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.336 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.336 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.336 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:30:56.337 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:30:56.337 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.337 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.337 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.337 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:30:56.337 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:30:56.337 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.337 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.337 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.337 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:30:56.337 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:30:56.337 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.337 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.337 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.337 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:30:56.337 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:30:56.338 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.338 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.338 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.338 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:30:56.338 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:30:56.338 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.338 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.338 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.338 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:30:56.338 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:30:56.338 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.338 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.338 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.338 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:30:56.338 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:30:56.339 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.339 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.339 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.339 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.339 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.339 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:56.339 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:56.339 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 231.00032476199999, 2023-12-06T18:30:56.339 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:30:56.339 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.339 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.339 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.339 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.339 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.339 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:30:56.339 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.340 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.340 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.340 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.340 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:56.340 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.340 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.340 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.340 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.340 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:30:56.340 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.340 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.340 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.340 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.340 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:30:56.340 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:30:56.341 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.341 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.341 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.341 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:30:56.341 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:30:56.341 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.341 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.341 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.341 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:30:56.341 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:30:56.341 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.341 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.341 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.341 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:30:56.341 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:30:56.342 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.342 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.342 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.342 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:30:56.342 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:30:56.342 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.342 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.342 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.342 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:30:56.342 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:30:56.342 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.342 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.342 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.342 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:30:56.342 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:30:56.343 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.343 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.343 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.343 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:30:56.343 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:30:56.343 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.343 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.343 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.343 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:30:56.343 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:30:56.343 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.343 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.343 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.343 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:30:56.343 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:30:56.343 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.344 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.344 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.344 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:30:56.344 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:30:56.344 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.344 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.344 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.344 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.344 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.344 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:56.344 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:56.344 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 230.99951915400001, 2023-12-06T18:30:56.344 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:30:56.344 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.345 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.345 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.345 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.345 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.345 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:30:56.345 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.345 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.345 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.345 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.345 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:56.345 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.345 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.345 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.345 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.345 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:30:56.346 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.346 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.346 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.346 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.346 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:30:56.346 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:30:56.346 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.346 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.346 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.346 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:30:56.346 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:30:56.346 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.346 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.346 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.347 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:30:56.347 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:30:56.347 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.347 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.347 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.347 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:30:56.347 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:30:56.347 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.347 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.347 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.347 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:30:56.347 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:30:56.347 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.347 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.347 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.348 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:30:56.348 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:30:56.348 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.348 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.348 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.348 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:30:56.348 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:30:56.348 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.348 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.348 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.348 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:30:56.348 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:30:56.348 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.348 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.348 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.348 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:30:56.349 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:30:56.349 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.349 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.349 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.349 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:30:56.349 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:30:56.349 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.349 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.349 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.349 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:30:56.349 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:30:56.349 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.349 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.349 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.349 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.349 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.350 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:56.350 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:56.350 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 230.99882834499999, 2023-12-06T18:30:56.350 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:30:56.350 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.350 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.350 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.350 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.350 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.350 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:30:56.350 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.350 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.350 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.350 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.351 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:56.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.351 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:30:56.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.351 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:30:56.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:30:56.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:30:56.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:30:56.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:30:56.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:30:56.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:30:56.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:30:56.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:30:56.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:30:56.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:30:56.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:30:56.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:30:56.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:30:56.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:30:56.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:30:56.354 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.354 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.354 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.354 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:30:56.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:30:56.354 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.354 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.354 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.354 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:30:56.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:30:56.354 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.354 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.354 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.354 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:30:56.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:30:56.355 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.355 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.355 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.355 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.355 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.355 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:56.355 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:56.357 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 230.99453083700001, 2023-12-06T18:30:56.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:30:56.357 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.357 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.357 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.357 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.357 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.357 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:30:56.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.357 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.358 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.358 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.358 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:56.358 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.358 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.358 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.358 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.358 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:30:56.358 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.358 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.358 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.358 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.358 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:30:56.358 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:30:56.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.359 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.359 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.359 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:30:56.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:30:56.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.359 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.359 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.359 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:30:56.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:30:56.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.359 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.359 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.359 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:30:56.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:30:56.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.360 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.360 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.360 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:30:56.360 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:30:56.360 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.360 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.360 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.360 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:30:56.360 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:30:56.360 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.360 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.360 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.360 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:30:56.360 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:30:56.360 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.360 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.360 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.361 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:30:56.361 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:30:56.361 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.361 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.361 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.361 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:30:56.361 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:30:56.361 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.361 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.361 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.361 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:30:56.361 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:30:56.361 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.361 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.361 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.362 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:30:56.362 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:30:56.362 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.362 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.362 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.362 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.362 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:56.362 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:56.362 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 230.993863162, 2023-12-06T18:30:56.362 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:30:56.362 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.362 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.362 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.362 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.363 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.363 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:30:56.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.363 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:56.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:30:56.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:30:56.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:30:56.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:30:56.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:30:56.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:30:56.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:30:56.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:30:56.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:30:56.365 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.365 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:30:56.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:30:56.365 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.365 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:30:56.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:30:56.365 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.365 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:30:56.366 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:30:56.366 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.366 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.366 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.366 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:30:56.366 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:30:56.366 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.366 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.366 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.366 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:30:56.366 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:30:56.366 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.366 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.366 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.366 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:30:56.367 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:30:56.367 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.367 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.367 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.367 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:30:56.367 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:30:56.367 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.367 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.367 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.367 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.367 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.367 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:56.367 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:56.367 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 230.99320773400001, 2023-12-06T18:30:56.367 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:30:56.367 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.368 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.368 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.368 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.368 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.368 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:30:56.368 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.368 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.368 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.368 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:56.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.368 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.368 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.368 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:30:56.369 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.369 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.369 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.369 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:30:56.369 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:30:56.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.369 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.369 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.369 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:30:56.369 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:30:56.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.369 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.369 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.369 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:30:56.369 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:30:56.370 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.370 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.370 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.370 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:30:56.370 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:30:56.370 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.370 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.370 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.370 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:30:56.370 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:30:56.370 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.370 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.370 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.370 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:30:56.370 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:30:56.371 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.371 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.371 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.371 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:30:56.371 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:30:56.371 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.371 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.371 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.371 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:30:56.371 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:30:56.371 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.371 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.371 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.371 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:30:56.371 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:30:56.372 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.372 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.372 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.372 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:30:56.372 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:30:56.372 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.372 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.372 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.372 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:30:56.372 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:30:56.372 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.372 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.372 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.372 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.372 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.372 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:30:56.373 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:56.373 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 230.95255318700001, 2023-12-06T18:30:56.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:30:56.373 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.373 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:56.373 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:56.373 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:30:56.373 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:30:56.373 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:30:56.373 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.373 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.373 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.373 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:56.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:30:56.374 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:30:56.374 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:30:56.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:30:56.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:30:56.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:30:56.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:30:56.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:30:56.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:30:56.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:30:56.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:30:56.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:30:56.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:30:56.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:30:56.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:30:56.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:30:56.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:30:56.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:30:56.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:30:56.377 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.377 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:30:56.377 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:30:56.377 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:30:56.377 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.377 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:30:56.377 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:30:56.377 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:30:56.377 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.377 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.377 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:30:56.377 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:30:56.378 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.378 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.378 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.378 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.378 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.378 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:56.379 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:56.264+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:56.380 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:30:56.380 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:56.376+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:56.481 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:56.477+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:56.557 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:56.557 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:56.557 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:56.557 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:56.566 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:30:56.608 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:56.603+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:56.683 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:56.679+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:56.741 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:56.741 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:56.741 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:56.742 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:56.742 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:56.750 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:30:56.784 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:56.781+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:56.886 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:56.882+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:56.931 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:56.931 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:56.931 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:56.931 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:56.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.932 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:30:56.932 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:56.932 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 215.11551868000001, 2023-12-06T18:30:56.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:30:56.932 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:56.932 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:30:56.932 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:56.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.932 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:56.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:56.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.933 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:56.933 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:30:56.933 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:56.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.933 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:56.933 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:30:56.933 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:30:56.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.933 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:56.933 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:30:56.933 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:30:56.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.934 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:56.934 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:30:56.934 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:30:56.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.934 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.934 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:56.934 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:30:56.934 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:30:56.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.934 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.934 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:56.934 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:30:56.934 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:30:56.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.935 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:30:56.935 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:30:56.935 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:30:56.935 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:56.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:56.935 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:56.935 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:30:56.935 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:30:56.935 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.935 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.935 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.935 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:56.935 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:56.935 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:56.940 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:30:56.986 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:56.983+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:57.077 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:57.077 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:57.077 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:57.078 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:57.087 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:30:57.087 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:57.084+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:57.188 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:57.184+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:57.224 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:57.225 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:57.225 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:57.225 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:57.225 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:57.234 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:30:57.319 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:57.315+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:57.390 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:30:57.386+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:30:57.418 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:57.419 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:57.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:57.419 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:30:57.419 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:57.419 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:57.419 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:57.419 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 96.319208962000005, 2023-12-06T18:30:57.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:30:57.419 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:57.419 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:57.419 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:57.419 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:57.419 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:57.420 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:30:57.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:57.420 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:57.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:57.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:57.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:57.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:57.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:57.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:57.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:57.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:30:57.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:57.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:57.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:57.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:57.421 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:30:57.421 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:30:57.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:57.421 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:57.421 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:57.421 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:30:57.421 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:30:57.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:57.421 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:57.421 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:57.421 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:30:57.421 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:30:57.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:57.421 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:57.421 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:57.421 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:30:57.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:30:57.422 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:57.422 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:57.422 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:57.422 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:30:57.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:30:57.422 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:57.422 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:57.422 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:57.422 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:30:57.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:30:57.422 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:57.422 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:57.422 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:57.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:57.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:57.423 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:30:57.423 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:57.423 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 36.300951660000003, 2023-12-06T18:30:57.423 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:30:57.423 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:30:57.423 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:30:57.423 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:30:57.423 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:30:57.423 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:57.423 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:30:57.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:57.423 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:30:57.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:57.423 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:30:57.424 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:57.424 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:57.424 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:57.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:57.424 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:30:57.424 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:30:57.424 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:30:57.424 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:57.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:57.424 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:30:57.424 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:30:57.424 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:30:57.424 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:57.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:57.424 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:30:57.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:30:57.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:30:57.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:57.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:57.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:30:57.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:30:57.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:30:57.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:57.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:57.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:30:57.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:30:57.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:30:57.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:57.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:57.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:30:57.426 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:30:57.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:30:57.426 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:30:57.426 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:30:57.426 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:30:57.426 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:30:57.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:30:57.426 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:57.426 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:57.426 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:57.426 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:30:57.426 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:30:57.426 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:57.429 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:30:57.491 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:57.487+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:57.569 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:57.569 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:57.569 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:57.569 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:57.578 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:30:57.621 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:57.617+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:57.693 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:57.690+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:57.750 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:57.750 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:57.750 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:57.751 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:57.751 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:57.759 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:30:57.795 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:57.791+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:57.889 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:57.890 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:57.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:57.890 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:57.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:57.890 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:30:57.890 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:57.890 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 156.396445038, 2023-12-06T18:30:57.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:30:57.890 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:57.890 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:57.890 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:57.893 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:57.894 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:57.894 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:30:57.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:57.894 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:57.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:57.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:57.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:57.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:57.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:57.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:57.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:57.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:30:57.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:57.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:57.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:57.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:57.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:30:57.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:30:57.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:57.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:57.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:57.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:30:57.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:30:57.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:57.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:57.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:57.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:30:57.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:30:57.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:57.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:57.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:57.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:30:57.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:30:57.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:57.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:57.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:57.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:30:57.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:30:57.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:57.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:57.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:57.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:30:57.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:30:57.897 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:57.897 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:57.897 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:57.898 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:57.898 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:57.898 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:57.899 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:30:57.900 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:57.896+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:58.000 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:57.997+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:58.030 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:58.030 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:58.030 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:58.030 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:58.039 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:30:58.102 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:58.098+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:58.168 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:58.168 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:58.168 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:58.177 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:30:58.203 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:58.199+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:58.304 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:58.300+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:58.387 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:58.387 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:58.387 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:58.387 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:58.388 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.388 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:30:58.388 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:58.388 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 241.77020249, 2023-12-06T18:30:58.388 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:30:58.388 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.388 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.388 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.388 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.388 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.388 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:58.388 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.388 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.388 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.388 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.389 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:30:58.389 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.389 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.389 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.389 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.389 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:30:58.389 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:30:58.389 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.389 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.389 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.389 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:30:58.389 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:30:58.389 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.389 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.389 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.390 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:30:58.390 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:30:58.390 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.390 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.390 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.390 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:30:58.390 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:30:58.390 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.390 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.390 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.390 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:30:58.390 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:30:58.390 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.390 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.391 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.391 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:30:58.391 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:30:58.391 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.391 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.391 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.391 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:30:58.391 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:30:58.391 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.391 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.391 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.391 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:30:58.391 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:30:58.391 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.391 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.392 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.392 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.392 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.392 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:30:58.392 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:58.392 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 240.74737548100001, 2023-12-06T18:30:58.392 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:30:58.392 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.392 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.392 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.392 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.392 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:58.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:30:58.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:30:58.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:30:58.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.394 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:30:58.394 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:30:58.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.394 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.394 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.394 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:30:58.394 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:30:58.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.394 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.394 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.395 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:30:58.395 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:30:58.395 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.395 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.395 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.395 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:30:58.395 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:30:58.395 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.395 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.395 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.395 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:30:58.395 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:30:58.395 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.395 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.396 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.396 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:30:58.396 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:30:58.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.396 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.396 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.396 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:30:58.396 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:30:58.396 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.396 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.396 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.396 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.396 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:30:58.396 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:58.396 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 240.73532884299999, 2023-12-06T18:30:58.397 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:30:58.397 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.397 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.397 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.397 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.397 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:58.397 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.397 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.397 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.397 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:30:58.397 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.397 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.397 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.398 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:30:58.398 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:30:58.398 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.398 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.398 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.398 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:30:58.398 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:30:58.398 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.398 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.398 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.398 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:30:58.398 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:30:58.398 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.398 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.398 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.399 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:30:58.399 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:30:58.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.399 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.399 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:30:58.399 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:30:58.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.399 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.399 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:30:58.399 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:30:58.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:30:58.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:30:58.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:30:58.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:30:58.400 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.400 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.400 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.400 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:30:58.400 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:58.401 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 240.71392823599999, 2023-12-06T18:30:58.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:30:58.401 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.401 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.401 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:58.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:30:58.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.402 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.402 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:30:58.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:30:58.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.402 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.402 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:30:58.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:30:58.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.402 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.403 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:30:58.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:30:58.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.403 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.403 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:30:58.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:30:58.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.403 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.403 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:30:58.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:30:58.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.403 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.403 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:30:58.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:30:58.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.404 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.404 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:30:58.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:30:58.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.404 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.404 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:30:58.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:30:58.404 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.404 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.404 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.405 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:30:58.405 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:58.405 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 240.71274891300001, 2023-12-06T18:30:58.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:30:58.405 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.405 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.405 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:58.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:30:58.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:30:58.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:30:58.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:30:58.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:30:58.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.407 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:30:58.407 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:30:58.407 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.407 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.407 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.407 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:30:58.407 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:30:58.407 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.407 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.407 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.407 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:30:58.407 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:30:58.407 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.407 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.407 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.408 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:30:58.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:30:58.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.408 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.408 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.408 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:30:58.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:30:58.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.408 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.408 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.408 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:30:58.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:30:58.408 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.408 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.408 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.409 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:30:58.409 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:58.409 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 240.71217693200001, 2023-12-06T18:30:58.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:30:58.409 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.409 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.409 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:58.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:30:58.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:30:58.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:30:58.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:30:58.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:30:58.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:30:58.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:30:58.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:30:58.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:30:58.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:30:58.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:30:58.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.412 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.412 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:30:58.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:30:58.412 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.412 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.412 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:30:58.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:30:58.412 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.412 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.412 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:30:58.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:30:58.412 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.412 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.413 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.413 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.413 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:30:58.413 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:58.413 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 240.26634146200001, 2023-12-06T18:30:58.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:30:58.413 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.413 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.413 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.413 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:58.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:30:58.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:30:58.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:30:58.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:30:58.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:30:58.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:30:58.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:30:58.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:30:58.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:30:58.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:30:58.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:30:58.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:30:58.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:30:58.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:30:58.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:30:58.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:30:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:30:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:30:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 240.264741606, 2023-12-06T18:30:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:30:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:58.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:30:58.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:30:58.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:30:58.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:30:58.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:30:58.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:30:58.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:30:58.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:30:58.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:30:58.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:30:58.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:30:58.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:30:58.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:30:58.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:30:58.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:30:58.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:30:58.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:30:58.421 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.421 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.421 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.421 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:30:58.421 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:58.421 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 239.88553598999999, 2023-12-06T18:30:58.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:30:58.421 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.421 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.422 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.422 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.422 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:58.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.422 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.422 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:30:58.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.422 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.422 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:30:58.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:30:58.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:30:58.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:30:58.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:30:58.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:30:58.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:30:58.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:30:58.424 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.424 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.424 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:30:58.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:30:58.424 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.424 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.424 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:30:58.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:30:58.424 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.424 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.424 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:30:58.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:30:58.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:30:58.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:30:58.425 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.425 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.425 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.425 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:30:58.425 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:58.425 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 239.64803380800001, 2023-12-06T18:30:58.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:30:58.426 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.426 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.426 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.426 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.426 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:58.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.426 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.426 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:30:58.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.426 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.427 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:30:58.427 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:30:58.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.427 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.427 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:30:58.427 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:30:58.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.427 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.427 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:30:58.427 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:30:58.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.427 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.428 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:30:58.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:30:58.428 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.428 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.428 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.428 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:30:58.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:30:58.428 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.428 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.428 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.428 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:30:58.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:30:58.428 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:30:58.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:30:58.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:30:58.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:30:58.429 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.429 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.429 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.429 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:30:58.429 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:58.430 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 239.64730580899999, 2023-12-06T18:30:58.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:30:58.430 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.430 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.430 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:58.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:30:58.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:30:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:30:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:30:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:30:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:30:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:30:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.432 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.432 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:30:58.432 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:30:58.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.432 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.432 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:30:58.432 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:30:58.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.432 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.432 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:30:58.432 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:30:58.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:30:58.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:30:58.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:30:58.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:30:58.433 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.433 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.433 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.434 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:30:58.434 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:58.434 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 239.64229339900001, 2023-12-06T18:30:58.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:30:58.434 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.434 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.434 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.434 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.434 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:58.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.434 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:30:58.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:30:58.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:30:58.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:30:58.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:30:58.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:30:58.436 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:30:58.436 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.436 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.436 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:30:58.436 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:30:58.436 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.436 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.436 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:30:58.436 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:30:58.436 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.436 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.436 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:30:58.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:30:58.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:30:58.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:30:58.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:30:58.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:30:58.437 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.437 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.437 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.438 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:30:58.438 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:58.438 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 239.63676631999999, 2023-12-06T18:30:58.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:30:58.438 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.438 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.438 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:58.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:30:58.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:30:58.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:30:58.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:30:58.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:30:58.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:30:58.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:30:58.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:30:58.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:30:58.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:30:58.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:30:58.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:30:58.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:30:58.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:30:58.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:30:58.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:30:58.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:30:58.441 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.441 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.441 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.442 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:30:58.442 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:58.442 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 239.63625548300001, 2023-12-06T18:30:58.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:30:58.442 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.442 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.442 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:58.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:30:58.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:30:58.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:30:58.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:30:58.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:30:58.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:30:58.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:30:58.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:30:58.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:30:58.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:30:58.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:30:58.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:30:58.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:30:58.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:30:58.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:30:58.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:30:58.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:30:58.446 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.446 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.446 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.446 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:30:58.446 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:58.446 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 239.63579622200001, 2023-12-06T18:30:58.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:30:58.446 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.446 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.446 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:58.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:30:58.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:30:58.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:30:58.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:30:58.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:30:58.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.448 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:30:58.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:30:58.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.448 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:30:58.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:30:58.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.448 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:30:58.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:30:58.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:30:58.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:30:58.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:30:58.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:30:58.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:30:58.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:30:58.450 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.450 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.450 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.450 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:30:58.450 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:58.450 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 233.05588041499999, 2023-12-06T18:30:58.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:30:58.450 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.450 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.451 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:58.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:30:58.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:30:58.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:30:58.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:30:58.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:30:58.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:30:58.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:30:58.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:30:58.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:30:58.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:30:58.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:30:58.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:30:58.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:30:58.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:30:58.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:30:58.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:30:58.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:30:58.454 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.454 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.454 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.454 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:30:58.454 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:58.454 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 233.04964261000001, 2023-12-06T18:30:58.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:30:58.455 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.455 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.455 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:58.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:30:58.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:30:58.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:30:58.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:30:58.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:30:58.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:30:58.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:30:58.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:30:58.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:30:58.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:30:58.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:30:58.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:30:58.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:30:58.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:30:58.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:30:58.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:30:58.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:30:58.458 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.458 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.458 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.458 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:30:58.458 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:58.459 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 233.04357953100001, 2023-12-06T18:30:58.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:30:58.459 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.459 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.459 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:58.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:30:58.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:30:58.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:30:58.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:30:58.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:30:58.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:30:58.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:30:58.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:30:58.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:30:58.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:30:58.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:30:58.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:30:58.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:30:58.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:30:58.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:30:58.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:30:58.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:30:58.462 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.462 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.462 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.463 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:30:58.463 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:58.463 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 232.90619244600001, 2023-12-06T18:30:58.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:30:58.463 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.463 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.463 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:58.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:30:58.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:30:58.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:30:58.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:30:58.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:30:58.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:30:58.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:30:58.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:30:58.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:30:58.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:30:58.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:30:58.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:30:58.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:30:58.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:30:58.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:30:58.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:30:58.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:30:58.466 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.466 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.466 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.467 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:30:58.467 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:58.467 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 232.89952171900001, 2023-12-06T18:30:58.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:30:58.467 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:58.467 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:30:58.467 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:58.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:58.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:58.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:58.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:30:58.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:58.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:58.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:30:58.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:30:58.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:58.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:30:58.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:30:58.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:58.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:30:58.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:30:58.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:58.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:30:58.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:30:58.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:58.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:30:58.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:30:58.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:30:58.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:30:58.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:30:58.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.470 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:30:58.470 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:30:58.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:30:58.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:58.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:58.470 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:58.470 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:30:58.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:30:58.470 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.470 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.470 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.470 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:58.471 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:58.471 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:58.472 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:30:58.473 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:58.468+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:58.573 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:58.570+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:58.606 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:58.606 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:58.606 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:58.606 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:58.615 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:30:58.699 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:30:58.697+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:30:58.775 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:58.772+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:58.786 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:58.786 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:58.786 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:58.786 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:58.786 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:58.795 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:30:58.877 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:58.873+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:58.964 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:58.964 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:58.964 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:58.964 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:30:58.964 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:58.973 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:30:58.978 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:58.974+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:59.078 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:59.075+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:59.102 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:59.102 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:59.102 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:30:59.102 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:59.111 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:30:59.180 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:30:59.176+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:30:59.238 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:59.238 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:30:59.238 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:30:59.238 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:30:59.238 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:59.247 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:30:59.291 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:59.287+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:59.377 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:30:59.377 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:30:59.377 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:30:59.377 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:30:59.377 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:59.377 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:30:59.377 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:59.378 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 37.715258015000003, 2023-12-06T18:30:59.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:30:59.378 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:30:59.378 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:30:59.378 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:30:59.378 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:30:59.378 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:30:59.378 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:30:59.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:59.378 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:30:59.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:59.378 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:30:59.378 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:59.379 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:59.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:59.379 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:59.379 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:30:59.379 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:30:59.379 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:30:59.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:59.379 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:59.379 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:30:59.379 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:30:59.379 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:30:59.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:59.379 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:59.379 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:30:59.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:30:59.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:30:59.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:59.380 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:59.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:30:59.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:30:59.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:30:59.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:59.380 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:59.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:30:59.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:30:59.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:30:59.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:59.380 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:59.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:30:59.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:30:59.381 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:30:59.381 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:30:59.381 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:30:59.381 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:30:59.381 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:30:59.381 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:30:59.381 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:59.381 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:59.381 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:59.381 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:30:59.381 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:30:59.381 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:30:59.383 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:30:59.379+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:30:59.386 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:30:59.484 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:30:59.480+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:30:59.535 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:59.535 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:59.535 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:30:59.535 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:59.544 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:30:59.589 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:59.585+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:59.682 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:59.682 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:30:59.682 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:30:59.682 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:30:59.683 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:59.685 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:59.681+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:30:59.691 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:30:59.787 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:30:59.783+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:30:59.868 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:30:59.869 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:30:59.869 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:30:59.869 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:30:59.869 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:30:59.877 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:30:59.888 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:30:59.883+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:30:59.988 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:30:59.985+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:00.057 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:00.058 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:00.058 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:00.058 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:00.067 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:31:00.089 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:00.086+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:00.191 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:00.187+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:00.197 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-12-06T18:31:00.197 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi061 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a" 2023-12-06T18:31:00.198 INFO:tasks.ceph.mon.a:Stopped 2023-12-06T18:31:00.198 INFO:tasks.mon_recovery:forming a minimal quorum for ['b', 'c', 'a'], then adding monitors 2023-12-06T18:31:00.198 INFO:tasks.ceph.mon.b:Restarting daemon 2023-12-06T18:31:00.198 DEBUG:teuthology.orchestra.run.smithi152:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2023-12-06T18:31:00.200 INFO:tasks.ceph.mon.b:Started 2023-12-06T18:31:00.201 INFO:tasks.ceph.mon.c:Restarting daemon 2023-12-06T18:31:00.201 DEBUG:teuthology.orchestra.run.smithi061:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2023-12-06T18:31:00.203 INFO:tasks.ceph.mon.c:Started 2023-12-06T18:31:00.203 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2023-12-06T18:31:00.203 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:31:00.205 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:00.205 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:00.205 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:00.205 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:00.206 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:00.216 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:31:00.302 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:00.298+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:00.393 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:00.389+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:00.457 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:00.457 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:00.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:31:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 245.04745793399999, 2023-12-06T18:31:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:31:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:31:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.458 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:31:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:31:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.459 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:31:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:31:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:31:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:31:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:31:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:31:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:31:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:31:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:31:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:31:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:31:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:31:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:31:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:31:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:31:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:31:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:31:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:31:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:31:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:31:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:31:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:31:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:31:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:31:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:31:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 242.93888469500001, 2023-12-06T18:31:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:31:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:31:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:00.464 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:31:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:31:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:31:00.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:31:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:31:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:31:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:31:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:31:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:31:00.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:31:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:31:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:31:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:31:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:31:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:31:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:31:00.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:31:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:31:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:31:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:31:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:31:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:31:00.468 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 242.938139463, 2023-12-06T18:31:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:31:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:31:00.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:31:00.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:31:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:31:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:31:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:31:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:31:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:31:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:31:00.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:31:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:31:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:31:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:31:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:31:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:31:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:31:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:31:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:31:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:31:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:31:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:31:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:31:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:31:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 242.93745570799999, 2023-12-06T18:31:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:31:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.474 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:31:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:31:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:31:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:31:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:31:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:31:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:31:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:31:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:31:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:31:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:31:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:31:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:31:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:31:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:31:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:31:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:31:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:31:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:31:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:31:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:31:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:31:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:31:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:00.479 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 242.93188245300001, 2023-12-06T18:31:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:31:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:31:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:31:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:31:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:31:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:31:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:31:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:31:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:31:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:31:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:31:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:31:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:31:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:31:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:31:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:31:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:31:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:31:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:31:00.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:31:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:31:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:31:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:31:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:31:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.484 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 242.93120881300001, 2023-12-06T18:31:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:31:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:31:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:00.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:31:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:31:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:31:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:31:00.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:31:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:31:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:31:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:31:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:31:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:31:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:31:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:31:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:31:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:31:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:31:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:31:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:31:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:31:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:31:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:31:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:31:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:31:00.489 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 242.93054661400001, 2023-12-06T18:31:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:31:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.490 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:31:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:31:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:31:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:31:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:31:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:31:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:31:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:31:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:31:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:31:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:31:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:31:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:31:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:31:00.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:31:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:31:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:31:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:31:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:31:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:31:00.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:31:00.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:31:00.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:31:00.495 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.495 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.495 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.495 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:00.495 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:00.495 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 242.857837179, 2023-12-06T18:31:00.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:31:00.495 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.496 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.496 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.496 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.496 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.496 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:31:00.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.496 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.496 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:00.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.496 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:00.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:31:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:31:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:31:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:31:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:31:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:31:00.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:31:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:31:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:31:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:31:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:31:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:31:00.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:31:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:31:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:31:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:31:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:31:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:31:00.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:31:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:31:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:31:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:31:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:00.500 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 242.85715547000001, 2023-12-06T18:31:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:31:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:31:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:31:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:31:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:31:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:31:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:31:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:31:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:31:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:31:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:31:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:31:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.506 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:00.506 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:00.506 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 241.77835138200001, 2023-12-06T18:31:00.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:31:00.506 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.506 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.506 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.506 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.506 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.506 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:31:00.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.506 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:00.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:00.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:31:00.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:31:00.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:31:00.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:31:00.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:31:00.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:31:00.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:31:00.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:31:00.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:31:00.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:31:00.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:31:00.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:31:00.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:31:00.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:31:00.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.510 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.510 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:31:00.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:31:00.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.510 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.510 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:31:00.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:31:00.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.510 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.510 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:31:00.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:31:00.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:31:00.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:31:00.511 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.511 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.511 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.511 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:00.511 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:00.511 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 241.77693480799999, 2023-12-06T18:31:00.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:31:00.511 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.511 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.512 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.512 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.512 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.512 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:31:00.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.512 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:00.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:00.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:31:00.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:31:00.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:31:00.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:31:00.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:31:00.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:31:00.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:31:00.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:31:00.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:31:00.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:31:00.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:31:00.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:31:00.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:31:00.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:31:00.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:31:00.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:31:00.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:31:00.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:31:00.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:31:00.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:31:00.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:31:00.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:31:00.516 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.516 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.516 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.516 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:00.516 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:00.516 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 241.77584968299999, 2023-12-06T18:31:00.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:31:00.517 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.517 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.517 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.517 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.517 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.517 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:31:00.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.517 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:00.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:00.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:31:00.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:31:00.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:31:00.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:31:00.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:31:00.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:31:00.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:31:00.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:31:00.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:31:00.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:31:00.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:31:00.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:31:00.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:31:00.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:31:00.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:31:00.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:31:00.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:31:00.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:31:00.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:31:00.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:31:00.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:31:00.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:31:00.521 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.521 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.522 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.522 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:00.522 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:00.522 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 241.740700581, 2023-12-06T18:31:00.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:31:00.522 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.522 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.522 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.522 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.522 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.522 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:31:00.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.522 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:00.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:00.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:31:00.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:31:00.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:31:00.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:31:00.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:31:00.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:31:00.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:31:00.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:31:00.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:31:00.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:31:00.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:31:00.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:31:00.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:31:00.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:31:00.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:31:00.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:31:00.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:31:00.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:31:00.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:31:00.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:31:00.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:31:00.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:31:00.527 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.527 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.527 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.527 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:00.527 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:00.527 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 235.190970056, 2023-12-06T18:31:00.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:31:00.527 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.527 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.527 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.528 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.528 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.528 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:31:00.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.528 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:00.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:00.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:31:00.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:31:00.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:31:00.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:31:00.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:31:00.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:31:00.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:31:00.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:31:00.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:31:00.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:31:00.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:31:00.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:31:00.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:31:00.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:31:00.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:31:00.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:31:00.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:31:00.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:31:00.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:31:00.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:31:00.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:31:00.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:31:00.532 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.532 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.532 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.532 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:00.532 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:00.532 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 235.19016444799999, 2023-12-06T18:31:00.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:31:00.532 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.533 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.533 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.533 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.533 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.533 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:31:00.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.533 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:00.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:00.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:31:00.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:31:00.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:31:00.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:31:00.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:31:00.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:31:00.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:31:00.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:31:00.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:31:00.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:31:00.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:31:00.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:31:00.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:31:00.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:31:00.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:31:00.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:31:00.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:31:00.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:31:00.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:31:00.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:31:00.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:31:00.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:31:00.537 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.537 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.537 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.538 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:00.538 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:00.538 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 235.189473639, 2023-12-06T18:31:00.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:31:00.538 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.538 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.538 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.538 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.538 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.538 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:31:00.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.538 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:00.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:00.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:31:00.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:31:00.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:31:00.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:31:00.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:31:00.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:31:00.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:31:00.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:31:00.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:31:00.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:31:00.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:31:00.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:31:00.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:31:00.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:31:00.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:31:00.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:31:00.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:31:00.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:31:00.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:31:00.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:31:00.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:31:00.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:31:00.543 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.543 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.543 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.543 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:00.543 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:00.543 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 235.18517613099999, 2023-12-06T18:31:00.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:31:00.543 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.543 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.543 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.544 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.544 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.544 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:31:00.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.544 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:00.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:00.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:31:00.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:31:00.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:31:00.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:31:00.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:31:00.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:31:00.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:31:00.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:31:00.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:31:00.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:31:00.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:31:00.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:31:00.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:31:00.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:31:00.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:31:00.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:31:00.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:31:00.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:31:00.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:31:00.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:31:00.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:31:00.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:31:00.548 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.548 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.548 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.548 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:00.548 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:00.549 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 235.184508456, 2023-12-06T18:31:00.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:31:00.549 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.549 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.549 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.549 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.549 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.549 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:31:00.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.549 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:00.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:00.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:31:00.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:31:00.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:31:00.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:31:00.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:31:00.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:31:00.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:31:00.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:31:00.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:31:00.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:31:00.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:31:00.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:31:00.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:31:00.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:31:00.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:31:00.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:31:00.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:31:00.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:31:00.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:31:00.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:31:00.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:31:00.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:31:00.553 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.553 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.553 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.554 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:00.554 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:00.554 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 235.18385302799999, 2023-12-06T18:31:00.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:31:00.554 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.554 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.554 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.554 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.554 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.554 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:31:00.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.554 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:00.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:00.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:31:00.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:31:00.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.556 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:31:00.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:31:00.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.556 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.556 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:31:00.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:31:00.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.556 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.556 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:31:00.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:31:00.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.556 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.556 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:31:00.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:31:00.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:31:00.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:31:00.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:31:00.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:31:00.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:31:00.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:31:00.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:31:00.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:31:00.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:31:00.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:31:00.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:31:00.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:31:00.559 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.559 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.559 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.559 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:00.559 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:00.559 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 235.14319848100001, 2023-12-06T18:31:00.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:31:00.559 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:00.559 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:00.559 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:00.559 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:00.559 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:00.559 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:31:00.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.559 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:00.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:00.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:00.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:00.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:00.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:00.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:00.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:31:00.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:31:00.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:00.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:31:00.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:31:00.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:00.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:31:00.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:31:00.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:00.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:31:00.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:31:00.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:00.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:31:00.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:31:00.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:00.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:31:00.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:31:00.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:00.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:31:00.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:31:00.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:00.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:31:00.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:31:00.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:00.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:31:00.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:31:00.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:00.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:31:00.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:31:00.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:00.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:00.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:00.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:31:00.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:31:00.564 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.564 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.564 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.564 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:00.564 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:00.564 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:00.565 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:31:00.566 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:00.562+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:00.667 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:00.663+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:00.745 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:00.745 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:00.745 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:00.745 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:00.754 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:31:00.767 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:00.764+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:00.868 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:00.865+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:00.933 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:00.933 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:00.933 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:00.933 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:00.933 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:00.943 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:31:00.969 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:00.966+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:01.071 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:01.067+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:01.089 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:01.089 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:01.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:01.089 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:01.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.089 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:31:01.089 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:01.089 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 219.273226556, 2023-12-06T18:31:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:31:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:31:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:01.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:31:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:31:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:31:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:31:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:31:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:31:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:01.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:31:01.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:31:01.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:01.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:31:01.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:31:01.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:01.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:31:01.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:31:01.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.093 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.093 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:01.093 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:31:01.093 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:31:01.093 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:01.093 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:01.093 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:01.093 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:01.093 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:01.093 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:01.099 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:31:01.186 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:01.180+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:01.273 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:01.269+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:01.278 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:01.278 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:01.278 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:01.279 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:01.288 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:31:01.374 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:01.370+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:01.467 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:01.467 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:01.467 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:01.467 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:01.467 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:01.475 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:01.471+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:01.476 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:31:01.575 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:01.572+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:01.665 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:01.666 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:01.666 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:01.666 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:01.666 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.666 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:01.666 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:01.666 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 100.566188655, 2023-12-06T18:31:01.666 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:31:01.666 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:01.666 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:01.666 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:01.666 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:01.667 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:01.667 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:31:01.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.667 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:01.667 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.667 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:01.667 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:01.667 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:01.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.667 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.667 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:01.667 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:01.667 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:01.667 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.667 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.668 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:01.668 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:31:01.668 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:31:01.668 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.668 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:01.668 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:31:01.668 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:31:01.668 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.668 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:01.668 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:31:01.668 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:31:01.668 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.668 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.668 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:01.669 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:31:01.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:31:01.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.669 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:01.669 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:31:01.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:31:01.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.669 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:01.669 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:31:01.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:31:01.669 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:01.669 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:01.669 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:01.669 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.670 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:31:01.670 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:01.670 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 40.547931353000003, 2023-12-06T18:31:01.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:31:01.670 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:01.670 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:01.670 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:01.670 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:01.670 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:01.670 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:31:01.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.670 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:01.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:01.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:01.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:01.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:01.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:01.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:01.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:01.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:31:01.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:31:01.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:01.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:31:01.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:31:01.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:01.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:31:01.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:31:01.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:01.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:31:01.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:31:01.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:01.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:31:01.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:31:01.673 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:01.673 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:01.673 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:01.673 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:31:01.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:31:01.673 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:01.673 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:01.673 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:01.673 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:01.673 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:01.673 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:01.675 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:31:01.677 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:01.673+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:01.809 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:01.805+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:01.837 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:01.837 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:01.838 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:01.838 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:01.850 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:31:01.891 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:01.888+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:01.980 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:01.976+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:02.001 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-12-06T18:31:02.021 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:02.021 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:02.021 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:02.021 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:02.021 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:02.031 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:31:02.081 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:02.077+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:02.181 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:02.178+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:02.209 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:02.210 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:02.210 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:02.210 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:02.210 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.210 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:02.210 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:02.210 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 160.71629131899999, 2023-12-06T18:31:02.210 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:31:02.210 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.210 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:02.210 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:02.210 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:02.210 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:02.211 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:31:02.211 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.211 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.211 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.211 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.211 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:02.211 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.211 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.211 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.211 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.211 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:02.211 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.211 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.211 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.211 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.211 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:31:02.212 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:31:02.212 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.212 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.212 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.212 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:31:02.212 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:31:02.212 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.212 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.212 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.212 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:31:02.212 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:31:02.212 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.212 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.212 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.212 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:31:02.213 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:31:02.213 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.213 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.213 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.213 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:31:02.213 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:31:02.213 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.213 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.213 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.213 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:31:02.213 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:31:02.213 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.213 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.213 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.213 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.213 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.214 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:02.219 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:31:02.282 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:02.279+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:02.351 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:02.351 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:02.351 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:02.351 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:02.360 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:31:02.383 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:02.379+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:02.485 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:02.481+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:02.531 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:02.531 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:02.531 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:02.531 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:02.531 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:02.540 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:31:02.585 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:02.581+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:02.687 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:02.683+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:02.715 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:02.715 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:02.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:02.715 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:02.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.715 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:31:02.715 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:02.715 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 246.097819961, 2023-12-06T18:31:02.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:31:02.715 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.715 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.716 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:02.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:02.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:31:02.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:31:02.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:31:02.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:31:02.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:31:02.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:31:02.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:31:02.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:31:02.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.718 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.718 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:31:02.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:31:02.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.718 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.718 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:31:02.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:31:02.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.718 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.718 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:31:02.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:31:02.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:31:02.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:31:02.719 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.719 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.719 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.719 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:31:02.719 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:02.719 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 245.074992952, 2023-12-06T18:31:02.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:31:02.719 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.719 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.720 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:02.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:02.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:31:02.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:31:02.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:31:02.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:31:02.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:31:02.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:31:02.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:31:02.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:31:02.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:31:02.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:31:02.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:31:02.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:31:02.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:31:02.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:31:02.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.723 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:31:02.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:31:02.723 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.723 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.723 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.723 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:31:02.723 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:02.723 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 245.06294631399999, 2023-12-06T18:31:02.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:31:02.724 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.724 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.724 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:02.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:02.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:31:02.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:31:02.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:31:02.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:31:02.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:31:02.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:31:02.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:31:02.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:31:02.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:31:02.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:31:02.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:31:02.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:31:02.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.727 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:31:02.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:31:02.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.727 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.727 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:31:02.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:31:02.727 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.727 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.727 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.727 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:31:02.727 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:02.727 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 245.04154570700001, 2023-12-06T18:31:02.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:31:02.728 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.728 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.728 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:02.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:02.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:31:02.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:31:02.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:31:02.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:31:02.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:31:02.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:31:02.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:31:02.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:31:02.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:31:02.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:31:02.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:31:02.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:31:02.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:31:02.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:31:02.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:31:02.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:31:02.731 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.731 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.731 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.731 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:31:02.732 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:02.732 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 245.04036638400001, 2023-12-06T18:31:02.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:31:02.732 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.732 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.732 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:02.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:02.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:31:02.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:31:02.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:31:02.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:31:02.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:31:02.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:31:02.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:31:02.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:31:02.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:31:02.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:31:02.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:31:02.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:31:02.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:31:02.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:31:02.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:31:02.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:31:02.735 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.735 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.735 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.735 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:31:02.736 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:02.736 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 245.039794403, 2023-12-06T18:31:02.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:31:02.736 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.736 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.736 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:02.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:02.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:31:02.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:31:02.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:31:02.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:31:02.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:31:02.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:31:02.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:31:02.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:31:02.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:31:02.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:31:02.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:31:02.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:31:02.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:31:02.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:31:02.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:31:02.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:31:02.739 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.739 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.739 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.739 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:31:02.739 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:02.740 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 244.59395893300001, 2023-12-06T18:31:02.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:31:02.740 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.740 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.740 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:02.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:02.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:31:02.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:31:02.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:31:02.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:31:02.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:31:02.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:31:02.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:31:02.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:31:02.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:31:02.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:31:02.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:31:02.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:31:02.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:31:02.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:31:02.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:31:02.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:31:02.743 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.743 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.743 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.743 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:31:02.744 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:02.744 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 244.592359077, 2023-12-06T18:31:02.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:31:02.744 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.744 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.744 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:02.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:02.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:31:02.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:31:02.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:31:02.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:31:02.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:31:02.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:31:02.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:31:02.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:31:02.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:31:02.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:31:02.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:31:02.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:31:02.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:31:02.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:31:02.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:31:02.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:31:02.747 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.747 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.747 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.748 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:31:02.748 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:02.748 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 244.21315346099999, 2023-12-06T18:31:02.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:31:02.748 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.748 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.748 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:02.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:02.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:31:02.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:31:02.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:31:02.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:31:02.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:31:02.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:31:02.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.750 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:31:02.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:31:02.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.750 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:31:02.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:31:02.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.750 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:31:02.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:31:02.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:31:02.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:31:02.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:31:02.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:31:02.751 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.751 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.751 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.752 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:31:02.752 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:02.752 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 243.975651279, 2023-12-06T18:31:02.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:31:02.752 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.752 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.752 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:02.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:02.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:31:02.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:31:02.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:31:02.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:31:02.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:31:02.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:31:02.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:31:02.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:31:02.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:31:02.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:31:02.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:31:02.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:31:02.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:31:02.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:31:02.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:31:02.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:31:02.755 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.755 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.756 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.756 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:31:02.756 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:02.756 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 243.97492328000001, 2023-12-06T18:31:02.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:31:02.756 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.756 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.756 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:02.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:02.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:31:02.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:31:02.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:31:02.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:31:02.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:31:02.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:31:02.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:31:02.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:31:02.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:31:02.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:31:02.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:31:02.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:31:02.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:31:02.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:31:02.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:31:02.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:31:02.759 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.760 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.760 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.760 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:31:02.760 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:02.760 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 243.96991087000001, 2023-12-06T18:31:02.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:31:02.760 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.760 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.760 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.760 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:02.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:02.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:31:02.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:31:02.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:31:02.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:31:02.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:31:02.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:31:02.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:31:02.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:31:02.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:31:02.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:31:02.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:31:02.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:31:02.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:31:02.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:31:02.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:31:02.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:31:02.763 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.764 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.764 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.764 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:31:02.764 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:02.764 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 243.96438379099999, 2023-12-06T18:31:02.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:31:02.764 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.764 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.764 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:02.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:02.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:31:02.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:31:02.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:31:02.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:31:02.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:31:02.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:31:02.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:31:02.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:31:02.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:31:02.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:31:02.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:31:02.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:31:02.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:31:02.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:31:02.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:31:02.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:31:02.768 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.768 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.768 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.768 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:31:02.768 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:02.768 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 243.96387295400001, 2023-12-06T18:31:02.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:31:02.768 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.768 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.768 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.768 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:02.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:02.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:31:02.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:31:02.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:31:02.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:31:02.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:31:02.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:31:02.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:31:02.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:31:02.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:31:02.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:31:02.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:31:02.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:31:02.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:31:02.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:31:02.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:31:02.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:31:02.772 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.772 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.772 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.772 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:31:02.772 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:02.772 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 243.96341369300001, 2023-12-06T18:31:02.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:31:02.772 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.773 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.773 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:02.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:02.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:31:02.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:31:02.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:31:02.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:31:02.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:31:02.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:31:02.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:31:02.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:31:02.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:31:02.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:31:02.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:31:02.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:31:02.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:31:02.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:31:02.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:31:02.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:31:02.776 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.776 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.776 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.776 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:31:02.776 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:02.777 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 237.38349788599999, 2023-12-06T18:31:02.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:31:02.777 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.777 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.777 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:02.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:02.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:31:02.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:31:02.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:31:02.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:31:02.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:31:02.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:31:02.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:31:02.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:31:02.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:31:02.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:31:02.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:31:02.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:31:02.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:31:02.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:31:02.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:31:02.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:31:02.780 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.780 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.780 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.781 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:31:02.781 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:02.781 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 237.377260081, 2023-12-06T18:31:02.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:31:02.781 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.781 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.781 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:02.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:02.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:31:02.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:31:02.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:31:02.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:31:02.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:31:02.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:31:02.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:31:02.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:31:02.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:31:02.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:31:02.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:31:02.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:31:02.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:31:02.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:31:02.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:31:02.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:31:02.784 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.784 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.784 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.784 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:31:02.785 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:02.785 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 237.371197002, 2023-12-06T18:31:02.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:31:02.785 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.785 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.785 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:02.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:02.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:31:02.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:31:02.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:31:02.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:31:02.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:31:02.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:31:02.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:31:02.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:31:02.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:31:02.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:31:02.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:31:02.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:31:02.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:31:02.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:31:02.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:31:02.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:31:02.788 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.788 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.788 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.788 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:31:02.788 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:02.789 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 237.233809917, 2023-12-06T18:31:02.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:31:02.789 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.789 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.789 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:02.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:02.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:31:02.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:31:02.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:31:02.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:31:02.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:31:02.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:31:02.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:31:02.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:31:02.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:31:02.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:31:02.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:31:02.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:31:02.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:31:02.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:31:02.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:31:02.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:31:02.792 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.792 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.792 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.792 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:31:02.792 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:02.792 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 237.22713919, 2023-12-06T18:31:02.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:31:02.793 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:02.793 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:02.793 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:02.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:02.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:02.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:02.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:02.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:02.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:02.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:31:02.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:31:02.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:02.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:31:02.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:31:02.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:02.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:31:02.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:31:02.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:02.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:31:02.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:31:02.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:02.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:31:02.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:31:02.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:02.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:31:02.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:31:02.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:02.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:31:02.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:31:02.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:02.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:02.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:02.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:31:02.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:31:02.796 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.796 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.796 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.796 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:02.796 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:02.797 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:02.798 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:31:02.799 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:02.795+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:02.899 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:02.895+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:02.931 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:02.931 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:02.931 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:02.931 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:02.940 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:31:03.000 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:02.997+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:03.070 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:03.070 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:03.070 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:03.071 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:03.071 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:03.080 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:31:03.124 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:03.120+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:03.202 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:03.199+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:03.209 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:03.209 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:03.209 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:03.209 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:31:03.209 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:03.218 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:31:03.304 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:03.300+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:03.400 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:03.401 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:03.401 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:03.401 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:03.404 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:03.400+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:03.410 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:31:03.506 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:03.502+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:03.538 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:03.538 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:03.538 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:03.538 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:03.538 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:03.547 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:31:03.606 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:03.602+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:03.678 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:03.678 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:03.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:03.678 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:03.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:03.678 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:31:03.679 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:03.679 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 42.016303841999999, 2023-12-06T18:31:03.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:31:03.679 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:03.679 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:03.679 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:03.679 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:03.679 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:03.679 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:31:03.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:03.679 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:03.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:03.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:03.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:03.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:03.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:03.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:03.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:03.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:03.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:03.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:03.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:03.681 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:03.681 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:31:03.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:31:03.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:03.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:03.681 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:03.681 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:31:03.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:31:03.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:03.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:03.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:03.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:31:03.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:31:03.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:03.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:03.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:03.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:31:03.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:31:03.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:03.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:03.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:03.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:31:03.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:31:03.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:03.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:03.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:03.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:31:03.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:31:03.684 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:03.684 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:03.684 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:03.684 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:03.685 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:03.685 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:03.687 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:31:03.706 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:03.703+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:03.807 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:03.804+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:03.824 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:03.825 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:03.825 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:03.825 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:03.835 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:31:03.909 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:03.905+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:03.976 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:03.976 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:03.976 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:03.976 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:03.976 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:03.986 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:31:04.031 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:04.027+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:04.111 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:04.107+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:04.123 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:04.123 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:04.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:04.123 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:31:04.123 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:04.132 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:31:04.211 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:04.208+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:04.310 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:04.310 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:04.310 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:04.310 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:04.313 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:04.309+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:04.319 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:31:04.414 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:04.410+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:04.460 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:04.460 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:04.460 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:04.460 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:04.460 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:04.469 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:31:04.555 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:04.551+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:04.615 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:04.611+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:04.713 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:04.713 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:04.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:04.713 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:04.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.713 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:31:04.713 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:04.714 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 249.30317649400001, 2023-12-06T18:31:04.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:31:04.714 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.714 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.714 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.714 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.714 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.714 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:31:04.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.714 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:04.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:04.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:31:04.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:31:04.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:31:04.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:31:04.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:31:04.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:31:04.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:31:04.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:31:04.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:31:04.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:31:04.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:31:04.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:31:04.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:31:04.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:31:04.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:31:04.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:31:04.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:31:04.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:31:04.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:31:04.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:31:04.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:31:04.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:31:04.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:31:04.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:31:04.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:31:04.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:31:04.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:31:04.720 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.721 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.721 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.721 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:04.721 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:04.721 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 247.194603255, 2023-12-06T18:31:04.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:31:04.721 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.721 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.721 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.721 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.721 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.721 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:31:04.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.722 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:04.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:04.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:31:04.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:31:04.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:31:04.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:31:04.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:31:04.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:31:04.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:31:04.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:31:04.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:31:04.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:31:04.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:31:04.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:31:04.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:31:04.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:31:04.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.726 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:31:04.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:31:04.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.726 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:31:04.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:31:04.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:31:04.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:31:04.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:31:04.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:31:04.727 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.727 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.727 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.727 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:04.728 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:04.728 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 247.19385802299999, 2023-12-06T18:31:04.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:31:04.728 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.728 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.728 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.728 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.728 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.728 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:31:04.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.728 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:04.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:04.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:31:04.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:31:04.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:31:04.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:31:04.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:31:04.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:31:04.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:31:04.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:31:04.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:31:04.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:31:04.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:31:04.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:31:04.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:31:04.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:31:04.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:31:04.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:31:04.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:31:04.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:31:04.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:31:04.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:31:04.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:31:04.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:31:04.734 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.734 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.734 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.734 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:04.734 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:04.734 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 247.19317426800001, 2023-12-06T18:31:04.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:31:04.734 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.734 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.735 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.735 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.735 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.735 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:31:04.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.735 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:04.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:04.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:31:04.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:31:04.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:31:04.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:31:04.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:31:04.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:31:04.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:31:04.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:31:04.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:31:04.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:31:04.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:31:04.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:31:04.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:31:04.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:31:04.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:31:04.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:31:04.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:31:04.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:31:04.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:31:04.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:31:04.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:31:04.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:31:04.740 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.740 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.740 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.740 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:04.741 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:04.741 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 247.18760101300001, 2023-12-06T18:31:04.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:31:04.741 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.741 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.741 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.741 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.741 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.741 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:31:04.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.741 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:04.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:04.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:31:04.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:31:04.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:31:04.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:31:04.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:31:04.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:31:04.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:31:04.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:31:04.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:31:04.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:31:04.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:31:04.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:31:04.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:31:04.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:31:04.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:31:04.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:31:04.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:31:04.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:31:04.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:31:04.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:31:04.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:31:04.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:31:04.747 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.747 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.747 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.747 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:04.747 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:04.747 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 247.186927373, 2023-12-06T18:31:04.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:31:04.747 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.748 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.748 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.748 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.748 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.749 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:31:04.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.749 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.749 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.749 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.749 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:04.749 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.749 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.749 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.749 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:04.750 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.750 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.750 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.750 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:31:04.750 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:31:04.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.750 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.750 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.750 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:31:04.750 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:31:04.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.751 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.751 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:31:04.751 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:31:04.751 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.751 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.751 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:31:04.751 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:31:04.751 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.751 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:31:04.752 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:31:04.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:31:04.752 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:31:04.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:31:04.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:31:04.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.753 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.753 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.753 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:31:04.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:31:04.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.753 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.753 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.753 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:31:04.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:31:04.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.754 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.754 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.754 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:31:04.754 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:31:04.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.754 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.754 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.754 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:31:04.754 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:31:04.754 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.754 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.754 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.755 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.755 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:04.755 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:04.755 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 247.186265174, 2023-12-06T18:31:04.755 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:31:04.755 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.755 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.755 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.755 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.755 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.755 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:31:04.755 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.756 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.756 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.756 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.756 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:04.756 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.756 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.756 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.756 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:04.756 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:31:04.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:31:04.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:31:04.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:31:04.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.758 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.758 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:31:04.758 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:31:04.758 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.758 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.758 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.758 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:31:04.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:31:04.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:31:04.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:31:04.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:31:04.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:31:04.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.760 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.760 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.760 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:31:04.760 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:31:04.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.760 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.760 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.760 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:31:04.760 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:31:04.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.760 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.760 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.761 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:31:04.761 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:31:04.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.761 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.761 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.761 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:31:04.761 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:31:04.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.761 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.761 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.761 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:31:04.762 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:31:04.762 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.762 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.762 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.762 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:04.762 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:04.762 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 247.11355573899999, 2023-12-06T18:31:04.762 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:31:04.762 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.762 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.763 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.763 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.763 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.763 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:31:04.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.763 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.763 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:04.763 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.764 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:04.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.764 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:31:04.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:31:04.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.764 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:31:04.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:31:04.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.765 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:31:04.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:31:04.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.765 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:31:04.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:31:04.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.766 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.766 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:31:04.766 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:31:04.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.766 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.766 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.766 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:31:04.766 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:31:04.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.766 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.766 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.766 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:31:04.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:31:04.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.767 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:31:04.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:31:04.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.767 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:31:04.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:31:04.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:31:04.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:31:04.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:31:04.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:31:04.768 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.768 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.768 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.769 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:04.769 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:04.769 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 247.11287403, 2023-12-06T18:31:04.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:31:04.769 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.769 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.769 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.769 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.769 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.769 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:31:04.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.769 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:04.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:04.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:31:04.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:31:04.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:31:04.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:31:04.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:31:04.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:31:04.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.772 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:31:04.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:31:04.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.772 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:31:04.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:31:04.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.772 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:31:04.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:31:04.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:31:04.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:31:04.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:31:04.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:31:04.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:31:04.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:31:04.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:31:04.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:31:04.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:31:04.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:31:04.775 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.775 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.775 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.775 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:04.775 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:04.775 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 246.034069942, 2023-12-06T18:31:04.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:31:04.775 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.775 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.775 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.776 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.776 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.776 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:31:04.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.776 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.776 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:04.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.776 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:04.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:31:04.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:31:04.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:31:04.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:31:04.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:31:04.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:31:04.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:31:04.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:31:04.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:31:04.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:31:04.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:31:04.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:31:04.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:31:04.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:31:04.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:31:04.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:31:04.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:31:04.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:31:04.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:31:04.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:31:04.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:31:04.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:31:04.781 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.781 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.781 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.781 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:04.782 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:04.782 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 246.03265336800001, 2023-12-06T18:31:04.782 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:31:04.782 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.782 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.782 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.782 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.782 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.782 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:31:04.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.782 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:04.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:04.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:31:04.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:31:04.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:31:04.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:31:04.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:31:04.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:31:04.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.785 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:31:04.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:31:04.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.785 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:31:04.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:31:04.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:31:04.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:31:04.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:31:04.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:31:04.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:31:04.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:31:04.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.787 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.787 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:31:04.787 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:31:04.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.787 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.787 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:31:04.787 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:31:04.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.787 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.788 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:31:04.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:31:04.788 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.788 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.788 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.788 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:04.788 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:04.788 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 246.03156824300001, 2023-12-06T18:31:04.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:31:04.788 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.788 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.789 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.789 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.789 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.789 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:31:04.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.789 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:04.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.790 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.790 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:04.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.790 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.790 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:31:04.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:31:04.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:31:04.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:31:04.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:31:04.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:31:04.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:31:04.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:31:04.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.792 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.792 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:31:04.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:31:04.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.792 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.792 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:31:04.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:31:04.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:31:04.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:31:04.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:31:04.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:31:04.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:31:04.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:31:04.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:31:04.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:31:04.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:31:04.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:31:04.794 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.794 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.795 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.795 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:04.795 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:04.795 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 245.99641914099999, 2023-12-06T18:31:04.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:31:04.795 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.795 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.795 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.795 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.795 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.796 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:31:04.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.796 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:04.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:04.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:31:04.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:31:04.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:31:04.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:31:04.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:31:04.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:31:04.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:31:04.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:31:04.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:31:04.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:31:04.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:31:04.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:31:04.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:31:04.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:31:04.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:31:04.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:31:04.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:31:04.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:31:04.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:31:04.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:31:04.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:31:04.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:31:04.801 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.801 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.801 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.801 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:04.801 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:04.801 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 239.44668861599999, 2023-12-06T18:31:04.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:31:04.802 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.802 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.802 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.802 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.802 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.802 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:31:04.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.802 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.802 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:04.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:04.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:31:04.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:31:04.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:31:04.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:31:04.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:31:04.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:31:04.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:31:04.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:31:04.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:31:04.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:31:04.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:31:04.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:31:04.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:31:04.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:31:04.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:31:04.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:31:04.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:31:04.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:31:04.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:31:04.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:31:04.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:31:04.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:31:04.807 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.807 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.807 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.808 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:04.808 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:04.808 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 239.44588300800001, 2023-12-06T18:31:04.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:31:04.808 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.808 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.808 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.808 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.808 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.808 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:31:04.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.809 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:04.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:04.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:31:04.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:31:04.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:31:04.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:31:04.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:31:04.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:31:04.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:31:04.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:31:04.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:31:04.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:31:04.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:31:04.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:31:04.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:31:04.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:31:04.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:31:04.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:31:04.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:31:04.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:31:04.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:31:04.814 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:31:04.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.814 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.814 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:31:04.814 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:31:04.814 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.814 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.814 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.815 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:04.815 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:04.815 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 239.44519219899999, 2023-12-06T18:31:04.815 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:31:04.815 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.815 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.815 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.815 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.815 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.815 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:31:04.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.816 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:04.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:04.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:31:04.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:31:04.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:31:04.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:31:04.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:31:04.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:31:04.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:31:04.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:31:04.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:31:04.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:31:04.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.819 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:31:04.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:31:04.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.819 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.819 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.819 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:31:04.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:31:04.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.819 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.819 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.819 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:31:04.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:31:04.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.820 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.820 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.820 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:31:04.820 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:31:04.820 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.820 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.820 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.820 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:31:04.820 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:31:04.820 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.820 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.820 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.821 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:31:04.821 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:31:04.821 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.821 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.821 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.821 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.821 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.821 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:04.821 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:04.821 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 239.44089469100001, 2023-12-06T18:31:04.821 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:31:04.821 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.821 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.821 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.822 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.822 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.822 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:31:04.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.822 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.822 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.822 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.822 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:04.822 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.822 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.822 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.822 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:04.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:31:04.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:31:04.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:31:04.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:31:04.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.824 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:31:04.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:31:04.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.824 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:31:04.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:31:04.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.824 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:31:04.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:31:04.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.825 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.825 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.825 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:31:04.825 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:31:04.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.825 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.825 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.825 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:31:04.825 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:31:04.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.825 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.825 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.825 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:31:04.826 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:31:04.826 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.826 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.826 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.826 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:31:04.826 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:31:04.826 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.826 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.826 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.826 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:31:04.826 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:31:04.826 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.826 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.827 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.827 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:31:04.827 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:31:04.827 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.827 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.827 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.827 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.827 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.827 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:04.827 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:04.827 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 239.44022701599999, 2023-12-06T18:31:04.827 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:31:04.827 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.828 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.828 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.828 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.828 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.828 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:31:04.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.828 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.828 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.828 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:04.828 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.829 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:04.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.829 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.829 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:31:04.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:31:04.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.829 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.829 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:31:04.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:31:04.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.830 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.830 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.830 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:31:04.830 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:31:04.830 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.830 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.830 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.830 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:31:04.830 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:31:04.830 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.830 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.831 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.831 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:31:04.831 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:31:04.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.831 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.831 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.831 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:31:04.831 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:31:04.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.831 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.831 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.831 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:31:04.831 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:31:04.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.832 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:31:04.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:31:04.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.832 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:31:04.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:31:04.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.832 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:31:04.833 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:31:04.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.833 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.833 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.833 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:31:04.833 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:31:04.833 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.833 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.833 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.833 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.833 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:04.834 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:04.834 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 239.43957158800001, 2023-12-06T18:31:04.834 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:31:04.834 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.834 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.834 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.834 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.834 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.834 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:31:04.834 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.834 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.834 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.835 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.835 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:04.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.835 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.835 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.835 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:04.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.835 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.835 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.835 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:31:04.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:31:04.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.836 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.836 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.836 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:31:04.836 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:31:04.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.836 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.836 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.836 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:31:04.836 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:31:04.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.836 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.836 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.837 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:31:04.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:31:04.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.837 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.837 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:31:04.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:31:04.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.837 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.837 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:31:04.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:31:04.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:31:04.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:31:04.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:31:04.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:31:04.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.839 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:31:04.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:31:04.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.839 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:31:04.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:31:04.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.839 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:31:04.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:31:04.840 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.840 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.840 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.840 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:04.840 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:04.840 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 239.398917041, 2023-12-06T18:31:04.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:31:04.841 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:04.841 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:04.841 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:04.841 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:04.841 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:04.841 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:31:04.841 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.841 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:04.841 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.841 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:04.841 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:04.841 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.842 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.842 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:04.842 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:04.842 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:04.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.842 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.842 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:04.842 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:31:04.842 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:31:04.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.842 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.842 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:04.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:31:04.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:31:04.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:04.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:31:04.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:31:04.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:04.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:31:04.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:31:04.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:04.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:31:04.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:31:04.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:04.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:31:04.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:31:04.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:04.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:31:04.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:31:04.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.845 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:04.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:31:04.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:31:04.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.845 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:04.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:31:04.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:31:04.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.845 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:04.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:31:04.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:31:04.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:04.846 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:04.846 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:04.846 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:31:04.846 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:31:04.846 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.846 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.846 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.846 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:04.846 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:04.846 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:04.848 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:31:04.849 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:04.844+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:04.949 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:04.945+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:05.025 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:05.025 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:05.025 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:05.025 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:05.034 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:31:05.049 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:05.046+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:05.150 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:05.146+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:05.214 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:05.214 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:05.214 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:05.214 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:05.214 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:05.223 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:31:05.267 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:05.263+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:05.352 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:05.348+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:05.361 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:31:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 223.54515612399999, 2023-12-06T18:31:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:31:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:31:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:31:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:31:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:31:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:31:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:31:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:31:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:31:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:31:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:31:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:31:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:31:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:31:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:31:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:31:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:05.365 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:05.370 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:31:05.455 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:05.451+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:05.548 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:05.548 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:05.548 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:05.548 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:05.554 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:05.550+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:05.558 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:31:05.654 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:05.651+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:05.736 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:05.736 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:05.736 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:05.736 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:05.736 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:05.745 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:31:05.787 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:05.783+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:05.856 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:05.853+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:05.929 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:05.930 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:05.930 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 104.82964997400001, 2023-12-06T18:31:05.930 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:31:05.930 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:05.930 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:05.930 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:05.930 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:05.930 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:05.930 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:31:05.930 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.930 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:05.930 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.931 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:05.931 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:05.931 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:05.931 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.931 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.931 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:05.931 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:05.931 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:05.931 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.931 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.931 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:05.931 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:31:05.931 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:31:05.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.932 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:05.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:31:05.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:31:05.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.932 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:05.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:31:05.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:31:05.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.932 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:05.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:31:05.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:31:05.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.933 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:05.933 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:31:05.933 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:31:05.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.933 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:05.933 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:31:05.933 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:31:05.933 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:05.933 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:05.934 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:05.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.934 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.934 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:31:05.934 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:05.934 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 44.811392671999997, 2023-12-06T18:31:05.934 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:31:05.934 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:05.934 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:05.934 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:05.934 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:05.934 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:05.934 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:31:05.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.935 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:05.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.935 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:05.935 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:05.935 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:05.935 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.935 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:05.935 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:05.935 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:05.935 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.936 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:05.936 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:31:05.936 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:31:05.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.936 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:05.936 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:31:05.936 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:31:05.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.936 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:05.936 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:31:05.936 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:31:05.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.937 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:05.937 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:31:05.937 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:31:05.937 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.937 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.937 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:05.937 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:31:05.937 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:31:05.937 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:05.937 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:05.937 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:05.937 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:31:05.938 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:31:05.938 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:05.938 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:05.938 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:05.938 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:05.938 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:05.938 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:05.939 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:31:05.957 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:05.953+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:06.059 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:06.055+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:06.068 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:06.069 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:06.069 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:06.069 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:06.078 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:31:06.160 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:06.156+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:06.208 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:06.208 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:06.208 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:06.208 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:06.208 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:06.217 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:31:06.261 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:06.257+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:06.345 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:06.346 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:06.346 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:06.346 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:06.346 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.346 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:06.346 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:06.346 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 164.852361519, 2023-12-06T18:31:06.346 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:31:06.346 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.346 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:06.346 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:06.346 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:06.347 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:06.347 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:31:06.347 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.347 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.347 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.347 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.347 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:06.347 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.347 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.347 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.347 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.347 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:06.347 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.347 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.347 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.348 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.348 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:31:06.348 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:31:06.348 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.348 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.348 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.348 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:31:06.348 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:31:06.348 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.348 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.348 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.348 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:31:06.348 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:31:06.348 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.349 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.349 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.349 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:31:06.349 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:31:06.349 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.349 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.349 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.349 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:31:06.349 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:31:06.349 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.349 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.349 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.349 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:31:06.349 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:31:06.350 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.350 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.350 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.350 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.350 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.350 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:06.355 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:31:06.361 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:06.357+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:06.462 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:06.458+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:06.484 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:06.484 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:06.484 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:06.484 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:06.493 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:31:06.563 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:06.559+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:06.625 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:06.625 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:06.625 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:06.625 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:06.625 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:06.635 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:31:06.664 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:06.661+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:06.765 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:06.761+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:06.804 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:06.805 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:06.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:06.805 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:06.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.805 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:31:06.805 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:06.805 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 250.18869209100001, 2023-12-06T18:31:06.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:31:06.805 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.805 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.805 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:06.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:06.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:31:06.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:31:06.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:31:06.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:31:06.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:31:06.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:31:06.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:31:06.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:31:06.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:31:06.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:31:06.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:31:06.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:31:06.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:31:06.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:31:06.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:31:06.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:31:06.809 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.809 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.810 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.810 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:31:06.810 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:06.810 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 249.16586508200001, 2023-12-06T18:31:06.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:31:06.810 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.810 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.810 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:06.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:06.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.812 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:31:06.812 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:31:06.812 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.812 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.812 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.812 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:31:06.812 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:31:06.813 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.813 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.813 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.813 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:31:06.813 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:31:06.813 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.814 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.814 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.814 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:31:06.814 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:31:06.814 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.814 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.814 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.815 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:31:06.815 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:31:06.815 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.816 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.816 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.816 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:31:06.816 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:31:06.816 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.817 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.817 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.817 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:31:06.817 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:31:06.817 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.817 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.817 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.817 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:31:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:31:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:31:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:06.818 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 249.153818444, 2023-12-06T18:31:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:31:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:06.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.820 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:31:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:31:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:31:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:31:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:31:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:31:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:31:06.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:31:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:31:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:31:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:31:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:31:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.822 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:31:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:31:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:31:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:31:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:31:06.823 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 249.13241783699999, 2023-12-06T18:31:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:31:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.824 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:31:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:31:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:31:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:31:06.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:31:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:31:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:31:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:31:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:31:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:31:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:31:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:31:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:31:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:31:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:31:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:31:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:31:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 249.13123851399999, 2023-12-06T18:31:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:31:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.828 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:31:06.829 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:31:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:31:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:31:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:31:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:31:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.830 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:31:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:31:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:31:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:31:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:31:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:31:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.831 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:31:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:31:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:31:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:31:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:31:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:06.832 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 249.13066653300001, 2023-12-06T18:31:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:31:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.833 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:31:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:31:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:31:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:31:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:31:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:31:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.834 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:31:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:31:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:31:06.835 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:31:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:31:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:31:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.836 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:31:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:31:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:31:06.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:31:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.838 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:31:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 248.68483106299999, 2023-12-06T18:31:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:31:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.839 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:06.840 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:31:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:31:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.841 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:31:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:31:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:31:06.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:31:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:31:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:31:06.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:31:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:31:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:31:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:31:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:31:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:31:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:31:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:31:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.845 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:31:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 248.68323120700001, 2023-12-06T18:31:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:31:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.846 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:31:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:31:06.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:31:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:31:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:31:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:31:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:31:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:31:06.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:31:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:31:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:31:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:31:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:31:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:31:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:31:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:31:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:31:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 248.304025591, 2023-12-06T18:31:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:31:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.850 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:31:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:31:06.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:31:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:31:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:31:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:31:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:31:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:31:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:31:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:31:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:31:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:31:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:31:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:31:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:31:06.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:31:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:31:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 248.06652340900001, 2023-12-06T18:31:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:31:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.854 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:31:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:31:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:31:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:31:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:31:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:31:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:31:06.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:31:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:31:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:31:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:31:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:31:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:31:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:31:06.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:31:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:31:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:31:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 248.06579540999999, 2023-12-06T18:31:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:31:06.858 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:31:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:31:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:31:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:31:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:31:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:31:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:31:06.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:31:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:31:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:31:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:31:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:31:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:31:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:31:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:31:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:31:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.862 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:31:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 248.06078299999999, 2023-12-06T18:31:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:31:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:31:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:31:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:31:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:31:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:31:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:31:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:31:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:31:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:31:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:31:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:31:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:31:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:31:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:31:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:31:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:31:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.866 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:31:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 248.055255921, 2023-12-06T18:31:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:31:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:06.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:31:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:31:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:31:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:31:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:31:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:31:06.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:31:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:31:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:31:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:31:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:31:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:31:06.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:31:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:31:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:31:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:31:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:31:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 248.05474508399999, 2023-12-06T18:31:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:31:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.871 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:31:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:31:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:31:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:31:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:31:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:31:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:31:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:31:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:31:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:31:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:31:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:31:06.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:31:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:31:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:31:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:31:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.877 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:31:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 248.05428582299999, 2023-12-06T18:31:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:31:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.878 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:31:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:31:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:31:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:31:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:31:06.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:31:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:31:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:31:06.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:31:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:31:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:31:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:31:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:31:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:31:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.884 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:31:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:31:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:31:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 241.47437001599999, 2023-12-06T18:31:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:31:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:31:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:31:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:31:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:31:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:31:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:31:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:31:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:31:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:31:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:31:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:31:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:31:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:31:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:31:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:31:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:31:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:31:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 241.46813221100001, 2023-12-06T18:31:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:31:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:31:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:31:06.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:31:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:31:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:31:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:31:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:31:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:31:06.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:31:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:31:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:31:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:31:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:31:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:31:06.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:31:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:31:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:31:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 241.46206913200001, 2023-12-06T18:31:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:31:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.893 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:31:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:31:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:31:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:31:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:31:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:31:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:31:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:31:06.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:31:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:31:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:31:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:31:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:31:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:31:06.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:31:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:31:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:31:06.897 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 241.32468204700001, 2023-12-06T18:31:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:31:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:06.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:31:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:31:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:31:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:31:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:31:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:31:06.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:31:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:31:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:31:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:31:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:31:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:31:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:31:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:31:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:31:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:31:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:31:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 241.31801132000001, 2023-12-06T18:31:06.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:31:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:06.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:31:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:31:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:31:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:31:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:31:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:31:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:31:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:31:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:31:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:31:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:31:06.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:31:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:31:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:31:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:31:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:31:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:06.905 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:06.905 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:06.907 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:31:06.908 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:06.903+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:07.008 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:07.004+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:07.078 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:07.078 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:07.078 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:07.078 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:07.087 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:31:07.129 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:07.125+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:07.210 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:07.206+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:07.258 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:07.258 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:07.258 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:07.259 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:07.259 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:07.267 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:31:07.311 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:07.307+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:07.400 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:07.400 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:07.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:07.401 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:31:07.401 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:07.410 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:31:07.412 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:07.408+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:07.513 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:07.509+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:07.541 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:07.541 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:07.542 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:07.542 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:07.551 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:31:07.614 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:07.610+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:07.680 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:07.680 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:07.680 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:07.680 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:07.680 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:07.689 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:31:07.715 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:07.711+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:07.816 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:07.812+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:07.823 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:07.823 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:07.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:07.823 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:07.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:07.824 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:31:07.824 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:07.824 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 46.161257157999998, 2023-12-06T18:31:07.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:31:07.824 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:07.824 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:07.824 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:07.824 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:07.824 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:07.824 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:31:07.824 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:07.824 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:07.824 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:07.824 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:07.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:07.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:07.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:07.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:07.825 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:07.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:07.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:07.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:07.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:07.825 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:07.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:31:07.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:31:07.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:07.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:07.825 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:07.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:31:07.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:31:07.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:07.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:07.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:07.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:31:07.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:31:07.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:07.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:07.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:07.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:31:07.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:31:07.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:07.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:07.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:07.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:31:07.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:31:07.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:07.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:07.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:07.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:31:07.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:31:07.827 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:07.827 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:07.828 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:07.828 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:07.828 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:07.828 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:07.832 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:31:07.917 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:07.913+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:07.970 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:07.970 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:07.970 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:07.971 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:07.979 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:31:08.024 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:08.020+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:08.119 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:08.119 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:08.119 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:08.120 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:08.120 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:08.120 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:08.116+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:08.129 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:31:08.220 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:07.955 106176== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:08.221 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:08.217+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:08.305 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:08.306 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:08.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:08.306 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:31:08.306 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:08.315 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:31:08.357 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:08.353+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:08.423 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:08.419+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:08.455 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:08.190 106176== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:08.498 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:08.498 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:08.498 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:08.499 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:08.508 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:31:08.523 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:08.519+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:08.616 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:08.346 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:08.624 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:08.620+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:08.685 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:08.686 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:08.686 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:08.686 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:08.686 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:08.695 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:31:08.725 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:08.721+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:08.826 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:08.822+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:08.860 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:08.591 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:08.890 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:08.890 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:08.890 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:08.890 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:08.890 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.891 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:31:08.891 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:08.891 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 253.47984879500001, 2023-12-06T18:31:08.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:31:08.891 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:08.891 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:08.891 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:08.891 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:08.891 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:08.891 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:31:08.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.891 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:08.891 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:08.891 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:08.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.892 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:08.892 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:08.892 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.892 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:08.892 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:31:08.892 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:31:08.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.892 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:08.892 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:31:08.892 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:31:08.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.892 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:08.893 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:31:08.893 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:31:08.893 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.893 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.893 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:08.893 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:31:08.893 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:31:08.893 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.893 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.893 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:08.893 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:31:08.893 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:31:08.893 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.893 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.893 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:31:08.893 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:31:08.893 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:31:08.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:08.894 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:31:08.894 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:31:08.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:08.894 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:31:08.894 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:31:08.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.895 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:08.895 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:31:08.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:31:08.895 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.895 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.895 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:08.895 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:31:08.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:31:08.895 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.895 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.895 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:08.895 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:31:08.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:31:08.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.896 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.896 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:08.896 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:31:08.896 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:31:08.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.896 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.896 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:08.896 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:31:08.896 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:31:08.896 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.896 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:08.896 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.896 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.896 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:08.896 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:08.897 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 251.371275556, 2023-12-06T18:31:08.897 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:31:08.897 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:08.897 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:08.897 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:08.897 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:08.897 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:08.897 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:31:08.897 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.897 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:08.897 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.897 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:08.897 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:08.897 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.897 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.897 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.897 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:08.898 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:08.898 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.898 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.898 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.898 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:08.898 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:31:08.898 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:31:08.898 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.898 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.898 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:08.898 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:31:08.898 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:31:08.898 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.898 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.898 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:08.899 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:31:08.899 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:31:08.899 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.899 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.899 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:08.899 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:31:08.899 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:31:08.899 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.899 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.899 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:08.899 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:31:08.899 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:31:08.899 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.899 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.899 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:08.899 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:31:08.899 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:31:08.900 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.900 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.900 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:08.900 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:31:08.900 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:31:08.900 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.900 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.900 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:08.900 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:31:08.900 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:31:08.900 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.900 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.900 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:08.900 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:31:08.900 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:31:08.900 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.901 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.901 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:08.901 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:31:08.901 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:31:08.901 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.901 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.901 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:08.901 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:31:08.901 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:31:08.901 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.901 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:08.901 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.901 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.901 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.901 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:08.901 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:08.902 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 251.37053032399999, 2023-12-06T18:31:08.902 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:31:08.902 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:08.902 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:08.902 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:08.902 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:08.902 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:08.902 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:31:08.902 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.902 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:08.902 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.902 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:08.902 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:08.903 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.903 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.903 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.903 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:08.903 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:08.903 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.903 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.903 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.903 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:08.903 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:31:08.903 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:31:08.903 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.903 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.903 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:08.903 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:31:08.904 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:31:08.904 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.904 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.904 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:08.904 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:31:08.904 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:31:08.904 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.904 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.904 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:08.904 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:31:08.904 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:31:08.904 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.904 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.904 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:08.904 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:31:08.905 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:31:08.905 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.905 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.905 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:08.905 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:31:08.905 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:31:08.905 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.905 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.905 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:08.905 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:31:08.905 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:31:08.905 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.905 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.906 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:08.906 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:31:08.906 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:31:08.906 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.906 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.906 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:08.906 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:31:08.906 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:31:08.906 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.906 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.906 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:08.906 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:31:08.906 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:31:08.906 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.906 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.907 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:08.907 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:31:08.907 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:31:08.907 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.907 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:08.907 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.907 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.907 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.907 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:08.907 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:08.907 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 251.369846569, 2023-12-06T18:31:08.907 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:31:08.907 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:08.907 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:08.907 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:08.907 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:08.907 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:08.908 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:31:08.908 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.908 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:08.908 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.908 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:08.908 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:08.908 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.908 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.908 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.908 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:08.908 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:08.908 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.908 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.908 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.908 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:08.909 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:31:08.909 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:31:08.909 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.909 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.909 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:08.909 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:31:08.909 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:31:08.909 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.909 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.909 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:08.909 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:31:08.909 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:31:08.909 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.909 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.909 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:08.910 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:31:08.910 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:31:08.910 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.910 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.910 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:08.910 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:31:08.910 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:31:08.910 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.910 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.910 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:08.910 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:31:08.910 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:31:08.910 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.910 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.910 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:08.911 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:31:08.911 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:31:08.911 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.911 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.911 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:08.911 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:31:08.911 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:31:08.911 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.911 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.911 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:08.911 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:31:08.911 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:31:08.911 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.911 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.912 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:08.912 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:31:08.912 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:31:08.912 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.912 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.912 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:08.912 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:31:08.912 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:31:08.912 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.912 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:08.912 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.912 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.912 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.912 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:08.912 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:08.912 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 251.364273314, 2023-12-06T18:31:08.913 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:31:08.913 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:08.913 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:08.913 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:08.913 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:08.913 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:08.913 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:31:08.913 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.913 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:08.913 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.913 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:08.913 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:08.913 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.913 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.913 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.913 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:08.914 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:08.914 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.914 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.914 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.914 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:08.914 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:31:08.914 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:31:08.914 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.914 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.914 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:08.914 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:31:08.914 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:31:08.914 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.914 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.915 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:08.915 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:31:08.915 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:31:08.915 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.915 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.915 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:08.915 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:31:08.915 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:31:08.915 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.915 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.915 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:08.915 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:31:08.915 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:31:08.916 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.916 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.916 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:08.916 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:31:08.916 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:31:08.916 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.916 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.916 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:08.916 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:31:08.916 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:31:08.916 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.916 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.916 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:08.916 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:31:08.916 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:31:08.917 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.917 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.917 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:08.917 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:31:08.917 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:31:08.917 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.917 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.917 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:08.917 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:31:08.917 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:31:08.917 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.917 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.917 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:08.918 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:31:08.918 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:31:08.918 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.918 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:08.918 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.918 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.918 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.918 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:08.919 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:08.919 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 251.363599674, 2023-12-06T18:31:08.919 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:31:08.919 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:08.919 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:08.919 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:08.919 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:08.919 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:08.919 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:31:08.919 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.919 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:08.919 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.919 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:08.919 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:08.920 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.920 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.920 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.920 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:08.920 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:08.920 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.920 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.920 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.920 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:08.920 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:31:08.920 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:31:08.920 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.920 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.920 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:08.921 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:31:08.921 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:31:08.921 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.921 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.921 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:08.921 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:31:08.921 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:31:08.921 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.921 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.921 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:08.921 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:31:08.921 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:31:08.921 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.921 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.921 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:08.922 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:31:08.922 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:31:08.922 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.922 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.922 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:08.922 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:31:08.922 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:31:08.922 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.922 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.922 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:08.922 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:31:08.922 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:31:08.922 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.922 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.922 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:08.923 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:31:08.923 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:31:08.923 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.923 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.923 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:08.923 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:31:08.923 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:31:08.923 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.923 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.923 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:08.923 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:31:08.923 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:31:08.923 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.923 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.923 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:08.923 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:31:08.924 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:31:08.924 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.924 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:08.924 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.924 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.924 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.924 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:08.924 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:08.924 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 251.362937475, 2023-12-06T18:31:08.924 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:31:08.924 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:08.924 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:08.924 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:08.924 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:08.925 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:08.925 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:31:08.925 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.925 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:08.925 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.925 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:08.925 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:08.925 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.925 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.925 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.925 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:08.925 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:08.925 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.925 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.925 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.926 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:08.926 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:31:08.926 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:31:08.926 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.926 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.926 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:08.926 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:31:08.926 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:31:08.927 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.927 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.927 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:08.927 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:31:08.927 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:31:08.927 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.927 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.927 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:08.927 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:31:08.927 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:31:08.927 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.927 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.927 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:08.928 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:31:08.928 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:31:08.928 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.928 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.928 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:08.928 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:31:08.928 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:31:08.928 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.928 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.928 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:08.928 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:31:08.928 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:31:08.928 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.928 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.929 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:08.929 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:31:08.929 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:31:08.929 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.929 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.929 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:08.929 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:31:08.929 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:31:08.929 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.929 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.929 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:08.929 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:31:08.929 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:31:08.930 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.930 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.930 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:08.930 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:31:08.930 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:31:08.930 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.930 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:08.930 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.930 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.930 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.930 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:08.930 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:08.931 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 251.29022803999999, 2023-12-06T18:31:08.931 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:31:08.931 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:08.931 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:08.931 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:08.931 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:08.931 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:08.931 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:31:08.931 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.931 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:08.931 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.931 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:08.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:08.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.932 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:08.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:08.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.932 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:08.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:31:08.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:31:08.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.933 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:08.933 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:31:08.933 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:31:08.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.933 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:08.933 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:31:08.933 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:31:08.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.933 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:08.933 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:31:08.933 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:31:08.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.934 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.934 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:08.934 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:31:08.934 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:31:08.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.934 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.934 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:08.934 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:31:08.934 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:31:08.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.935 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:08.935 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:31:08.935 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:31:08.935 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.935 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:08.935 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:31:08.935 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:31:08.935 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.936 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:08.936 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:31:08.936 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:31:08.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.936 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:08.936 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:31:08.936 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:31:08.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.936 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:08.936 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:31:08.937 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:31:08.937 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.937 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:08.937 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.937 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.937 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.937 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:08.937 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:08.937 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 251.289546331, 2023-12-06T18:31:08.937 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:31:08.937 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:08.937 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:08.937 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:08.938 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:08.938 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:08.938 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:31:08.938 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.938 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:08.938 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.938 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:08.938 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:08.938 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.938 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.938 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.938 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:08.939 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:08.939 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.939 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.939 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.939 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:08.939 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:31:08.939 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:31:08.939 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.939 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.939 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:08.939 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:31:08.939 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:31:08.940 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.940 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.940 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:08.940 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:31:08.940 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:31:08.940 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.940 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.940 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:08.940 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:31:08.940 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:31:08.940 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.940 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.941 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:08.941 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:31:08.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:31:08.941 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.941 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:08.941 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:31:08.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:31:08.941 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.941 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:08.941 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:31:08.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:31:08.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:08.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:31:08.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:31:08.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:08.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:31:08.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:31:08.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.943 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:08.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:31:08.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:31:08.943 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.943 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:08.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:31:08.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:31:08.943 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.943 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:08.943 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.944 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.944 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:08.944 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:08.944 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 250.210742243, 2023-12-06T18:31:08.944 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:31:08.944 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:08.944 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:08.944 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:08.944 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:08.944 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:08.944 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:31:08.945 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.945 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:08.945 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.945 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:08.945 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:08.945 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.945 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.945 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.945 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:08.945 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:08.945 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.945 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.946 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.946 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:08.946 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:31:08.946 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:31:08.946 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.946 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.946 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:08.946 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:31:08.946 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:31:08.946 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.946 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.946 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:08.947 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:31:08.947 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:31:08.947 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.947 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.947 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:08.947 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:31:08.947 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:31:08.947 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.947 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.947 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:08.947 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:31:08.947 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:31:08.947 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.948 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.948 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:08.948 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:31:08.948 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:31:08.948 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.948 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.948 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:08.948 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:31:08.948 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:31:08.948 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.948 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.948 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:08.948 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:31:08.949 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:31:08.949 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.949 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.949 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:08.949 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:31:08.949 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:31:08.949 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.949 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.949 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:08.949 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:31:08.949 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:31:08.949 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.949 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.950 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:08.950 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:31:08.950 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:31:08.950 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.950 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:08.950 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.950 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.950 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.950 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:08.950 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:08.950 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 250.20932566900001, 2023-12-06T18:31:08.950 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:31:08.951 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:08.951 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:08.951 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:08.951 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:08.951 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:08.951 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:31:08.951 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.951 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:08.951 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.951 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:08.951 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:08.952 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.952 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.952 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.952 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:08.952 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:08.952 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.952 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.952 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.952 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:08.952 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:31:08.952 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:31:08.952 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.952 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.952 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:08.953 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:31:08.953 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:31:08.953 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.953 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.953 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:08.953 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:31:08.953 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:31:08.953 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.953 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.953 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:08.953 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:31:08.953 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:31:08.953 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.953 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.954 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:08.954 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:31:08.954 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:31:08.954 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.954 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.954 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:08.954 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:31:08.954 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:31:08.954 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.954 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.954 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:08.955 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:31:08.955 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:31:08.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.955 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.955 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:08.955 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:31:08.955 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:31:08.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.955 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.955 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:08.955 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:31:08.955 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:31:08.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.956 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.956 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:08.956 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:31:08.956 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:31:08.956 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.956 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.956 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:08.956 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:31:08.956 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:31:08.956 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.956 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:08.956 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.956 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.956 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.957 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:08.957 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:08.957 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 250.20824054400001, 2023-12-06T18:31:08.957 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:31:08.957 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:08.957 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:08.957 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:08.957 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:08.957 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:08.957 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:31:08.957 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.957 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:08.957 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.958 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:08.958 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:08.958 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.958 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.958 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.958 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:08.958 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:08.958 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.958 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.958 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.958 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:08.958 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:31:08.959 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:31:08.959 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.959 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.959 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:08.959 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:31:08.959 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:31:08.959 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.959 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.959 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:08.959 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:31:08.959 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:31:08.959 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.959 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.960 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:08.960 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:31:08.960 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:31:08.960 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.960 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.960 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:08.960 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:31:08.960 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:31:08.960 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.960 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.960 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:08.960 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:31:08.961 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:31:08.961 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.961 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.961 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:08.961 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:31:08.961 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:31:08.961 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.961 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.961 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:08.961 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:31:08.961 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:31:08.961 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.961 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.961 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:08.962 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:31:08.962 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:31:08.962 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.962 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.962 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:08.962 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:31:08.962 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:31:08.962 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.962 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.962 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:08.962 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:31:08.962 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:31:08.963 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.963 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:08.963 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.963 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.963 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.963 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:08.963 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:08.963 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 250.17309144199999, 2023-12-06T18:31:08.963 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:31:08.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:08.963 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:08.963 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:08.964 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:08.964 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:08.964 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:31:08.964 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.964 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:08.964 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.964 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:08.964 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:08.964 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.964 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.964 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.964 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:08.964 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:08.964 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.965 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.965 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.965 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:08.965 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:31:08.965 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:31:08.965 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.965 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.965 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:08.965 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:31:08.965 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:31:08.965 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.965 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.965 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:08.965 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:31:08.966 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:31:08.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.966 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.966 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:08.966 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:31:08.966 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:31:08.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.966 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.966 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:08.966 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:31:08.966 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:31:08.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.966 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.967 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:08.967 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:31:08.967 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:31:08.967 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.967 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.967 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:08.967 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:31:08.967 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:31:08.967 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.967 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.967 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:08.967 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:31:08.967 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:31:08.967 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.968 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.968 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:08.968 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:31:08.968 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:31:08.968 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.968 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.968 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:08.968 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:31:08.968 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:31:08.968 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.968 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.968 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:08.969 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:31:08.969 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:31:08.969 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.969 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:08.969 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.969 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.969 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:08.969 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:08.969 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 243.62336091700001, 2023-12-06T18:31:08.969 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:31:08.969 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:08.969 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:08.969 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:08.969 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:08.970 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:08.970 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:31:08.970 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.970 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:08.970 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.970 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:08.970 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:08.970 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.970 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.970 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.970 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:08.970 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:08.971 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.971 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.971 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.971 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:08.971 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:31:08.971 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:31:08.971 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.971 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.971 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:08.971 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:31:08.971 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:31:08.971 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.971 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.972 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:08.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:31:08.972 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:31:08.972 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.972 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.972 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:08.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:31:08.972 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:31:08.972 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.972 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.972 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:08.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:31:08.972 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:31:08.973 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.973 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.973 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:08.973 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:31:08.973 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:31:08.973 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.973 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.973 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:08.973 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:31:08.973 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:31:08.973 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.973 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.973 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:08.974 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:31:08.974 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:31:08.974 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.974 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.974 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:08.974 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:31:08.974 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:31:08.974 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.974 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.974 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:08.974 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:31:08.974 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:31:08.974 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.975 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.975 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:08.975 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:31:08.975 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:31:08.975 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.975 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:08.975 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.975 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.975 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.975 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:08.975 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:08.975 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 243.62255530900001, 2023-12-06T18:31:08.975 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:31:08.975 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:08.976 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:08.976 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:08.976 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:08.976 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:08.976 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:31:08.976 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.976 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:08.976 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.976 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:08.976 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:08.976 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.976 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.976 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.976 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:08.977 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:08.977 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.977 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.977 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:08.977 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:31:08.977 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:31:08.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.977 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.977 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:08.977 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:31:08.977 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:31:08.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.978 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.978 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:08.978 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:31:08.978 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:31:08.978 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.978 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.978 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:08.978 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:31:08.978 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:31:08.978 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.978 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.978 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:08.979 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:31:08.979 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:31:08.979 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.979 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.979 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:08.979 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:31:08.979 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:31:08.979 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.979 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.979 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:08.979 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:31:08.979 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:31:08.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.980 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:08.980 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:31:08.980 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:31:08.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.980 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:08.980 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:31:08.980 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:31:08.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.980 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:08.981 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:31:08.981 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:31:08.981 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.981 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.981 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:08.981 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:31:08.981 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:31:08.981 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.981 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:08.981 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.981 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.981 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.981 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:08.982 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:08.982 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 243.62186449999999, 2023-12-06T18:31:08.982 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:31:08.982 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:08.982 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:08.982 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:08.982 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:08.982 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:08.982 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:31:08.982 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.982 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:08.983 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.983 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:08.983 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:08.983 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.983 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.983 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.983 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:08.983 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:08.983 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.983 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.983 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.983 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:08.983 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:31:08.984 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:31:08.984 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.984 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.984 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:08.984 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:31:08.984 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:31:08.984 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.984 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.984 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:08.984 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:31:08.984 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:31:08.984 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.985 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.985 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:08.985 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:31:08.985 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:31:08.985 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.985 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.985 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:08.985 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:31:08.985 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:31:08.985 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.985 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.985 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:08.985 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:31:08.986 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:31:08.986 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.986 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.986 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:08.986 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:31:08.986 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:31:08.986 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.986 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.986 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:08.986 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:31:08.986 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:31:08.986 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.987 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.987 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:08.987 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:31:08.987 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:31:08.987 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.987 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.987 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:08.987 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:31:08.987 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:31:08.987 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.987 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.987 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:08.987 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:31:08.987 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:31:08.988 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.988 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:08.988 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.988 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.988 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.988 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:08.988 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:08.988 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 243.61756699200001, 2023-12-06T18:31:08.988 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:31:08.988 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:08.988 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:08.988 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:08.988 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:08.989 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:08.989 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:31:08.989 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.989 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:08.989 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.989 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:08.989 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:08.989 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.989 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.989 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.989 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:08.989 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:08.990 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.990 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.990 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.990 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:08.990 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:31:08.990 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:31:08.990 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.990 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.990 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:08.990 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:31:08.993 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:31:08.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.993 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.993 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:08.993 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:31:08.993 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:31:08.994 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.994 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.994 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:08.994 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:31:08.994 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:31:08.994 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.994 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.994 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:08.994 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:31:08.994 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:31:08.994 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.995 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.995 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:08.995 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:31:08.995 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:31:08.995 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.995 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.995 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:08.995 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:31:08.995 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:31:08.995 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.995 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.995 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:08.995 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:31:08.995 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:31:08.996 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.996 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.996 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:08.996 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:31:08.996 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:31:08.996 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.996 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.996 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:08.996 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:31:08.996 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:31:08.996 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.996 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.997 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:08.997 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:31:08.997 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:31:08.997 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.997 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:08.997 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:08.997 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.997 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.997 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:08.997 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:08.997 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 243.61689931699999, 2023-12-06T18:31:08.997 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:31:08.997 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:08.998 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:08.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:08.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:08.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:08.998 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:31:08.998 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.998 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:08.998 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.998 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:08.998 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:08.998 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.998 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.998 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.999 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:08.999 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:08.999 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:08.999 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.999 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.999 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:08.999 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:31:08.999 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:31:08.999 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:08.999 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:08.999 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:08.999 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:31:08.999 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:31:09.000 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.000 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.000 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:09.000 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:31:09.000 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:31:09.000 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.000 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.000 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:09.000 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:31:09.000 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:31:09.000 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.000 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.001 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:09.001 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:31:09.001 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:31:09.001 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.001 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.001 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:09.001 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:31:09.001 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:31:09.001 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.001 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.001 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:09.001 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:31:09.001 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:31:09.002 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.002 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.002 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:09.002 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:31:09.002 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:31:09.002 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.002 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.002 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:09.002 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:31:09.002 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:31:09.002 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.002 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.002 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:09.003 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:31:09.003 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:31:09.003 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.003 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.003 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:09.003 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:31:09.003 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:31:09.003 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:09.003 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:09.003 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:09.003 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.003 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.003 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:09.003 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:09.004 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 243.616243889, 2023-12-06T18:31:09.004 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:31:09.004 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:09.004 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:09.004 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:09.004 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:09.004 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:09.004 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:31:09.004 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.004 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:09.004 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.004 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:09.004 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:09.004 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:09.005 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.005 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.005 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:09.005 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:09.005 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:09.005 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.005 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.005 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:09.005 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:31:09.005 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:31:09.005 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.005 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.005 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:09.006 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:31:09.006 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:31:09.006 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.006 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.006 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:09.006 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:31:09.006 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:31:09.006 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.006 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.006 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:09.006 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:31:09.006 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:31:09.007 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.007 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.007 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:09.007 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:31:09.007 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:31:09.007 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.007 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.007 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:09.007 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:31:09.007 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:31:09.007 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.007 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.008 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:09.008 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:31:09.008 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:31:09.008 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.008 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.008 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:09.008 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:31:09.008 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:31:09.008 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.008 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.008 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:09.009 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:31:09.009 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:31:09.009 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.009 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.009 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:09.009 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:31:09.009 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:31:09.009 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.010 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.010 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:09.010 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:31:09.010 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:31:09.010 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:09.010 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:09.010 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:09.010 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.010 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.010 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:09.010 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:09.011 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 243.575589342, 2023-12-06T18:31:09.011 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:31:09.011 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:09.011 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:09.011 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:09.011 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:09.011 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:09.011 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:31:09.011 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.011 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:09.011 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.011 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:09.011 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:09.012 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:09.012 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.012 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.012 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:09.012 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:09.012 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:09.012 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.012 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.012 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:09.012 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:31:09.012 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:31:09.013 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.013 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.013 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:09.013 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:31:09.013 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:31:09.013 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.013 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.013 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:09.013 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:31:09.013 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:31:09.013 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.013 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.013 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:09.013 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:31:09.014 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:31:09.014 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.014 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.014 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:09.014 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:31:09.014 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:31:09.014 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.014 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.014 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:09.014 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:31:09.014 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:31:09.014 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:09.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:31:09.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:31:09.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:09.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:31:09.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:31:09.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:09.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:31:09.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:31:09.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:09.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:31:09.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:31:09.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:09.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:31:09.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:31:09.016 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:09.016 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:09.016 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:09.017 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:09.017 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:09.017 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:09.018 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:31:09.019 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:09.015+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:09.120 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:09.116+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:09.197 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:09.198 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:09.198 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:09.198 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:09.207 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:31:09.221 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:09.217+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:09.321 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:09.318+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:09.419 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:09.419 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:09.419 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:09.419 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:09.419 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:09.422 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:09.418+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:09.431 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:31:09.523 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:09.520+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:09.615 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:09.615 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:09.615 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:09.615 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:09.615 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.615 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:31:09.615 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:09.616 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 227.798610776, 2023-12-06T18:31:09.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:31:09.616 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:09.616 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:31:09.616 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:09.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:09.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:09.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:09.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.616 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.616 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:09.616 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:09.616 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:09.616 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:09.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:31:09.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:31:09.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:09.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:31:09.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:31:09.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.617 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:09.617 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:31:09.617 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:31:09.617 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.617 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:09.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:31:09.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:31:09.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:09.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:31:09.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:31:09.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.618 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:09.618 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:31:09.618 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:31:09.618 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:09.618 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:09.619 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:09.619 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:31:09.619 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:31:09.619 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:09.619 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:09.619 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:09.619 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:09.619 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:09.619 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:09.624 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:31:09.624 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:09.620+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:09.725 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:09.722+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:09.806 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:09.806 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:09.806 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:09.806 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:09.815 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:31:09.827 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:09.823+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:09.927 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:09.923+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:09.993 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:09.993 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:09.993 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:09.993 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:09.993 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:10.002 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:31:10.046 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:10.042+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:10.129 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:10.125+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:10.143 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:10.143 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:10.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:10.144 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:10.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:10.144 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:10.144 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:10.144 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 109.044272319, 2023-12-06T18:31:10.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:31:10.144 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:10.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:10.144 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:10.144 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:10.144 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:10.144 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:31:10.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:10.144 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:10.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:10.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:10.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:10.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:10.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:10.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:10.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:10.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:10.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:10.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:10.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:10.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:10.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:31:10.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:31:10.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:10.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:10.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:10.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:31:10.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:31:10.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:10.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:10.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:10.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:31:10.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:31:10.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:10.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:10.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:10.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:31:10.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:31:10.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:10.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:10.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:10.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:31:10.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:31:10.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:10.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:10.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:10.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:31:10.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:31:10.147 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:10.147 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:10.147 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:10.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:10.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:10.148 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:31:10.148 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:10.148 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 49.026015016999999, 2023-12-06T18:31:10.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:31:10.148 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:10.148 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:10.148 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:10.148 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:10.148 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:10.148 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:31:10.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:10.148 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:10.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:10.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:10.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:10.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:10.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:10.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:10.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:10.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:10.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:10.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:10.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:10.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:10.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:31:10.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:31:10.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:10.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:10.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:10.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:31:10.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:31:10.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:10.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:10.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:10.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:31:10.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:31:10.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:10.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:10.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:10.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:31:10.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:31:10.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:10.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:10.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:10.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:31:10.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:31:10.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:10.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:10.151 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:10.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:31:10.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:31:10.151 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:10.151 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:10.151 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:10.151 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:10.151 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:10.151 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:10.153 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:31:10.230 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:10.226+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:10.283 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:10.284 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:10.284 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:10.284 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:10.294 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:31:10.331 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:10.328+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:10.423 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:10.424 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:10.424 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:10.424 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:10.424 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:10.432 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:10.428+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:10.433 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:31:10.533 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:10.530+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:10.565 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:10.565 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:10.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:10.565 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:10.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:10.566 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:10.566 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:10.566 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 169.07209547799999, 2023-12-06T18:31:10.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:31:10.566 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:10.566 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:10.566 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:10.566 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:10.566 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:10.566 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:31:10.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:10.566 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:10.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:10.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:10.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:10.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:10.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:10.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:10.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:10.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:10.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:10.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:10.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:10.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:10.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:31:10.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:31:10.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:10.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:10.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:10.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:31:10.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:31:10.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:10.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:10.568 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:10.568 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:31:10.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:31:10.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:10.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:10.568 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:10.568 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:31:10.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:31:10.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:10.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:10.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:10.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:31:10.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:31:10.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:10.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:10.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:10.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:31:10.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:31:10.569 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:10.569 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:10.569 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:10.570 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:10.570 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:10.570 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:10.575 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:31:10.634 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:10.631+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:10.710 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:10.710 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:10.710 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:10.710 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:10.720 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:31:10.735 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:10.731+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:10.836 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:10.832+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:10.850 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:10.850 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:10.850 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:10.850 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:10.850 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:10.859 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:31:10.944 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:10.940+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:11.039 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:11.035+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:11.093 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.093 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:31:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 254.45813520900001, 2023-12-06T18:31:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:31:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:11.094 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:31:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:31:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:31:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:31:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:31:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:31:11.095 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:31:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:31:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:31:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:31:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:31:11.096 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:31:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:31:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:31:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:31:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:31:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.097 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:31:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 253.43530820000001, 2023-12-06T18:31:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:31:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.098 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:31:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:31:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:31:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:31:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.099 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:31:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:31:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:31:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:31:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.100 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:31:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:31:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:31:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:31:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:31:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:31:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.101 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:31:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:31:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:31:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 253.42326156199999, 2023-12-06T18:31:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:31:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.102 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:31:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:31:11.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:31:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:31:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:31:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:31:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:31:11.104 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:31:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:31:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:31:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:31:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:31:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:31:11.105 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:31:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:31:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:31:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:31:11.106 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 253.40186095499999, 2023-12-06T18:31:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:31:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:31:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:31:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:31:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:31:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:31:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:31:11.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:31:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:31:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:31:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:31:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:31:11.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:31:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:31:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:31:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:31:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:31:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.110 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:31:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 253.40068163199999, 2023-12-06T18:31:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:31:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.111 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:31:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:31:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:31:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:31:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.112 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:31:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:31:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:31:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:31:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:31:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:31:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.113 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:31:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:31:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:31:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:31:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:31:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:31:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.114 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:31:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 253.40010965100001, 2023-12-06T18:31:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:31:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:31:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:31:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:31:11.116 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:31:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:31:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:31:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:31:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:31:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:31:11.117 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:31:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:31:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:31:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:31:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:31:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.118 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:31:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:31:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:31:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 252.95427418099999, 2023-12-06T18:31:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:31:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.119 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:31:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:31:11.120 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:31:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:31:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:31:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:31:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:31:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:31:11.121 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:31:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:31:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:31:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:31:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:31:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:31:11.122 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:31:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:31:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:31:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 252.952674325, 2023-12-06T18:31:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:31:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.123 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.124 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:31:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:31:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:31:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:31:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:31:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:31:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:31:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:31:11.125 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.126 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.126 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.126 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:31:11.126 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:31:11.126 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.126 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.126 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.126 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:31:11.126 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:31:11.126 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.126 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.126 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.126 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:31:11.126 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:31:11.126 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.127 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.127 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.127 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:31:11.127 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:31:11.127 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.127 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.127 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.127 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.127 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.127 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:31:11.127 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:11.127 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 252.573468709, 2023-12-06T18:31:11.127 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:31:11.127 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:31:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:31:11.128 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:31:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:31:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:31:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:31:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:31:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:31:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.129 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:31:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:31:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:31:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:31:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:31:11.130 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:31:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:31:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:31:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:31:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 252.33596652700001, 2023-12-06T18:31:11.131 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:31:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:11.132 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:31:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:31:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:31:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:31:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:31:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:31:11.133 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:31:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:31:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:31:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:31:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.134 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:31:11.135 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:31:11.135 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.135 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:31:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:31:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:31:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:31:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.136 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:31:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 252.33523852799999, 2023-12-06T18:31:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:31:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.137 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:31:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:31:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:31:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:31:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:31:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:31:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:31:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:31:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:31:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:31:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:31:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:31:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:31:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:31:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:31:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:31:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.140 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:31:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 252.33022611800001, 2023-12-06T18:31:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:31:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.141 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:31:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:31:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:31:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:31:11.142 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:31:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:31:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:31:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:31:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:31:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:31:11.143 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:31:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:31:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:31:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:31:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:31:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:31:11.144 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:31:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 252.324699039, 2023-12-06T18:31:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:31:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.145 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.146 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:31:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:31:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:31:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:31:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:31:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:31:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.147 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:31:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:31:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:31:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:31:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:31:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:31:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.148 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:31:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:31:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:31:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:31:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:31:11.149 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 252.32418820199999, 2023-12-06T18:31:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:31:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:11.150 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:31:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:31:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:31:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:31:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:31:11.151 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:31:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:31:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:31:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:31:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:31:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:31:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:31:11.152 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:31:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:31:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:31:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:31:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:31:11.153 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 252.32372894100001, 2023-12-06T18:31:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:31:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.154 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:31:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:31:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:31:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:31:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:31:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:31:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.155 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:31:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:31:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:31:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:31:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:31:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:31:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:31:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:31:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:31:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:31:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:31:11.157 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 245.74381313399999, 2023-12-06T18:31:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:31:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.158 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:31:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:31:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:31:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:31:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:31:11.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:31:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:31:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:31:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:31:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:31:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:31:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:31:11.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:31:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:31:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:31:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:31:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:31:11.161 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 245.73757532900001, 2023-12-06T18:31:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:31:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.162 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:31:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:31:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:31:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:31:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:31:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:31:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.163 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:31:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:31:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:31:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:31:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:31:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:31:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.164 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:31:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:31:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:31:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:31:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:31:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 245.73151225000001, 2023-12-06T18:31:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:31:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.165 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:31:11.166 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:31:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:31:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:31:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:31:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:31:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:31:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:31:11.167 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:31:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:31:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:31:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:31:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:31:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:31:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.168 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:31:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:31:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:31:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 245.59412516500001, 2023-12-06T18:31:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:31:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.169 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:31:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:31:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.170 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:31:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:31:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:31:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:31:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:31:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:31:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.171 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:31:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:31:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:31:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:31:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:31:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:31:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:31:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:31:11.172 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:31:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 245.58745443800001, 2023-12-06T18:31:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:31:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.173 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:31:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:31:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:31:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:31:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.174 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:31:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:31:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:31:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:31:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:31:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:31:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.175 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:31:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:31:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:31:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:31:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:31:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:31:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:11.176 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:11.177 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:11.178 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:31:11.178 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:11.174+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:11.279 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:11.275+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:11.350 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:11.350 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:11.350 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:11.350 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:11.360 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:31:11.401 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:11.398+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:11.481 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:11.478+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:11.582 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:11.578+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:11.599 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:11.600 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:11.600 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:11.600 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:11.600 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:11.609 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:31:11.694 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:11.690+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:11.784 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:11.781+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:11.785 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:31:11.785 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:11.794 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:31:11.885 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:11.882+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:11.986 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:11.982+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:12.008 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:12.008 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:12.008 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:12.009 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:12.019 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:31:12.105 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:12.102+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:12.188 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:12.185+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:12.190 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:12.190 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:12.191 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:12.191 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:12.191 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:12.200 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:31:12.289 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:12.286+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:12.371 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:12.371 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:31:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 50.709500491, 2023-12-06T18:31:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:31:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:31:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:12.372 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:12.373 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:31:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:31:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:31:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:31:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:12.374 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:31:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:31:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:31:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:31:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:31:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:31:12.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:31:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:31:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:12.376 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:12.376 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:12.381 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:31:12.390 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:12.387+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:12.491 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:12.487+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:12.542 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:12.542 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:12.542 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:12.543 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:12.555 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:31:12.599 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:12.595+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:12.692 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:12.693 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:12.693 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:12.693 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:12.693 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:12.694 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:12.690+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:12.702 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:31:12.795 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:12.791+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:12.879 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:12.879 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:12.879 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:12.880 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:31:12.880 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:12.888 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:31:12.930 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:12.926+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:12.996 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:12.993+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:13.066 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:13.066 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:13.066 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:13.066 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:13.077 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:31:13.097 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:13.094+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:13.198 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:13.195+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:13.250 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:13.290 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:13.290 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:13.290 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:13.291 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:13.292 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:31:13.300 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:13.296+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:13.401 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:13.397+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:13.487 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:13.487 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:13.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:13.487 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:13.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.488 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:31:13.488 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:13.488 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 258.07664992799999, 2023-12-06T18:31:13.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:31:13.488 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.488 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.488 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.488 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.488 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.488 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:31:13.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.488 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:13.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:13.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:31:13.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:31:13.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.490 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.490 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:31:13.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:31:13.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.490 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.490 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:31:13.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:31:13.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.490 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.490 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:31:13.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:31:13.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:31:13.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:31:13.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:31:13.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:31:13.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:31:13.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:31:13.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:31:13.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:31:13.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:31:13.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:31:13.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:31:13.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:31:13.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:31:13.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:31:13.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:31:13.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:31:13.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:31:13.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:31:13.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:31:13.493 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.494 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.494 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.494 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:13.494 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:13.494 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 255.96807668899999, 2023-12-06T18:31:13.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:31:13.494 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.494 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.494 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.494 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.494 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.495 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:31:13.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.495 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:13.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:13.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.496 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:31:13.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:31:13.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.496 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:31:13.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:31:13.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.496 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:31:13.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:31:13.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:31:13.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:31:13.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:31:13.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:31:13.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:31:13.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:31:13.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:31:13.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:31:13.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:31:13.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:31:13.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:31:13.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:31:13.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:31:13.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:31:13.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:31:13.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:31:13.499 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.499 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.499 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.499 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:13.499 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:13.499 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 255.967331457, 2023-12-06T18:31:13.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:31:13.500 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.500 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.500 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.500 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.500 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.500 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:31:13.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.500 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:13.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.501 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.501 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:13.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.501 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.501 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:31:13.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:31:13.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.501 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.501 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:31:13.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:31:13.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:31:13.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:31:13.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:31:13.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:31:13.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:31:13.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:31:13.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:31:13.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:31:13.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:31:13.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:31:13.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:31:13.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:31:13.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:31:13.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:31:13.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:31:13.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:31:13.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:31:13.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:31:13.505 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.505 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.505 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.505 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:13.505 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:13.505 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 255.96664770199999, 2023-12-06T18:31:13.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:31:13.505 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.505 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.505 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.505 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.505 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.505 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:31:13.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.505 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:13.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:13.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:31:13.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:31:13.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:31:13.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:31:13.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:31:13.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:31:13.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:31:13.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:31:13.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:31:13.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:31:13.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:31:13.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:31:13.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:31:13.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:31:13.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:31:13.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:31:13.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:31:13.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:31:13.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:31:13.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:31:13.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.510 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.510 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:31:13.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:31:13.510 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.510 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.510 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.510 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:13.510 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:13.510 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 255.96107444699999, 2023-12-06T18:31:13.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:31:13.511 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.511 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.511 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.511 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.511 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.511 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:31:13.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.511 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:13.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:13.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:31:13.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:31:13.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:31:13.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:31:13.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:31:13.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:31:13.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:31:13.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:31:13.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:31:13.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:31:13.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:31:13.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:31:13.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:31:13.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:31:13.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:31:13.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:31:13.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:31:13.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:31:13.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:31:13.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:31:13.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:31:13.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:31:13.515 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.515 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.515 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.516 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:13.516 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:13.516 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 255.96040080700001, 2023-12-06T18:31:13.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:31:13.516 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.516 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.516 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.516 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.516 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.516 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:31:13.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.516 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:13.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:13.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:31:13.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:31:13.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:31:13.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:31:13.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:31:13.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:31:13.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:31:13.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:31:13.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:31:13.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:31:13.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:31:13.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:31:13.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:31:13.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:31:13.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:31:13.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:31:13.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:31:13.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:31:13.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:31:13.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:31:13.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:31:13.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:31:13.521 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.521 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.521 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.521 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:13.521 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:13.521 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 255.95973860800001, 2023-12-06T18:31:13.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:31:13.522 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.522 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.522 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.522 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.522 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.522 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:31:13.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.522 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.522 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.522 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:13.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:13.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:31:13.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:31:13.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:31:13.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:31:13.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:31:13.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:31:13.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:31:13.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:31:13.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:31:13.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:31:13.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:31:13.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:31:13.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:31:13.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:31:13.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:31:13.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:31:13.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:31:13.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:31:13.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:31:13.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:31:13.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:31:13.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:31:13.527 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.527 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.527 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.527 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:13.527 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:13.527 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 255.887029173, 2023-12-06T18:31:13.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:31:13.527 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.527 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.527 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.527 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.527 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.527 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:31:13.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.528 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:13.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:13.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:31:13.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:31:13.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:31:13.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:31:13.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:31:13.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:31:13.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:31:13.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:31:13.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:31:13.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:31:13.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:31:13.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:31:13.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:31:13.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:31:13.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:31:13.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:31:13.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:31:13.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:31:13.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:31:13.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:31:13.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:31:13.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:31:13.532 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.532 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.532 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.532 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:13.532 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:13.533 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 255.88634746400001, 2023-12-06T18:31:13.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:31:13.533 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.533 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.533 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.533 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.533 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.533 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:31:13.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.533 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:13.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:13.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:31:13.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:31:13.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:31:13.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:31:13.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:31:13.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:31:13.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:31:13.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:31:13.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:31:13.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:31:13.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:31:13.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:31:13.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:31:13.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:31:13.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:31:13.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:31:13.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:31:13.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:31:13.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:31:13.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:31:13.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:31:13.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:31:13.538 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.538 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.538 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.538 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:13.538 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:13.538 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 254.80754337600001, 2023-12-06T18:31:13.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:31:13.538 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.538 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.538 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.538 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.538 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.539 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:31:13.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.539 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:13.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:13.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:31:13.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:31:13.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:31:13.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:31:13.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:31:13.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:31:13.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:31:13.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:31:13.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:31:13.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:31:13.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:31:13.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:31:13.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:31:13.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:31:13.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:31:13.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:31:13.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:31:13.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:31:13.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:31:13.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:31:13.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:31:13.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:31:13.543 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.543 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.543 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.543 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:13.544 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:13.544 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 254.80612680199999, 2023-12-06T18:31:13.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:31:13.544 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.544 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.544 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.544 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.544 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.544 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:31:13.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.544 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:13.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:13.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:31:13.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:31:13.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:31:13.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:31:13.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:31:13.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:31:13.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:31:13.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:31:13.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:31:13.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:31:13.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:31:13.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:31:13.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:31:13.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:31:13.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:31:13.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:31:13.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:31:13.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:31:13.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:31:13.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:31:13.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:31:13.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:31:13.549 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.549 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.549 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.550 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:13.550 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:13.550 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 254.80504167699999, 2023-12-06T18:31:13.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:31:13.550 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.550 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.550 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.550 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.550 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.550 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:31:13.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.550 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:13.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:13.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:31:13.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:31:13.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:31:13.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:31:13.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:31:13.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:31:13.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:31:13.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:31:13.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:31:13.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:31:13.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:31:13.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:31:13.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:31:13.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:31:13.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:31:13.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:31:13.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:31:13.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:31:13.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:31:13.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:31:13.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:31:13.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:31:13.555 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.555 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.555 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.555 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:13.555 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:13.555 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 254.769892575, 2023-12-06T18:31:13.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:31:13.555 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.556 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.556 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.556 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.556 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.556 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:31:13.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.556 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.556 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.556 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:13.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.556 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.556 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:13.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:31:13.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:31:13.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:31:13.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:31:13.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:31:13.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:31:13.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:31:13.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:31:13.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:31:13.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:31:13.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:31:13.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:31:13.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:31:13.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:31:13.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:31:13.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:31:13.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:31:13.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:31:13.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:31:13.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:31:13.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:31:13.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:31:13.560 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.560 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.560 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.561 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:13.561 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:13.561 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 248.22016205, 2023-12-06T18:31:13.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:31:13.561 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.561 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.561 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.561 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.561 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.561 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:31:13.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.561 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:13.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:13.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:31:13.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:31:13.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:31:13.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:31:13.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:31:13.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:31:13.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:31:13.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:31:13.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:31:13.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:31:13.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:31:13.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:31:13.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:31:13.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:31:13.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:31:13.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:31:13.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:31:13.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:31:13.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:31:13.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:31:13.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:31:13.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:31:13.566 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.566 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.566 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.566 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:13.566 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:13.566 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 248.21935644199999, 2023-12-06T18:31:13.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:31:13.566 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.566 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.566 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.567 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.567 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.567 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:31:13.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.567 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.567 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.567 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:13.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.567 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.567 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:13.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.568 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.568 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:31:13.568 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:31:13.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.568 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.568 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:31:13.568 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:31:13.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.568 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.568 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:31:13.568 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:31:13.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:31:13.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:31:13.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:31:13.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:31:13.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:31:13.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:31:13.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.570 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.570 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:31:13.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:31:13.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.570 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.570 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:31:13.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:31:13.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.570 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.570 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:31:13.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:31:13.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:31:13.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:31:13.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:31:13.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:31:13.571 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.571 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.571 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.572 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:13.572 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:13.572 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 248.218665633, 2023-12-06T18:31:13.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:31:13.572 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.572 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.572 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.572 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.572 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.572 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:31:13.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.572 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.572 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:13.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.573 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.573 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:13.573 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.573 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.573 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:31:13.573 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:31:13.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.573 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.573 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:31:13.573 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:31:13.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:31:13.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:31:13.574 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:31:13.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:31:13.574 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:31:13.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:31:13.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.575 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.575 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:31:13.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:31:13.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.575 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.575 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:31:13.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:31:13.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.575 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.575 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:31:13.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:31:13.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.576 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.576 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:31:13.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:31:13.576 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.576 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.576 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.576 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:31:13.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:31:13.576 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.576 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.576 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.576 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:31:13.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:31:13.576 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.576 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.576 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.577 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:13.577 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:13.577 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 248.21436812499999, 2023-12-06T18:31:13.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:31:13.577 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.577 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.577 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.577 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.577 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.577 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:31:13.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.577 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.577 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:13.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:13.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:31:13.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:31:13.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.579 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.579 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.579 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:31:13.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:31:13.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.579 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.579 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.579 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:31:13.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:31:13.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.579 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.579 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.579 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:31:13.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:31:13.580 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.580 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.580 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:31:13.580 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:31:13.580 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.580 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.580 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:31:13.580 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:31:13.580 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.580 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:31:13.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:31:13.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.581 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.581 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:31:13.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:31:13.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.581 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.581 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:31:13.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:31:13.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.582 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.582 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:31:13.582 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:31:13.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.582 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.582 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:31:13.582 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:31:13.582 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.582 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.582 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.583 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:13.583 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:13.583 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 248.21370045, 2023-12-06T18:31:13.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:31:13.583 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.583 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.583 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.583 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.583 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.583 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:31:13.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.583 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.584 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:13.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.584 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:13.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.584 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:31:13.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:31:13.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.585 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:31:13.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:31:13.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.585 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.585 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:31:13.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:31:13.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.585 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.585 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:31:13.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:31:13.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.585 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:31:13.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:31:13.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:31:13.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:31:13.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:31:13.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:31:13.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:31:13.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:31:13.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:31:13.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:31:13.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:31:13.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:31:13.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.588 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.588 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.588 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:31:13.588 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:31:13.588 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.588 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.588 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.588 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.588 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.588 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:13.588 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:13.588 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 248.21304502199999, 2023-12-06T18:31:13.588 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:31:13.588 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.589 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.589 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.589 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.589 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.589 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:31:13.589 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.589 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.589 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.589 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:13.589 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.589 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.589 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.589 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:13.590 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.590 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.590 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.590 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.590 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:31:13.590 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:31:13.590 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.590 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.590 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.590 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:31:13.590 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:31:13.590 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.590 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.590 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.591 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:31:13.591 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:31:13.591 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.591 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.591 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.591 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:31:13.591 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:31:13.591 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.591 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.591 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.591 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:31:13.591 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:31:13.591 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.591 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.591 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.591 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:31:13.592 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:31:13.592 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.592 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.592 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.592 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:31:13.592 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:31:13.592 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.592 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.592 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.592 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:31:13.592 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:31:13.592 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.592 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.592 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:31:13.593 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:31:13.593 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.593 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.593 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:31:13.593 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:31:13.593 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.593 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.593 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:31:13.593 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:31:13.593 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 248.17239047499999, 2023-12-06T18:31:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:31:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:13.594 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:31:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:13.595 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:31:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:31:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:31:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:31:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:31:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:31:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.596 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:31:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:31:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:31:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:31:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:31:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:31:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.597 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:31:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:31:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:31:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:31:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:31:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:31:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:31:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:31:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:31:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:31:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:13.599 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:13.599 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:13.601 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:31:13.601 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:13.597+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:13.702 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:13.698+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:13.778 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:13.778 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:13.778 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:13.778 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:13.788 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:31:13.831 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:13.826+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:13.904 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:13.900+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:13.964 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:13.964 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:13.964 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:13.965 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:13.965 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:13.973 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:31:14.020 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:14.016+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:14.106 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:14.102+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:14.113 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:14.113 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:14.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:14.113 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:14.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.113 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:31:14.113 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:14.113 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 232.29753275499999, 2023-12-06T18:31:14.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:31:14.114 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:14.114 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:31:14.114 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:14.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:14.114 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:14.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:14.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:14.114 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:14.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:14.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:14.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:31:14.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:31:14.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:14.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:31:14.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:31:14.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:14.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:31:14.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:31:14.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:14.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:31:14.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:31:14.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:14.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:31:14.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:31:14.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:14.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:31:14.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:31:14.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:14.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:31:14.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:31:14.117 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:14.117 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:14.117 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:14.117 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:14.117 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:14.117 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:14.122 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:31:14.207 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:14.203+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:14.300 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:14.301 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:14.301 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:14.301 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:14.310 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:31:14.311 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:14.307+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:14.433 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:14.429+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:14.447 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:14.447 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:14.448 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:14.448 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:14.448 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:14.456 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:31:14.545 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:14.541+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:14.613 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:14.609+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:14.637 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:14.637 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:14.637 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:14.637 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:14.637 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.638 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:14.638 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:14.638 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 113.537902304, 2023-12-06T18:31:14.638 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:31:14.638 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:14.638 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:14.638 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:14.638 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:14.638 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:14.638 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:31:14.638 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.638 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:14.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:14.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:14.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:14.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:14.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:14.639 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:14.639 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.639 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.639 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:14.639 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:31:14.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:31:14.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:14.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:31:14.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:31:14.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.640 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:14.640 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:31:14.640 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:31:14.640 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.640 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:14.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:31:14.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:31:14.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:14.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:31:14.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:31:14.641 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.641 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.641 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:14.641 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:31:14.641 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:31:14.642 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:14.642 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:14.642 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:14.642 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.642 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.642 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:31:14.642 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:14.642 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 53.519645001999997, 2023-12-06T18:31:14.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:31:14.642 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:14.642 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:14.642 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:14.643 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:14.643 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:14.643 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:31:14.643 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.643 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:14.643 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.643 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:14.643 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:14.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:14.643 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.643 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.643 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:14.643 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:14.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:14.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:14.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:31:14.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:31:14.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:14.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:31:14.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:31:14.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.645 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:14.645 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:31:14.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:31:14.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.645 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:14.645 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:31:14.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:31:14.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.645 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:14.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:31:14.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:31:14.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:14.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:14.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:14.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:31:14.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:31:14.646 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:14.646 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:14.646 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:14.646 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:14.646 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:14.646 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:14.648 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:31:14.714 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:14.710+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:14.787 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:14.788 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:14.788 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:14.788 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:14.797 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:31:14.815 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:14.811+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:14.915 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:14.912+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:14.965 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:14.966 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:14.966 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:14.966 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:14.966 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:14.975 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:31:15.019 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:15.015+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:15.106 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:15.106 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:15.106 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:15.106 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:15.106 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.106 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:15.107 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:15.107 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 173.61287686599999, 2023-12-06T18:31:15.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:31:15.107 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.107 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:15.107 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:15.107 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:15.107 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:15.107 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:31:15.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.107 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.107 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.107 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.107 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:15.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:15.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:31:15.108 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:31:15.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.108 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.108 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.108 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:31:15.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:31:15.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:31:15.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:31:15.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:31:15.109 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:31:15.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.109 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.109 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.109 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:31:15.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:31:15.110 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.110 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.110 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.110 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:31:15.110 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:31:15.110 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.110 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.110 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.110 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.110 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.110 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:15.115 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:31:15.117 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:15.113+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:15.218 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:15.214+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:15.284 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:15.284 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:15.285 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:15.285 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:15.293 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:31:15.334 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:15.331+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:15.419 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:15.416+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:15.466 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:15.467 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:15.467 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:15.467 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:15.467 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:15.479 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:31:15.520 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:15.516+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:15.621 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:15.618+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:15.689 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:15.690 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:15.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:15.690 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:15.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.690 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:31:15.690 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:15.690 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 259.07376107699997, 2023-12-06T18:31:15.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:31:15.690 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.690 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.690 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:15.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:15.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:31:15.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:31:15.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:31:15.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:31:15.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:31:15.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:31:15.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:31:15.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:31:15.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:31:15.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:31:15.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.693 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:31:15.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:31:15.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.693 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:31:15.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:31:15.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.693 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:31:15.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:31:15.694 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.694 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.694 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.694 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:31:15.694 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:15.694 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 258.050934068, 2023-12-06T18:31:15.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:31:15.694 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.694 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.694 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:15.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:15.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:31:15.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:31:15.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:31:15.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:31:15.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:31:15.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:31:15.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:31:15.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:31:15.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:31:15.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:31:15.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:31:15.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:31:15.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:31:15.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:31:15.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:31:15.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:31:15.698 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.698 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.698 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.698 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:31:15.698 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:15.698 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 258.03888742999999, 2023-12-06T18:31:15.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:31:15.698 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.698 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.698 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.698 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:15.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.699 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:15.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.699 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:31:15.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:31:15.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.699 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:31:15.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:31:15.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:31:15.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:31:15.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:31:15.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:31:15.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:31:15.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:31:15.701 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.701 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.701 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.701 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:31:15.701 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:31:15.701 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.701 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.701 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.701 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:31:15.701 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:31:15.701 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.701 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.701 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.701 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:31:15.701 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:31:15.702 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.702 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.702 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.702 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.702 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.702 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:31:15.702 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:15.702 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 258.01748682300001, 2023-12-06T18:31:15.702 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:31:15.702 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.702 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.702 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.702 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.702 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.703 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:15.703 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.703 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.703 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.703 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.703 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:15.703 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.703 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.703 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.703 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.703 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:31:15.703 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:31:15.703 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.703 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:31:15.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:31:15.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:31:15.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:31:15.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:31:15.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:31:15.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:31:15.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:31:15.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:31:15.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:31:15.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:31:15.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:31:15.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.706 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:31:15.706 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:31:15.706 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.706 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.706 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.706 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.706 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.706 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:31:15.706 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:15.706 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 258.01630749999998, 2023-12-06T18:31:15.706 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:31:15.706 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.706 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.706 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.706 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.707 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.707 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:15.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.707 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.707 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:15.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.707 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.707 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:31:15.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:31:15.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:31:15.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:31:15.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:31:15.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:31:15.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:31:15.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:31:15.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:31:15.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:31:15.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:31:15.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:31:15.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:31:15.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:31:15.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.710 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:31:15.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:31:15.710 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.710 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.710 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.710 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:31:15.710 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:15.710 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 258.01573551899997, 2023-12-06T18:31:15.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:31:15.710 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.710 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.710 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.711 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:15.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.711 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:15.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.711 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:31:15.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:31:15.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:31:15.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:31:15.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:31:15.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:31:15.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:31:15.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:31:15.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:31:15.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:31:15.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:31:15.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:31:15.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:31:15.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:31:15.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.714 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:31:15.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:31:15.714 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.714 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.714 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.714 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:31:15.714 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:15.714 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 257.56990004900001, 2023-12-06T18:31:15.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:31:15.714 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.714 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.714 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:15.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:15.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:31:15.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:31:15.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:31:15.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:31:15.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:31:15.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:31:15.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:31:15.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:31:15.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:31:15.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:31:15.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:31:15.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:31:15.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:31:15.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:31:15.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.718 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.718 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:31:15.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:31:15.718 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.718 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.718 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.718 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:31:15.718 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:15.718 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 257.56830019300003, 2023-12-06T18:31:15.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:31:15.719 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.719 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.719 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:15.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:15.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:31:15.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:31:15.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:31:15.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:31:15.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:31:15.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:31:15.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:31:15.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:31:15.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:31:15.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:31:15.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:31:15.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:31:15.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:31:15.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:31:15.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:31:15.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:31:15.722 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.722 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.722 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.722 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:31:15.722 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:15.723 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 257.18909457699999, 2023-12-06T18:31:15.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:31:15.723 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.723 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.723 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.723 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:15.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.723 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:15.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:31:15.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:31:15.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:31:15.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:31:15.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:31:15.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:31:15.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:31:15.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:31:15.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:31:15.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:31:15.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:31:15.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:31:15.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:31:15.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:31:15.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:31:15.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:31:15.726 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.726 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.726 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.726 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:31:15.726 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:15.727 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 256.95159239499998, 2023-12-06T18:31:15.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:31:15.727 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.727 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.727 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.727 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.727 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:15.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.727 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.727 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:15.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:31:15.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:31:15.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:31:15.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:31:15.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:31:15.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:31:15.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:31:15.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:31:15.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:31:15.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:31:15.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:31:15.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:31:15.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:31:15.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:31:15.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:31:15.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:31:15.730 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.730 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.730 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.730 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:31:15.730 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:15.731 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 256.95086439599999, 2023-12-06T18:31:15.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:31:15.731 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.731 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.731 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:15.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:15.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:31:15.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:31:15.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:31:15.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:31:15.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:31:15.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:31:15.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:31:15.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:31:15.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:31:15.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:31:15.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:31:15.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:31:15.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:31:15.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:31:15.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:31:15.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:31:15.734 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.734 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.734 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.734 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:31:15.735 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:15.735 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 256.94585198599998, 2023-12-06T18:31:15.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:31:15.735 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.735 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.735 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:15.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:15.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:31:15.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:31:15.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:31:15.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:31:15.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:31:15.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:31:15.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:31:15.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:31:15.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:31:15.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:31:15.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:31:15.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:31:15.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:31:15.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:31:15.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:31:15.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:31:15.738 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.738 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.738 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.739 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:31:15.739 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:15.739 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 256.94032490699999, 2023-12-06T18:31:15.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:31:15.739 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.739 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.739 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:15.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:15.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:31:15.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:31:15.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:31:15.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:31:15.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:31:15.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:31:15.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:31:15.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:31:15.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:31:15.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:31:15.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:31:15.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:31:15.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:31:15.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:31:15.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:31:15.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:31:15.742 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.742 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.742 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.743 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:31:15.743 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:15.743 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 256.93981407000001, 2023-12-06T18:31:15.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:31:15.743 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.743 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.743 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:15.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:15.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:31:15.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:31:15.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:31:15.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:31:15.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:31:15.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:31:15.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:31:15.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:31:15.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:31:15.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:31:15.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:31:15.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:31:15.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:31:15.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:31:15.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:31:15.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:31:15.747 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.747 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.747 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.748 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:31:15.748 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:15.748 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 256.93935480900001, 2023-12-06T18:31:15.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:31:15.748 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.748 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.748 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:15.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:15.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:31:15.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:31:15.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:31:15.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:31:15.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:31:15.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:31:15.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.750 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:31:15.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:31:15.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.750 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:31:15.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:31:15.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:31:15.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:31:15.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:31:15.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:31:15.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:31:15.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:31:15.752 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.752 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.752 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.752 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:31:15.752 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:15.752 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 250.35943900199999, 2023-12-06T18:31:15.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:31:15.752 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.752 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.752 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:15.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:15.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:31:15.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:31:15.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:31:15.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:31:15.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:31:15.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:31:15.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:31:15.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:31:15.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:31:15.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:31:15.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:31:15.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:31:15.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:31:15.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:31:15.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:31:15.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:31:15.756 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.756 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.756 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.756 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:31:15.756 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:15.756 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 250.353201197, 2023-12-06T18:31:15.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:31:15.756 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.756 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.756 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:15.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:15.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:31:15.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:31:15.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:31:15.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:31:15.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:31:15.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:31:15.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:31:15.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:31:15.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:31:15.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:31:15.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:31:15.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:31:15.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:31:15.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:31:15.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.760 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:31:15.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:31:15.760 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.760 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.760 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.760 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:31:15.760 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:15.760 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 250.347138118, 2023-12-06T18:31:15.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:31:15.760 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.760 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.760 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:15.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:15.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:31:15.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:31:15.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:31:15.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:31:15.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:31:15.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:31:15.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:31:15.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:31:15.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:31:15.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:31:15.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:31:15.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:31:15.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:31:15.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:31:15.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:31:15.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:31:15.764 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.764 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.764 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.764 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:31:15.764 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:15.764 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 250.209751033, 2023-12-06T18:31:15.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:31:15.764 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.765 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.765 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:15.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:15.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:31:15.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:31:15.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:31:15.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:31:15.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:31:15.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:31:15.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:31:15.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:31:15.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:31:15.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:31:15.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:31:15.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:31:15.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:31:15.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:31:15.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.768 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.768 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:31:15.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:31:15.768 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.768 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.768 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.768 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:31:15.768 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:15.768 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 250.203080306, 2023-12-06T18:31:15.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:31:15.768 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:15.769 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:15.769 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:15.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:15.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:15.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:15.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:15.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:15.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:15.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:31:15.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:31:15.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:15.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:31:15.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:31:15.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:15.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:31:15.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:31:15.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:15.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:31:15.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:31:15.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:15.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:31:15.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:31:15.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:15.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:31:15.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:31:15.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:15.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:31:15.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:31:15.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:15.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:15.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:15.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:31:15.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:31:15.772 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.772 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.772 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.772 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:15.772 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:15.772 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:15.774 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:31:15.774 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:15.770+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:15.874 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:15.871+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:15.946 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:15.946 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:15.946 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:15.946 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:15.955 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:31:15.996 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:15.993+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:16.076 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:16.073+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:16.126 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:16.126 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:16.126 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:16.126 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:16.126 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:16.135 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:31:16.177 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:16.174+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:16.278 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:16.274+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:16.306 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:16.306 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:16.306 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:16.306 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:31:16.306 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:16.314 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:31:16.378 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:16.375+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:16.442 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:16.442 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:16.442 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:16.442 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:16.451 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:31:16.480 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:16.476+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:16.579 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:16.579 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:16.580 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:16.580 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:16.580 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:16.580 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:16.577+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:16.588 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:31:16.681 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:16.677+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:16.717 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:16.718 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:16.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:16.718 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:16.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:16.718 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:31:16.718 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:16.718 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 55.056184598000002, 2023-12-06T18:31:16.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:31:16.718 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:16.718 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:16.718 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:16.719 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:16.719 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:16.719 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:31:16.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:16.719 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:16.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:16.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:16.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:16.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:16.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:16.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:16.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:16.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:16.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:16.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:16.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:16.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:16.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:31:16.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:31:16.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:16.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:16.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:16.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:31:16.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:31:16.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:16.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:16.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:16.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:31:16.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:31:16.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:16.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:16.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:16.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:31:16.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:31:16.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:16.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:16.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:16.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:31:16.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:31:16.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:16.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:16.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:16.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:31:16.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:31:16.721 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:16.721 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:16.722 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:16.722 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:16.722 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:16.722 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:16.727 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:31:16.782 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:16.779+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:16.877 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:16.877 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:16.877 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:16.877 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:16.883 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:16.880+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:16.886 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:31:16.984 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:16.980+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:17.063 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:17.063 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:17.064 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:17.064 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:17.064 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:17.074 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:31:17.119 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:17.115+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:17.196 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:17.192+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:17.210 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:17.211 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:17.211 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:17.211 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:31:17.211 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:17.220 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:31:17.305 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:17.301+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:17.388 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:17.384+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:17.397 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:17.397 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:17.397 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:17.397 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:17.407 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:31:17.493 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:17.489+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:17.587 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:17.587 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:17.587 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:17.587 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:17.587 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:17.589 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:17.585+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:17.599 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:31:17.690 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:17.687+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:17.791 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:17.787+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:17.839 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:17.839 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:17.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:31:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 262.42853606099999, 2023-12-06T18:31:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:31:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:31:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:17.841 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.841 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.841 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.841 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.841 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:17.841 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.841 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.841 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.841 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.841 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:31:17.841 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:31:17.841 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.841 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.841 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.841 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:31:17.841 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:31:17.841 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.842 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.842 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.842 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:31:17.842 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:31:17.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.842 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.842 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.842 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:31:17.842 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:31:17.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.842 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.842 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.842 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:31:17.842 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:31:17.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.842 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:31:17.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:31:17.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:31:17.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:31:17.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:31:17.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:31:17.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:31:17.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:31:17.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:31:17.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:31:17.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:31:17.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:31:17.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:31:17.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:31:17.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:31:17.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.845 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:31:17.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:31:17.845 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.845 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.845 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.845 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:17.845 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:17.845 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 260.31996282199998, 2023-12-06T18:31:17.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:31:17.845 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.845 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.845 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.846 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.846 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.846 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:31:17.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.846 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.846 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.846 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.846 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:17.846 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.846 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.846 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.846 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:17.846 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.846 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.847 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.847 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:31:17.847 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:31:17.847 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.847 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.847 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.847 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:31:17.847 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:31:17.847 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.847 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.847 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.847 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:31:17.847 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:31:17.847 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:31:17.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:31:17.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:31:17.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:31:17.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:31:17.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:31:17.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:31:17.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:31:17.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:31:17.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:31:17.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:31:17.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:31:17.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:31:17.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:31:17.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:31:17.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:31:17.850 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.850 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.850 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.850 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:17.850 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:17.851 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 260.31921758999999, 2023-12-06T18:31:17.851 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:31:17.851 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.851 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.851 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.851 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.851 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.851 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:31:17.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.851 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.851 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:17.851 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.852 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:17.852 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.852 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.852 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.852 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.852 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:31:17.852 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:31:17.852 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.852 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.852 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.852 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:31:17.852 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:31:17.852 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.852 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.852 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.852 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:31:17.853 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:31:17.853 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.853 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.853 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.853 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:31:17.853 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:31:17.853 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.853 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.853 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.853 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:31:17.853 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:31:17.853 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.853 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.853 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.853 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:31:17.853 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:31:17.854 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.854 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.854 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.854 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:31:17.854 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:31:17.854 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.854 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.854 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.854 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:31:17.854 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:31:17.854 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.854 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.854 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.854 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:31:17.854 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:31:17.854 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.855 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.855 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.855 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:31:17.855 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:31:17.855 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.855 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.855 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.855 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:31:17.855 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:31:17.855 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.855 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.855 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.855 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.855 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.855 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:17.855 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:17.855 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 260.31853383499998, 2023-12-06T18:31:17.856 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:31:17.856 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.856 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.856 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.856 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.856 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.856 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:31:17.856 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.856 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.856 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.856 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.856 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:17.856 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.856 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.856 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.857 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.857 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:17.857 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.857 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.857 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.857 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.857 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:31:17.857 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:31:17.857 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.857 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.857 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.857 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:31:17.857 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:31:17.857 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.857 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.857 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.858 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:31:17.858 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:31:17.858 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.858 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.858 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.858 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:31:17.858 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:31:17.858 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.858 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.858 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.858 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:31:17.858 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:31:17.858 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.858 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.859 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:31:17.859 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:31:17.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.859 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:31:17.859 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:31:17.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.859 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:31:17.859 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:31:17.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.860 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.860 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:31:17.860 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:31:17.860 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.860 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.860 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.860 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:31:17.860 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:31:17.860 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.860 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.860 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.860 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:31:17.860 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:31:17.860 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.860 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.861 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.861 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.861 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.861 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:17.861 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:17.861 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 260.31296057999998, 2023-12-06T18:31:17.861 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:31:17.861 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.861 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.861 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.861 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.861 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.861 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:31:17.861 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.861 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.861 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.861 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.862 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:17.862 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.862 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.862 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.862 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.862 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:17.862 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.862 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.862 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.862 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.862 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:31:17.862 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:31:17.862 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.862 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.862 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.862 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:31:17.863 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:31:17.863 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.863 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.863 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.863 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:31:17.863 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:31:17.863 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.863 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.863 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.863 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:31:17.863 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:31:17.863 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.863 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.863 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.863 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:31:17.863 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:31:17.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.864 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.864 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.864 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:31:17.864 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:31:17.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.864 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.864 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.864 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:31:17.864 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:31:17.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.864 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.864 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.864 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:31:17.864 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:31:17.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.865 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.865 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.865 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:31:17.865 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:31:17.865 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.865 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.865 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.865 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:31:17.865 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:31:17.865 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.865 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.865 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.865 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:31:17.865 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:31:17.865 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.865 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.866 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.866 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.866 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.866 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:17.866 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:17.866 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 260.31228693999998, 2023-12-06T18:31:17.866 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:31:17.866 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.866 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.866 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.866 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.866 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.866 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:31:17.866 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.866 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.867 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.867 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.867 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:17.867 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.867 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.867 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.867 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.867 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:17.867 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.867 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.867 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.867 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.867 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:31:17.867 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:31:17.867 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.867 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.868 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.868 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:31:17.868 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:31:17.868 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.868 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.868 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.868 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:31:17.868 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:31:17.868 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.868 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.868 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.868 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:31:17.868 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:31:17.868 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.868 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.869 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.869 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:31:17.869 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:31:17.869 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.869 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.869 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.869 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:31:17.869 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:31:17.869 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.869 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.869 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.869 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:31:17.869 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:31:17.869 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.869 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.870 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.870 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:31:17.870 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:31:17.870 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.870 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.870 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.870 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:31:17.870 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:31:17.870 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.870 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.870 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.870 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:31:17.870 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:31:17.870 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.870 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.871 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.871 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:31:17.871 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:31:17.871 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.871 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.871 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.871 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.871 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.871 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:17.871 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:17.871 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 260.311624741, 2023-12-06T18:31:17.871 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:31:17.871 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.871 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.871 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.871 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.872 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.872 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:31:17.872 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.872 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.872 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.872 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.872 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:17.872 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.872 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.872 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.872 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.872 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:17.872 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.872 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.872 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.872 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.873 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:31:17.873 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:31:17.873 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.873 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.873 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.873 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:31:17.873 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:31:17.873 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.873 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.873 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.873 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:31:17.873 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:31:17.873 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.873 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.873 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.873 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:31:17.874 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:31:17.874 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.874 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.874 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.874 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:31:17.874 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:31:17.874 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.874 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.874 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.874 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:31:17.874 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:31:17.874 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.874 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.874 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.874 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:31:17.875 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:31:17.875 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.875 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.875 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.875 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:31:17.875 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:31:17.875 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.875 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.875 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.875 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:31:17.875 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:31:17.875 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.875 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.875 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.875 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:31:17.875 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:31:17.876 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.876 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.876 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.876 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:31:17.876 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:31:17.876 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.876 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.876 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.876 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.876 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.876 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:17.876 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:17.876 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 260.23891530600002, 2023-12-06T18:31:17.876 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:31:17.876 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.876 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.877 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.877 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.877 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.877 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:31:17.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.877 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.877 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.877 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.877 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:17.877 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.877 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.877 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.877 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:17.877 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.878 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.878 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.878 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:31:17.878 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:31:17.878 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.878 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.878 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.878 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:31:17.878 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:31:17.878 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.878 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.878 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.878 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:31:17.878 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:31:17.878 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.879 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.879 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.879 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:31:17.879 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:31:17.879 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.879 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.879 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.879 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:31:17.879 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:31:17.879 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.879 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.879 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.879 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:31:17.879 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:31:17.879 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.879 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.880 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.880 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:31:17.880 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:31:17.880 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.880 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.880 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.880 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:31:17.880 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:31:17.880 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.880 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.880 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.880 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:31:17.880 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:31:17.880 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.880 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.880 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.881 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:31:17.881 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:31:17.881 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.881 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.881 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.881 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:31:17.881 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:31:17.881 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.881 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.881 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.881 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.881 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.881 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:17.881 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:17.881 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 260.23823359699998, 2023-12-06T18:31:17.881 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:31:17.882 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.882 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.882 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.882 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.882 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.882 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:31:17.882 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.882 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.882 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.882 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.882 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:17.882 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.882 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.882 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.882 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.883 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:17.883 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.883 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.883 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.883 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.883 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:31:17.883 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:31:17.883 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.883 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.883 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.883 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:31:17.883 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:31:17.883 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.883 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.883 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.883 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:31:17.884 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:31:17.884 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.884 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.884 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.884 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:31:17.884 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:31:17.884 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.884 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.884 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.884 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:31:17.884 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:31:17.884 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.884 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.884 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.884 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:31:17.884 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:31:17.884 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.885 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.885 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.885 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:31:17.885 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:31:17.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.885 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.885 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.885 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:31:17.885 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:31:17.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.885 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.885 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.885 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:31:17.885 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:31:17.885 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.886 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.886 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.886 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:31:17.886 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:31:17.886 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.886 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.886 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.886 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:31:17.886 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:31:17.886 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.886 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.886 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.886 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.886 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.886 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:17.887 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:17.887 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 259.15942950900001, 2023-12-06T18:31:17.887 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:31:17.887 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.887 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.887 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.887 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.887 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.887 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:31:17.887 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.887 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.887 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.887 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.887 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:17.887 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.888 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.888 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.888 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.888 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:17.888 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.888 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.888 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.888 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.888 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:31:17.888 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:31:17.888 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.888 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.888 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.888 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:31:17.888 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:31:17.888 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.889 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.889 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.889 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:31:17.889 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:31:17.889 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.889 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.889 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.889 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:31:17.889 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:31:17.889 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.889 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.889 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.889 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:31:17.889 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:31:17.889 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.889 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.890 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.890 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:31:17.890 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:31:17.890 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.890 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.890 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.890 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:31:17.890 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:31:17.890 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.890 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.890 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.890 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:31:17.890 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:31:17.890 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.890 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.891 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:31:17.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:31:17.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.891 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.891 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:31:17.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:31:17.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.891 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.891 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:31:17.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:31:17.891 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.891 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.892 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.892 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:17.892 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:17.892 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 259.15801293499999, 2023-12-06T18:31:17.892 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:31:17.892 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.892 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.892 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.892 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.892 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.892 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:31:17.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.892 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.892 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.893 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:17.893 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.893 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.893 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.893 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.893 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:17.893 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.893 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.893 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.893 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.893 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:31:17.893 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:31:17.893 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.893 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.894 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:31:17.894 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:31:17.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.894 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:31:17.894 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:31:17.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.894 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:31:17.894 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:31:17.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.895 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.895 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:31:17.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:31:17.895 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.895 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.895 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.895 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:31:17.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:31:17.895 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.895 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.895 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.895 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:31:17.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:31:17.895 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.895 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.896 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.896 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:31:17.896 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:31:17.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.896 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.896 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.896 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:31:17.896 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:31:17.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.896 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.896 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.896 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:31:17.896 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:31:17.896 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.896 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.896 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.897 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:31:17.897 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:31:17.897 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.897 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.897 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.897 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.897 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.897 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:17.897 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:17.897 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 259.15692781000001, 2023-12-06T18:31:17.897 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:31:17.897 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.897 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.897 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.897 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.897 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.898 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:31:17.898 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.898 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.898 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.898 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.898 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:17.898 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.898 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.898 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.898 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.898 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:17.898 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.898 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.898 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.898 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.899 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:31:17.899 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:31:17.899 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.899 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.899 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.899 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:31:17.899 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:31:17.899 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.899 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.899 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.899 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:31:17.899 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:31:17.899 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.899 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.899 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.899 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:31:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:31:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:31:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:31:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:31:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:31:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:31:17.900 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:31:17.901 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.901 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.901 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.901 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:31:17.901 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:31:17.901 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.901 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.901 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.901 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:31:17.901 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:31:17.901 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.901 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.901 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.901 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:31:17.901 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:31:17.902 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.902 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.902 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.902 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:31:17.902 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:31:17.902 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.902 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.902 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.902 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.902 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.902 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:17.902 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:17.902 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 259.12177870800002, 2023-12-06T18:31:17.902 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:31:17.902 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.903 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.903 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.903 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.903 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.903 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:31:17.903 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.903 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.903 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.903 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.903 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:17.903 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.903 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.903 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.903 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.903 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:17.904 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.904 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.904 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.904 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.904 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:31:17.904 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:31:17.904 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.904 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.904 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.904 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:31:17.904 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:31:17.904 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.904 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.904 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.904 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:31:17.904 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:31:17.905 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.905 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.905 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.905 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:31:17.905 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:31:17.905 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.905 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.905 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.905 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:31:17.905 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:31:17.905 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.905 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.905 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.905 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:31:17.905 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:31:17.905 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.906 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.906 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.906 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:31:17.906 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:31:17.906 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.906 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.906 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.906 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:31:17.906 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:31:17.906 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.906 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.906 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.906 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:31:17.906 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:31:17.906 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.907 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.907 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.907 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:31:17.907 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:31:17.907 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.907 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.907 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.907 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:31:17.907 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:31:17.907 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.907 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.907 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.907 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.907 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.907 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:17.907 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:17.908 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 252.57204818299999, 2023-12-06T18:31:17.908 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:31:17.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.908 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.908 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.908 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.908 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.908 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:31:17.908 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.908 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.908 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.909 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.909 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:17.909 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.909 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.909 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.909 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.909 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:17.909 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.909 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.909 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.909 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.909 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:31:17.909 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:31:17.909 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.909 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.909 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.910 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:31:17.910 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:31:17.910 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.910 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.910 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.910 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:31:17.910 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:31:17.910 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.910 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.910 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.910 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:31:17.910 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:31:17.910 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.910 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.910 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.911 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:31:17.911 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:31:17.911 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.911 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.911 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.911 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:31:17.911 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:31:17.911 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.911 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.911 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.911 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:31:17.911 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:31:17.911 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.911 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.911 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.911 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:31:17.912 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:31:17.912 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.912 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.912 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.912 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:31:17.912 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:31:17.912 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.912 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.912 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.912 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:31:17.912 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:31:17.912 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.912 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.912 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.912 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:31:17.913 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:31:17.913 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.913 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.913 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.913 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.913 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.913 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:17.913 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:17.913 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 252.57124257500001, 2023-12-06T18:31:17.913 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:31:17.913 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.913 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.913 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.913 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.913 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.913 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:31:17.914 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.914 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.914 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.914 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.914 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:17.914 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.914 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.914 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.914 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.914 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:17.914 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.914 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.914 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.914 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.914 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:31:17.915 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:31:17.915 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.915 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.915 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.915 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:31:17.915 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:31:17.915 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.915 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.915 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.915 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:31:17.915 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:31:17.915 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.915 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.915 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.915 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:31:17.915 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:31:17.916 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.916 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.916 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.916 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:31:17.916 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:31:17.916 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.916 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.916 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.916 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:31:17.916 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:31:17.916 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.916 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.916 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.916 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:31:17.916 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:31:17.916 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.917 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.917 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.917 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:31:17.917 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:31:17.917 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.917 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.917 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.917 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:31:17.917 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:31:17.917 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.917 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.917 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.917 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:31:17.917 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:31:17.917 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.917 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.918 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.918 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:31:17.918 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:31:17.918 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.918 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.918 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.918 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.918 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.918 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:17.918 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:17.918 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 252.57055176599999, 2023-12-06T18:31:17.918 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:31:17.918 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.918 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.918 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.919 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.919 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.919 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:31:17.919 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.919 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.919 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.919 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.919 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:17.919 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.919 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.919 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.919 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.919 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:17.919 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.919 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.919 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.920 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.920 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:31:17.920 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:31:17.920 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.920 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.920 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.920 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:31:17.920 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:31:17.920 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.920 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.920 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.920 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:31:17.920 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:31:17.920 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.920 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.921 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.921 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:31:17.921 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:31:17.921 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.921 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.921 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.921 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:31:17.921 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:31:17.921 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.921 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.921 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.921 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:31:17.921 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:31:17.921 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.921 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.922 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.922 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:31:17.922 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:31:17.922 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.922 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.922 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.922 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:31:17.922 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:31:17.922 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.922 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.922 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.922 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:31:17.922 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:31:17.922 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.922 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.923 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.923 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:31:17.923 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:31:17.923 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.923 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.923 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.923 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:31:17.923 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:31:17.923 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.923 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.923 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.923 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.923 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.923 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:17.923 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:17.923 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 252.56625425799999, 2023-12-06T18:31:17.923 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:31:17.924 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.924 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.924 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.924 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.924 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.924 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:31:17.924 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.924 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.924 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.924 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.924 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:17.924 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.924 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.924 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.924 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.924 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:17.925 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.925 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.925 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.925 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.925 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:31:17.925 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:31:17.925 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.925 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.925 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.925 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:31:17.925 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:31:17.925 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.925 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.925 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.926 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:31:17.926 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:31:17.926 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.926 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.926 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.926 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:31:17.926 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:31:17.926 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.926 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.926 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.926 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:31:17.926 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:31:17.926 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.926 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.926 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.927 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:31:17.927 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:31:17.927 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.927 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.927 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.927 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:31:17.927 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:31:17.927 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.927 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.927 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.927 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:31:17.927 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:31:17.927 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.927 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.927 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.928 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:31:17.928 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:31:17.928 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.928 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.928 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.928 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:31:17.928 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:31:17.928 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.928 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.928 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.928 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:31:17.928 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:31:17.928 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.928 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.928 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.928 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.929 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.929 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:17.929 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:17.929 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 252.565586583, 2023-12-06T18:31:17.929 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:31:17.929 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.929 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.929 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.929 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.929 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.929 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:31:17.929 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.929 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.929 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.929 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.929 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:17.930 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.930 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.930 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.930 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.930 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:17.930 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.930 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.930 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.930 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.930 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:31:17.930 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:31:17.930 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.930 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.930 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.930 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:31:17.931 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:31:17.931 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.931 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.931 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.931 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:31:17.931 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:31:17.931 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.931 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.931 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.931 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:31:17.931 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:31:17.931 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.931 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.931 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.931 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:31:17.931 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:31:17.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.932 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:31:17.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:31:17.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.932 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:31:17.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:31:17.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.932 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:31:17.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:31:17.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.933 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.933 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:31:17.933 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:31:17.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.933 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.933 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:31:17.933 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:31:17.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.933 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.933 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:31:17.933 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:31:17.933 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.934 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.934 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.934 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.934 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:17.934 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:17.934 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 252.56493115500001, 2023-12-06T18:31:17.934 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:31:17.934 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.934 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.934 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.934 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.934 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.934 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:31:17.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.934 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.935 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.935 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:17.935 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.935 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.935 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.935 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:17.935 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.935 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.935 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.935 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:31:17.935 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:31:17.935 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.936 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.936 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:31:17.936 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:31:17.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.936 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.936 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:31:17.936 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:31:17.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.936 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.936 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:31:17.936 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:31:17.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.937 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.937 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:31:17.937 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:31:17.937 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.937 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.937 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.937 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:31:17.937 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:31:17.937 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.937 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.937 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.937 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:31:17.937 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:31:17.937 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.937 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.938 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.938 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:31:17.938 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:31:17.938 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.938 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.938 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.938 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:31:17.938 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:31:17.938 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.938 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.938 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.938 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:31:17.938 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:31:17.938 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.938 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.939 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.939 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:31:17.939 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:31:17.939 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.939 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.939 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.939 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.939 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.939 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:17.939 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:17.939 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 252.52427660800001, 2023-12-06T18:31:17.939 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:31:17.939 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:17.939 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:17.939 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:17.939 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:17.940 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:17.940 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:31:17.940 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.940 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:17.940 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.940 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:17.940 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:17.940 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.940 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.940 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.940 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:17.940 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:17.940 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:17.940 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.940 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.941 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:17.941 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:31:17.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:31:17.941 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.941 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:17.941 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:31:17.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:31:17.941 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.941 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:17.941 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:31:17.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:31:17.941 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.941 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:17.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:31:17.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:31:17.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:17.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:31:17.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:31:17.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:17.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:31:17.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:31:17.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:17.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:31:17.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:31:17.943 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.943 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:17.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:31:17.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:31:17.943 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.943 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:17.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:31:17.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:31:17.943 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.943 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:17.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:31:17.944 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:31:17.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:17.944 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:17.944 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:17.944 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:31:17.944 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:31:17.944 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.944 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.944 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.944 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:17.944 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:17.944 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:17.946 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:31:17.946 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:17.942+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:18.046 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:18.043+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:18.123 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:18.123 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:18.132 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:31:18.145 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:31:18.145 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:31:18.147 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:18.143+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:18.241 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:31:18.242 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:31:18.248 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:18.245+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:18.349 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:18.345+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:18.357 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:31:18.357 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:31:18.358 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:18.359 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:18.368 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:31:18.450 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:18.446+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:18.455 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:31:18.456 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:31:18.562 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:18.558+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:18.563 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:31:18.563 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:31:18.563 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:18.563 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:18.572 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:31:18.652 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:31:18.652 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:31:18.653 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:18.648+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:18.714 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:18.714 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:18.724 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:31:18.739 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:31:18.739 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:31:18.752 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:18.748+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:18.826 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:31:18.826 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:31:18.852 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:18.849+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:18.878 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:18.878 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:18.888 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:31:18.954 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:18.950+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:19.027 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:19.027 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:19.036 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:31:19.078 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:19.075+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:19.156 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:19.152+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:19.178 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:19.178 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:19.190 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:31:19.257 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:19.253+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:19.274 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is unset 2023-12-06T18:31:19.357 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:19.354+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:19.362 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:19.362 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:19.371 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:31:19.458 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:19.455+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:19.542 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:19.542 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:19.551 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:31:19.559 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:19.555+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:19.660 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:19.656+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:19.682 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:19.682 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:19.692 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:31:19.761 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:19.758+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:19.824 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:19.824 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:19.833 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:31:19.862 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:19.858+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:19.963 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:19.960+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:20.014 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:20.014 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:20.024 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:31:20.065 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:20.062+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:20.166 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:20.162+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:20.184 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:20.184 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:20.194 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:31:20.267 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:20.263+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:20.327 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:20.327 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:20.337 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:31:20.368 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:20.364+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:20.434 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":34,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:31:20.448 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2023-12-06T18:31:20.448 INFO:tasks.ceph.mon.a:Restarting daemon 2023-12-06T18:31:20.448 DEBUG:teuthology.orchestra.run.smithi061:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2023-12-06T18:31:20.471 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:20.471 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:20.481 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:31:20.488 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:20.486+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:20.489 INFO:tasks.ceph.mon.a:Started 2023-12-06T18:31:20.489 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2023-12-06T18:31:20.490 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:31:20.569 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:20.566+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:20.610 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:20.610 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:20.620 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:31:20.670 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:20.666+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:20.771 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:20.768+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:20.790 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:20.790 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:20.800 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:31:20.837 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:31:20.881 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:20.877+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:20.971 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:20.971 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:20.973 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:20.969+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:20.981 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:31:21.069 INFO:teuthology.orchestra.run.smithi061.stderr:set osd.1 primary-affinity to 1 (8655362) 2023-12-06T18:31:21.073 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:21.069+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:21.174 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:21.171+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:21.177 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:31:21.177 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:31:21.178 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:21.178 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:21.178 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:21.178 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:21.188 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:31:21.254 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":34,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":3,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:31:21.287 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-12-06T18:31:21.331 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:31:21.332 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:31:21.332 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:21.329+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:21.365 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:21.365 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:21.365 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:21.365 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:21.365 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:21.375 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:31:21.376 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:21.373+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:21.450 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:31:21.451 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:31:21.498 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:21.495+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:21.540 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:21.540 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:21.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:21.540 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:31:21.540 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:21.550 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:31:21.563 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:31:21.563 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:31:21.607 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:21.604+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:21.653 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:31:21.653 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:31:21.681 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:21.677+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:21.700 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:21.700 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:21.700 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:21.700 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:21.709 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:31:21.743 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:31:21.744 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:31:21.781 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:21.777+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:21.837 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:31:21.837 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:31:21.882 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:21.880+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:21.896 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:21.896 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:21.896 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:21.896 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:21.896 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:21.905 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:31:21.937 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:31:21.937 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:31:21.983 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:21.980+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:22.084 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:22.082+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:22.138 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:22.138 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:22.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:22.138 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:22.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.138 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:31:22.138 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:22.138 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 266.72687245499998, 2023-12-06T18:31:22.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:31:22.138 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.139 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.139 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.139 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.139 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.139 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:31:22.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.139 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:22.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:22.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:31:22.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:31:22.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:31:22.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:31:22.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:31:22.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:31:22.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:31:22.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:31:22.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:31:22.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:31:22.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:31:22.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:31:22.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:31:22.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:31:22.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:31:22.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.143 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:31:22.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:31:22.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.143 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:31:22.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:31:22.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.143 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:31:22.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:31:22.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:31:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:31:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:31:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:31:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:31:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:31:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.144 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.145 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.145 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:22.145 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:22.145 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 264.61829921600003, 2023-12-06T18:31:22.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:31:22.145 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.145 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.145 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.145 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.145 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.145 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:31:22.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.146 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:22.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:22.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:31:22.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:31:22.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:31:22.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:31:22.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:31:22.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:31:22.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:31:22.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:31:22.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:31:22.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:31:22.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:31:22.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:31:22.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:31:22.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:31:22.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:31:22.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:31:22.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.151 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:31:22.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:31:22.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.151 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:31:22.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:31:22.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.151 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:31:22.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:31:22.151 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.151 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.151 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.152 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:22.152 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:22.152 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 264.61755398399998, 2023-12-06T18:31:22.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:31:22.152 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.152 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.152 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.152 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.152 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.152 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:31:22.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.152 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.153 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.153 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:22.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.153 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.153 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:22.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.153 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.153 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:31:22.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:31:22.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:31:22.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:31:22.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:31:22.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:31:22.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:31:22.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:31:22.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:31:22.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:31:22.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:31:22.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:31:22.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:31:22.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:31:22.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:31:22.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:31:22.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:31:22.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:31:22.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:31:22.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:31:22.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:31:22.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:31:22.157 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.157 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.157 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.157 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:22.157 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:22.157 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 264.61687022900003, 2023-12-06T18:31:22.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:31:22.158 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.158 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.158 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.158 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.158 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.158 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:31:22.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.158 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.158 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:22.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.159 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.159 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:22.159 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.159 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.159 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:31:22.159 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:31:22.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.159 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.159 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:31:22.159 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:31:22.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:31:22.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:31:22.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:31:22.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:31:22.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:31:22.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:31:22.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:31:22.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:31:22.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:31:22.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:31:22.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:31:22.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:31:22.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.162 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:31:22.162 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:31:22.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.162 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.162 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:31:22.162 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:31:22.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.162 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.162 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:31:22.162 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:31:22.162 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.163 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.163 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.163 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.163 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.163 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:22.163 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:22.163 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 264.61129697400003, 2023-12-06T18:31:22.163 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:31:22.163 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.163 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.163 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.163 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:31:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:31:22.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:31:22.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:31:22.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:31:22.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:31:22.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:31:22.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.166 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.166 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:31:22.166 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:31:22.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.166 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.166 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.166 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:31:22.166 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:31:22.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.166 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.166 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.166 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:31:22.166 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:31:22.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.167 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.167 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.167 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:31:22.167 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:31:22.167 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.167 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.167 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.167 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:31:22.167 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:31:22.167 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.167 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.167 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.167 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:31:22.167 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:31:22.168 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.168 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.168 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.168 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:31:22.168 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:31:22.168 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.168 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.168 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.168 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:31:22.168 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:31:22.168 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.168 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.168 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.168 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.169 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.169 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:22.169 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:22.169 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 264.61062333400002, 2023-12-06T18:31:22.169 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:31:22.169 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.169 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.169 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.169 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.169 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:31:22.169 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.169 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.169 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.169 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.170 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:22.170 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.170 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.170 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.170 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:22.170 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.170 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.170 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.170 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:31:22.170 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:31:22.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.171 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.171 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.171 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:31:22.171 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:31:22.171 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.171 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.171 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.171 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:31:22.171 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:31:22.171 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.171 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.171 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.171 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:31:22.171 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:31:22.171 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.172 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.172 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.172 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:31:22.172 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:31:22.172 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.172 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.172 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.172 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:31:22.172 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:31:22.172 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.172 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.172 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.172 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:31:22.172 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:31:22.172 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.173 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.173 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.173 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:31:22.173 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:31:22.173 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.173 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.173 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.173 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:31:22.173 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:31:22.173 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.173 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.173 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.173 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:31:22.173 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:31:22.173 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.174 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.174 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.174 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:31:22.174 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:31:22.174 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.174 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.174 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.174 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.174 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.174 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:22.174 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:22.174 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 264.60996113499999, 2023-12-06T18:31:22.175 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:31:22.175 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.175 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.175 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.175 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.175 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.175 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:31:22.175 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.175 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.175 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.175 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.175 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:31:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:31:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:31:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:31:22.176 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.177 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.177 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.177 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:31:22.177 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:31:22.177 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.177 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.177 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.177 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:31:22.177 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:31:22.177 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.177 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.177 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.177 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:31:22.177 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:31:22.177 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.177 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.178 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.178 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:31:22.178 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:31:22.178 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.178 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.178 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.178 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:31:22.178 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:31:22.178 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.178 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.178 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.178 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:31:22.179 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:31:22.179 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.179 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.179 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.179 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:31:22.179 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:31:22.179 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.179 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.179 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.179 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:31:22.179 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:31:22.179 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.179 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.179 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.179 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:31:22.180 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:31:22.180 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.180 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.180 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.180 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.180 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.180 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:22.180 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:22.180 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 264.53725170000001, 2023-12-06T18:31:22.180 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:31:22.180 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.180 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.180 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.180 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.180 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.181 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:31:22.181 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.181 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.181 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.181 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.181 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:22.181 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.181 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.181 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.181 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.181 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:22.181 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.181 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.181 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.181 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.181 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:31:22.181 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:31:22.182 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.182 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.182 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.182 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:31:22.182 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:31:22.182 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.182 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.182 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.182 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:31:22.182 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:31:22.182 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.182 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.182 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.182 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:31:22.183 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:31:22.183 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.183 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.183 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.183 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:31:22.183 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:31:22.183 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.183 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.183 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.183 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:31:22.183 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:31:22.183 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.183 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.183 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.183 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:31:22.183 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:31:22.183 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.184 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.184 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.184 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:31:22.184 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:31:22.184 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.184 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.184 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.184 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:31:22.184 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:31:22.184 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.184 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.184 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.184 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:31:22.185 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:31:22.185 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.185 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.185 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.185 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:31:22.185 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:31:22.185 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.185 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.185 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.185 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.185 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.185 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:22.185 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:22.185 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 264.53656999100002, 2023-12-06T18:31:22.185 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:31:22.186 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.186 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.186 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.186 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.186 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.186 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:31:22.186 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.186 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.186 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.186 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.186 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:22.186 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.186 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:31:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:31:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:31:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:31:22.187 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:31:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:31:22.188 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.189 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.189 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.189 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:31:22.189 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:31:22.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.189 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.189 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.189 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:31:22.189 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:31:22.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.189 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.189 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.189 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:31:22.189 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:31:22.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.190 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.190 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.190 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:31:22.190 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:31:22.190 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.190 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.190 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.190 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:31:22.190 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:31:22.190 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.190 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.190 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.190 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:31:22.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:31:22.191 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.191 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.191 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.191 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:31:22.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:31:22.191 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.191 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.191 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.191 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:31:22.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:31:22.191 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.191 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.191 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.191 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.191 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.192 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:22.192 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:22.192 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 263.45776590299999, 2023-12-06T18:31:22.192 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:31:22.192 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.192 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.192 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.192 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.192 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.192 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:31:22.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.192 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.192 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.192 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.192 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:22.193 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.193 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.193 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.193 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.193 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:22.193 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.193 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.193 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.193 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.193 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:31:22.193 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:31:22.193 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.193 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.193 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.193 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:31:22.193 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:31:22.194 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.194 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.194 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.194 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:31:22.194 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:31:22.194 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.194 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.194 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.194 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:31:22.194 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:31:22.194 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.194 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.194 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.194 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:31:22.195 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:31:22.195 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.195 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.195 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.195 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:31:22.195 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:31:22.195 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.195 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.195 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.195 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:31:22.195 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:31:22.195 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.195 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.195 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.195 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:31:22.195 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:31:22.195 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.196 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.196 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.196 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:31:22.196 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:31:22.196 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.196 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.196 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.196 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:31:22.196 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:31:22.196 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.196 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.196 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.196 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:31:22.196 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:31:22.196 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.196 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.197 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.197 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.197 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.197 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:22.197 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:22.197 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 263.45634932899998, 2023-12-06T18:31:22.197 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:31:22.197 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.197 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.197 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.197 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.197 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.197 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:31:22.197 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.198 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.198 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.198 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.198 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:22.198 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.198 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.198 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.198 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.198 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:22.198 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.198 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.198 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.198 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.198 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:31:22.198 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:31:22.199 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.199 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.199 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.199 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:31:22.199 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:31:22.199 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.199 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.199 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.199 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:31:22.199 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:31:22.199 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.199 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.199 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.199 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:31:22.199 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:31:22.200 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.200 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.200 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.200 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:31:22.200 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:31:22.200 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.200 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.200 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.200 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:31:22.200 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:31:22.200 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.200 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.200 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.200 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:31:22.200 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:31:22.200 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.201 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.201 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.201 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:31:22.201 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:31:22.201 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.201 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.201 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.201 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:31:22.201 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:31:22.201 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.201 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.201 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.201 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:31:22.201 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:31:22.201 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.201 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.202 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.202 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:31:22.202 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:31:22.202 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.202 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.202 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.202 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.202 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.202 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:22.202 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:22.202 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 263.455264204, 2023-12-06T18:31:22.202 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:31:22.202 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.202 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.202 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.203 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.203 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.203 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:31:22.203 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.203 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.203 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.203 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.203 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:22.203 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.203 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.203 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.203 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.203 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:22.203 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.203 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.203 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.204 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.204 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:31:22.204 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:31:22.204 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.204 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.204 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.204 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:31:22.204 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:31:22.204 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.204 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.204 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.204 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:31:22.204 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:31:22.204 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.204 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.204 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.205 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:31:22.205 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:31:22.205 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.205 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.205 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.205 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:31:22.205 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:31:22.205 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.205 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.205 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.205 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:31:22.205 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:31:22.205 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.205 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.205 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.206 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:31:22.206 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:31:22.206 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.206 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.206 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.206 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:31:22.206 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:31:22.206 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.206 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.206 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.206 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:31:22.206 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:31:22.206 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.206 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.206 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.207 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:31:22.207 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:31:22.207 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.207 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.207 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.207 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:31:22.207 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:31:22.207 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.207 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.207 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.207 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.207 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.207 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:22.207 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:22.207 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 263.42011510200001, 2023-12-06T18:31:22.208 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:31:22.208 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.208 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.208 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.208 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.208 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.208 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:31:22.208 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.208 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.208 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.208 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.208 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:22.208 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.208 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.208 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.209 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.209 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:22.209 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.209 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.209 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.209 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.209 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:31:22.209 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:31:22.209 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.209 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.209 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.209 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:31:22.209 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:31:22.209 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.209 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.209 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:31:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:31:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:31:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:31:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:31:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:31:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.210 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:31:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:31:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:31:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:31:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:31:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:31:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.211 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.212 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:31:22.212 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:31:22.212 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.212 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.212 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.212 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:31:22.212 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:31:22.212 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.212 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.212 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.212 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:31:22.212 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:31:22.212 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.212 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.212 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.213 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.213 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.213 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:22.213 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:22.213 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 256.87038457699998, 2023-12-06T18:31:22.213 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:31:22.213 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.213 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.213 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.213 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.213 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.213 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:31:22.213 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.213 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.213 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.214 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.214 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:22.214 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.214 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.214 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.214 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:22.214 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.214 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.214 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.214 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:31:22.215 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:31:22.215 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.215 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.215 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.215 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:31:22.215 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:31:22.215 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.215 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.215 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.215 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:31:22.215 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:31:22.215 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.215 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.215 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.215 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:31:22.216 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:31:22.216 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.216 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.216 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.216 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:31:22.216 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:31:22.216 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.216 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.216 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.216 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:31:22.216 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:31:22.216 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.216 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.216 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.216 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:31:22.217 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:31:22.217 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.217 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.217 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.217 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:31:22.217 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:31:22.217 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.217 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.217 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.217 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:31:22.217 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:31:22.217 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.217 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.217 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.217 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:31:22.218 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:31:22.218 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.218 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.218 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.218 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:31:22.218 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:31:22.218 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.218 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.218 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.218 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.218 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.218 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:22.219 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:22.219 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 256.86957896899997, 2023-12-06T18:31:22.219 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:31:22.219 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.219 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.219 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.219 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.219 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.219 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:31:22.219 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.219 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.219 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.219 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.219 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:22.219 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.220 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.220 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.220 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.220 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:22.220 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.220 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.220 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.220 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.220 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:31:22.220 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:31:22.220 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.220 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.220 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.220 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:31:22.220 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:31:22.221 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.221 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.221 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.221 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:31:22.221 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:31:22.221 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.221 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.221 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.221 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:31:22.221 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:31:22.221 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.221 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.221 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.221 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:31:22.221 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:31:22.222 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.222 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.222 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.222 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:31:22.222 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:31:22.222 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.222 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.222 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.222 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:31:22.222 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:31:22.222 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.222 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.222 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.222 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:31:22.222 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:31:22.223 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.223 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.223 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.223 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:31:22.223 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:31:22.223 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.223 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.223 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.223 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:31:22.223 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:31:22.223 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.223 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.223 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.223 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:31:22.223 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:31:22.224 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.224 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.224 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.224 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.224 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.224 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:22.224 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:22.224 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 256.86888815999998, 2023-12-06T18:31:22.224 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:31:22.224 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.224 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.224 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.224 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.224 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.224 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:31:22.225 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.225 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.225 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.225 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.225 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:22.225 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.225 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.225 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.225 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.225 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:22.225 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.225 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.225 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.225 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.226 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:31:22.226 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:31:22.226 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.226 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.226 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.226 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:31:22.226 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:31:22.226 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.226 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.226 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.226 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:31:22.226 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:31:22.226 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.227 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.227 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.227 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:31:22.227 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:31:22.227 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.227 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.227 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.227 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:31:22.227 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:31:22.227 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.227 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.227 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.227 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:31:22.227 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:31:22.227 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.228 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.228 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.228 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:31:22.228 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:31:22.228 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.228 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.228 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.228 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:31:22.228 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:31:22.228 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.228 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.228 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.228 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:31:22.228 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:31:22.228 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.229 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.229 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.229 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:31:22.229 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:31:22.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.229 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.229 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.229 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:31:22.229 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:31:22.229 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.229 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.229 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.229 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.229 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:22.230 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:22.230 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 256.864590652, 2023-12-06T18:31:22.230 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:31:22.230 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.230 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.230 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.230 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.230 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.230 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:31:22.230 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.230 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.230 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.230 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.230 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:22.231 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.231 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.231 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.231 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.231 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:22.231 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.231 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.231 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.231 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.231 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:31:22.231 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:31:22.231 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.231 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.231 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.232 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:31:22.232 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:31:22.232 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.232 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.232 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.232 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:31:22.232 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:31:22.232 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.232 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.232 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.232 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:31:22.232 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:31:22.232 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.232 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.233 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.233 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:31:22.233 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:31:22.233 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.233 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.233 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.233 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:31:22.233 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:31:22.233 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.233 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.233 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.233 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:31:22.233 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:31:22.233 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.234 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.234 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.234 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:31:22.234 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:31:22.234 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.234 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.234 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.234 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:31:22.234 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:31:22.234 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.234 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.234 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.234 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:31:22.234 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:31:22.235 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.235 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.235 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.235 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:31:22.235 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:31:22.235 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.235 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.235 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.235 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.235 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.235 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:22.235 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:22.235 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 256.86392297700002, 2023-12-06T18:31:22.235 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:31:22.235 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.236 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.236 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.236 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.236 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.236 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:31:22.236 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.236 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.236 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.236 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.236 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:22.236 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.236 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.236 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.236 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.236 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:22.237 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.237 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.237 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.237 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.237 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:31:22.237 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:31:22.237 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.237 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.237 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.237 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:31:22.237 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:31:22.237 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.237 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.237 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.237 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:31:22.238 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:31:22.238 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.238 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.238 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.238 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:31:22.238 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:31:22.238 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.238 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.238 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.238 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:31:22.238 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:31:22.238 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.238 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.238 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.238 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:31:22.239 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:31:22.239 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.239 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.239 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.239 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:31:22.239 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:31:22.239 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.239 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.239 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.239 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:31:22.239 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:31:22.239 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.239 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.239 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.239 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:31:22.240 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:31:22.240 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.240 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.240 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.240 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:31:22.240 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:31:22.240 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.240 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.240 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.240 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:31:22.240 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:31:22.240 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.240 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.240 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.240 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.241 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.241 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:22.241 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:22.241 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 256.863267549, 2023-12-06T18:31:22.241 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:31:22.241 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.241 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.241 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.241 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.241 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.241 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:31:22.241 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.241 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.241 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.241 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.241 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:22.242 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.242 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.242 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.242 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.242 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:22.242 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.242 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.242 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.242 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.242 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:31:22.242 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:31:22.242 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.242 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.242 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.243 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:31:22.243 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:31:22.243 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.243 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.243 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.243 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:31:22.243 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:31:22.243 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.243 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.243 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.243 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:31:22.243 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:31:22.243 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.243 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.243 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.244 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:31:22.244 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:31:22.244 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.244 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.244 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.244 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:31:22.244 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:31:22.244 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.244 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.244 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.244 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:31:22.245 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:31:22.245 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.245 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.245 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.245 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:31:22.245 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:31:22.245 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.245 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.245 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.245 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:31:22.245 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:31:22.245 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.245 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.245 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.245 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:31:22.246 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:31:22.246 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.246 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.246 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.246 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:31:22.246 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:31:22.246 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.246 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.246 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.246 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.246 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.246 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:22.246 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:22.246 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 256.82261300200003, 2023-12-06T18:31:22.246 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:31:22.247 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.247 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:22.247 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:22.247 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:22.247 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:22.247 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:31:22.247 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.247 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.247 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.247 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.247 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:22.247 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.247 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.247 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.247 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.247 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:22.248 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.248 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.248 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.248 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.248 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:31:22.248 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:31:22.248 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.248 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.248 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.248 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:31:22.248 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:31:22.248 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.248 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.248 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.248 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:31:22.249 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:31:22.249 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.249 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.249 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.249 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:31:22.249 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:31:22.249 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.249 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.249 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.249 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:31:22.249 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:31:22.249 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.249 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.249 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.249 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:31:22.250 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:31:22.250 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.250 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.250 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:22.250 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:31:22.250 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:31:22.250 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.250 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.250 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:22.250 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:31:22.250 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:31:22.250 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.250 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.250 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:22.250 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:31:22.251 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:31:22.251 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.251 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.251 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:22.251 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:31:22.251 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:31:22.251 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.251 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.251 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.251 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:31:22.251 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:31:22.251 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.251 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.251 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.251 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.251 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.252 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:22.253 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:31:22.253 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:22.250+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:22.354 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:22.351+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:22.432 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:22.432 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:22.433 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:22.433 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:22.442 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:31:22.455 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:22.452+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:22.556 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:22.553+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:22.590 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:22.590 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:22.590 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:22.590 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:22.590 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:22.599 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:31:22.619 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is set 2023-12-06T18:31:22.657 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:22.654+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:22.758 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:22.755+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:22.772 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:22.772 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:22.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:22.772 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:22.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.772 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:31:22.772 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:22.772 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 240.95635238, 2023-12-06T18:31:22.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:31:22.773 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:22.773 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:31:22.773 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:22.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:22.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:22.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:22.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:22.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:22.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:22.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:31:22.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:31:22.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:22.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:31:22.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:31:22.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:22.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:31:22.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:31:22.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:22.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:31:22.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:31:22.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:22.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:31:22.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:31:22.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:22.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:31:22.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:31:22.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:22.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:22.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:22.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:31:22.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:31:22.776 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.776 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.776 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.776 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:22.776 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:22.776 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:22.782 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:31:22.859 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:22.856+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:22.927 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:22.927 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:22.927 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:22.927 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:22.938 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:31:22.959 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:22.956+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:23.060 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:23.057+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:23.118 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:23.118 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:23.118 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:23.118 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:23.118 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:23.127 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:31:23.171 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:23.168+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:23.262 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:23.259+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:23.280 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:23.280 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:23.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:23.280 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:23.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:23.280 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:23.280 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:23.280 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 122.180516743, 2023-12-06T18:31:23.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:31:23.280 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:23.281 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:23.281 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:23.281 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:23.281 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:23.281 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:31:23.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:23.281 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:23.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:23.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:23.281 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:23.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:23.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:23.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:23.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:23.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:23.282 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:23.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:23.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:23.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:23.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:31:23.282 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:31:23.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:23.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:23.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:23.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:31:23.282 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:31:23.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:23.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:23.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:23.283 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:31:23.283 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:31:23.283 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:23.283 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:23.283 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:23.283 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:31:23.283 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:31:23.283 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:23.283 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:23.283 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:23.283 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:31:23.283 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:31:23.283 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:23.283 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:23.284 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:23.284 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:31:23.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:31:23.284 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:23.284 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:23.284 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:23.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:23.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:23.284 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:31:23.284 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:23.284 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 62.162259441000003, 2023-12-06T18:31:23.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:31:23.284 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:23.284 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:23.284 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:23.285 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:23.285 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:23.285 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:31:23.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:23.285 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:23.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:23.285 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:23.285 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:23.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:23.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:23.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:23.285 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:23.285 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:23.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:23.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:23.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:23.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:23.286 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:31:23.286 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:31:23.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:23.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:23.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:23.286 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:31:23.286 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:31:23.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:23.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:23.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:23.286 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:31:23.286 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:31:23.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:23.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:23.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:23.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:31:23.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:31:23.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:23.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:23.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:23.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:31:23.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:31:23.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:23.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:23.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:23.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:31:23.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:31:23.287 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:23.287 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:23.287 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:23.288 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:23.288 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:23.288 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:23.289 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:31:23.363 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:23.360+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:23.418 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:23.418 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:23.418 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:23.418 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:23.427 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:31:23.470 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:23.465+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:23.565 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:23.561+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:23.604 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:23.604 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:23.604 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:23.604 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:23.604 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:23.613 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:31:23.698 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:23.694+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:23.766 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:23.764+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:23.782 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:23.782 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:23.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:23.782 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:23.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:23.782 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:23.782 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:23.782 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 182.28887764199999, 2023-12-06T18:31:23.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:31:23.783 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:23.783 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:23.783 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:23.783 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:23.783 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:23.783 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:31:23.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:23.783 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:23.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:23.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:23.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:23.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:23.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:23.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:23.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:23.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:23.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:23.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:23.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:23.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:23.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:31:23.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:31:23.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:23.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:23.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:23.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:31:23.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:31:23.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:23.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:23.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:23.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:31:23.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:31:23.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:23.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:23.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:23.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:31:23.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:31:23.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:23.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:23.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:23.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:31:23.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:31:23.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:23.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:23.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:23.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:31:23.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:31:23.786 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:23.786 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:23.786 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:23.786 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:23.786 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:23.786 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:23.791 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:31:23.867 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:23.865+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:23.951 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:31:23.962 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:23.962 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:23.962 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:23.962 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:23.968 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:23.964+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:23.971 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:31:24.068 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:24.066+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:24.141 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:24.141 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:24.141 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:24.141 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:24.141 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:24.151 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:31:24.194 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:24.191+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:24.264 INFO:teuthology.orchestra.run.smithi061.stderr:2023-12-06T18:31:24.262+0000 7f7148b20640 0 --2- 172.21.15.61:0/3447376971 >> [v2:172.21.15.61:3301/0,v1:172.21.15.61:6790/0] conn(0x7f7144141a00 0x7f7144141e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2023-12-06T18:31:24.269 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:31:24.270 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:24.266+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:24.328 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:24.328 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:24.328 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:24.328 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:24.329 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.329 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:31:24.329 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:24.329 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 267.70571674299998, 2023-12-06T18:31:24.329 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:31:24.329 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.329 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.329 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.330 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.330 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.330 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:24.330 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.330 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.330 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.330 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.330 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:24.331 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.331 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.331 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.331 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:31:24.331 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:31:24.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.332 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.332 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.332 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:31:24.332 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:31:24.332 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.332 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.332 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.333 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:31:24.333 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:31:24.333 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.333 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.333 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.333 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:31:24.333 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:31:24.334 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.334 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.334 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.334 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:31:24.334 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:31:24.334 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.334 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.334 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.334 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:31:24.334 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:31:24.335 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.335 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.335 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.335 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:31:24.335 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:31:24.335 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.335 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.335 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.335 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:31:24.335 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:31:24.335 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.335 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.335 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.335 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.335 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.336 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:31:24.336 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:24.336 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 266.68288973400001, 2023-12-06T18:31:24.336 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:31:24.336 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.336 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.336 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.336 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.336 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.336 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:24.336 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.336 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.336 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.336 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.336 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:24.336 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.337 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.337 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.337 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.337 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:31:24.337 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:31:24.337 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.337 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.337 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.337 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:31:24.337 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:31:24.337 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.337 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.338 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.338 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:31:24.338 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:31:24.338 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.338 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.338 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.338 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:31:24.338 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:31:24.338 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.339 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.339 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.339 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:31:24.339 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:31:24.339 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.339 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.339 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.339 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:31:24.339 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:31:24.339 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.339 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.339 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.340 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:31:24.340 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:31:24.340 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.340 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.340 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.340 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:31:24.340 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:31:24.340 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.340 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.340 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.340 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.340 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.341 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:31:24.341 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:24.341 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 266.670843096, 2023-12-06T18:31:24.341 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:31:24.341 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.341 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.341 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.341 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.341 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.341 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:24.341 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.342 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.342 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.342 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.342 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:24.342 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.342 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.343 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.343 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.343 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:31:24.343 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:31:24.343 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.343 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.344 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.344 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:31:24.344 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:31:24.344 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.344 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.344 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.344 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:31:24.344 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:31:24.344 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.345 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.345 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.345 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:31:24.345 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:31:24.345 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.345 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.345 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.345 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:31:24.346 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:31:24.346 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.346 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.346 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.346 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:31:24.347 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:31:24.347 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.347 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.347 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.347 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:31:24.347 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:31:24.347 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.347 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.347 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.347 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:31:24.347 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:31:24.347 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.349 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.349 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.349 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.349 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.349 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:31:24.349 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:24.350 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 266.64944248900002, 2023-12-06T18:31:24.350 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:31:24.350 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.350 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.350 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.350 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.350 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.350 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:24.350 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.350 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.350 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.351 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.351 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:24.351 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.351 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.351 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.351 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.351 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:31:24.351 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:31:24.351 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.351 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.351 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.351 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:31:24.352 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:31:24.352 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.352 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.352 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.352 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:31:24.352 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:31:24.352 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.352 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.352 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.352 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:31:24.352 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:31:24.352 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.352 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.352 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.352 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:31:24.353 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:31:24.353 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.353 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.353 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.353 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:31:24.353 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:31:24.353 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.353 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.353 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.353 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:31:24.353 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:31:24.354 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.354 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.354 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.354 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:31:24.354 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:31:24.354 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.354 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.354 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.354 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.354 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.354 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:31:24.354 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:24.354 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 266.64826316599999, 2023-12-06T18:31:24.355 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:31:24.355 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.355 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.355 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.355 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.355 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.355 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:24.355 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.355 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.355 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.355 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.355 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:24.355 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.355 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.355 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.356 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.356 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:31:24.356 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:31:24.356 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.356 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.356 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.356 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:31:24.356 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:31:24.356 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.356 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.356 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.356 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:31:24.356 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:31:24.356 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.356 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.357 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.357 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:31:24.357 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:31:24.357 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.357 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.357 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.357 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:31:24.357 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:31:24.357 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.357 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.357 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.357 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:31:24.357 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:31:24.357 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.357 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.358 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.358 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:31:24.358 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:31:24.358 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.358 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.358 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.358 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:31:24.358 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:31:24.358 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.358 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.358 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.358 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.358 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.359 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:31:24.359 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:24.359 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 266.64769118499999, 2023-12-06T18:31:24.359 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:31:24.359 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.359 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.359 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.359 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.359 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.359 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:24.359 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.359 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.359 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.359 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:31:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:31:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:31:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:31:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:31:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:31:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:31:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:31:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:31:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:31:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:31:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:31:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:31:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:31:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:31:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:31:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:31:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 266.20185571500002, 2023-12-06T18:31:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:31:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:31:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:31:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:31:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:31:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:31:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:31:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:31:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:31:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:31:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:31:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:31:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:31:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:31:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:31:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:31:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:31:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:31:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 266.20025585899998, 2023-12-06T18:31:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:31:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:31:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:31:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:31:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:31:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:31:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:31:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:31:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:31:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:31:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:31:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:31:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:31:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:31:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:31:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:31:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:31:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:31:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 265.821050243, 2023-12-06T18:31:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:31:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:24.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.373 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.373 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.373 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.373 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:24.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.373 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.373 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.373 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.373 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:31:24.374 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:31:24.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.374 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.374 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:31:24.374 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:31:24.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.374 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.374 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:31:24.374 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:31:24.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:31:24.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:31:24.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:31:24.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:31:24.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:31:24.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:31:24.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.376 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.376 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:31:24.376 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:31:24.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.376 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.376 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:31:24.376 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:31:24.376 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.377 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.377 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.377 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.377 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:31:24.377 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:24.377 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 265.58354806099999, 2023-12-06T18:31:24.377 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:31:24.377 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.377 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.377 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.377 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.377 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.377 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:24.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.378 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.378 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:24.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.379 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.379 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.379 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:31:24.379 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:31:24.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.379 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.379 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:31:24.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:31:24.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.380 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:31:24.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:31:24.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.380 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:31:24.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:31:24.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.381 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.381 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.381 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:31:24.381 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:31:24.381 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.381 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.381 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.381 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:31:24.381 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:31:24.381 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.381 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.381 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.381 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:31:24.381 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:31:24.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.382 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.382 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.382 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:31:24.382 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:31:24.382 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.382 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.382 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.382 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.382 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:31:24.382 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:24.382 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 265.582820062, 2023-12-06T18:31:24.383 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:31:24.383 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.383 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.383 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.383 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.383 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:24.383 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.383 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.383 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:24.383 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.383 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.384 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:31:24.384 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:31:24.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.384 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.384 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.384 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:31:24.384 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:31:24.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.384 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.384 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.384 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:31:24.384 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:31:24.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.384 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.385 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.385 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:31:24.385 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:31:24.385 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.385 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.385 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.385 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:31:24.386 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:31:24.386 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.386 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.386 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.386 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:31:24.386 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:31:24.387 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.387 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.387 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.387 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:31:24.387 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:31:24.387 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.387 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.387 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.388 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:31:24.388 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:31:24.388 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.388 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.388 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.388 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.388 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.388 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:31:24.388 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:24.388 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 265.57780765199999, 2023-12-06T18:31:24.388 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:31:24.388 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.389 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.389 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.389 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.389 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.389 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:24.389 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.389 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.389 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.389 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.389 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:24.389 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.389 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.389 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.389 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.390 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:31:24.390 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:31:24.390 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.390 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.390 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.390 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:31:24.390 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:31:24.390 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.390 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.390 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.390 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:31:24.391 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:31:24.391 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.391 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.391 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.391 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:31:24.391 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:31:24.391 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.391 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.392 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.392 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:31:24.392 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:31:24.392 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.392 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.392 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.392 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:31:24.392 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:31:24.392 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:31:24.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:31:24.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:31:24.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:31:24.394 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.394 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.394 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.394 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.394 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:31:24.394 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:24.394 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 265.572280573, 2023-12-06T18:31:24.394 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:31:24.395 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.395 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.395 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.395 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.395 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.395 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:24.396 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.396 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.396 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.396 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:24.396 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.397 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.397 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:31:24.397 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:31:24.397 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.397 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.398 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:31:24.398 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:31:24.398 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.398 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.398 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.398 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:31:24.398 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:31:24.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.399 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.399 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:31:24.399 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:31:24.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:31:24.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:31:24.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:31:24.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:31:24.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:31:24.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:31:24.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.402 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:31:24.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:31:24.402 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.402 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.402 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.402 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:31:24.403 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:24.403 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 265.57176973600002, 2023-12-06T18:31:24.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:31:24.403 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.403 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.403 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.403 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.403 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:24.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.404 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.404 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:24.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.404 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:31:24.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:31:24.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:31:24.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:31:24.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:31:24.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:31:24.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:31:24.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:31:24.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:31:24.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:31:24.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.407 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.407 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:31:24.407 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:31:24.407 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.407 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.407 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.407 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:31:24.407 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:31:24.407 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.407 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.407 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.407 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:31:24.407 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:31:24.407 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.407 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.408 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.408 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.408 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:31:24.408 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:24.408 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 265.57131047500002, 2023-12-06T18:31:24.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:31:24.408 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.408 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.408 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.408 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:24.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:24.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:31:24.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:31:24.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:31:24.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:31:24.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:31:24.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:31:24.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:31:24.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:31:24.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:31:24.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:31:24.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:31:24.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:31:24.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.412 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.412 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:31:24.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:31:24.412 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.412 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.412 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:31:24.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:31:24.412 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.412 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.412 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.412 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.412 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:31:24.412 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:24.413 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 258.991394668, 2023-12-06T18:31:24.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:31:24.413 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.413 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.413 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.413 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:24.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.413 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:24.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:31:24.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:31:24.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:31:24.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:31:24.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:31:24.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:31:24.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:31:24.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:31:24.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:31:24.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:31:24.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:31:24.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:31:24.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:31:24.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:31:24.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.417 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:31:24.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:31:24.417 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.417 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.417 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.417 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:31:24.417 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:24.417 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 258.98515686299999, 2023-12-06T18:31:24.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:31:24.417 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.417 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.418 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:24.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:24.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:31:24.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:31:24.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:31:24.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:31:24.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:31:24.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:31:24.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:31:24.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:31:24.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:31:24.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:31:24.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:31:24.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:31:24.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:31:24.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:31:24.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:31:24.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:31:24.421 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.421 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.421 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.422 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:31:24.422 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:24.422 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 258.97909378399999, 2023-12-06T18:31:24.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:31:24.422 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.422 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.422 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.422 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.422 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:24.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:24.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:31:24.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:31:24.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:31:24.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:31:24.424 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.424 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.424 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:31:24.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:31:24.424 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.424 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.424 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:31:24.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:31:24.424 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.424 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.424 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:31:24.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:31:24.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:31:24.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:31:24.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:31:24.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:31:24.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:31:24.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:31:24.426 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.426 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.426 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.426 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:31:24.426 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:24.426 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 258.84170669899999, 2023-12-06T18:31:24.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:31:24.426 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.426 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.426 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.427 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.427 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:24.427 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.427 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.427 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:24.427 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.427 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.427 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:31:24.427 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:31:24.428 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.428 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.428 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.428 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:31:24.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:31:24.428 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.428 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.428 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.428 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:31:24.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:31:24.428 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.428 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.428 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:31:24.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:31:24.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:31:24.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:31:24.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:31:24.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:31:24.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:31:24.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:31:24.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:31:24.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:31:24.430 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.430 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.430 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.431 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:31:24.431 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:24.431 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 258.83503597200001, 2023-12-06T18:31:24.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:31:24.431 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:24.431 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:24.431 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:24.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:24.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:24.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:24.432 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:24.432 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:24.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.432 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:24.432 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:31:24.432 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:31:24.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.432 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:24.432 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:31:24.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:31:24.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:24.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:31:24.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:31:24.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:24.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:31:24.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:31:24.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:24.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:31:24.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:31:24.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.434 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:24.434 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:31:24.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:31:24.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.434 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:24.434 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:31:24.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:31:24.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:24.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:24.434 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:24.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:31:24.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:31:24.435 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.435 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.435 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.435 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:24.435 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:24.435 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:24.436 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:31:24.437 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:24.434+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:24.477 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:31:24.477 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:31:24.537 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:24.534+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:24.570 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:31:24.571 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:31:24.606 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:24.606 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:24.606 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:24.606 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:24.615 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:31:24.635 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-12-06T18:31:24.657 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:24.653+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:24.662 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:31:24.662 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:31:24.739 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:24.736+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:24.787 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:24.788 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:24.788 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:24.788 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:24.788 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:24.791 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:31:24.791 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:31:24.797 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:31:24.840 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:24.837+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:24.903 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:31:24.903 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:31:24.941 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:24.937+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:24.948 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:24.948 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:24.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:24.948 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:31:24.948 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:24.957 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:31:24.991 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:31:24.992 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:31:25.032 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":34,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":7,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:31:25.042 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:25.039+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:25.086 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:31:25.087 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:31:25.144 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:25.141+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:25.153 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:25.153 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:25.153 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:25.153 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:25.164 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:31:25.228 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:31:25.228 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:31:25.244 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:25.242+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:25.293 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:25.293 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:25.303 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:31:25.348 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:25.344+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:25.431 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:25.432 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:25.440 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:31:25.482 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:25.478+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:25.547 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:25.544+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:25.590 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:25.590 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:25.600 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:31:25.648 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:25.644+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:25.749 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:25.746+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:25.781 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:25.781 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:25.791 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:31:25.872 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:25.603 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:25.873 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:25.870+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:25.941 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:25.672 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:25.950 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:25.947+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:25.970 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:25.971 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:25.981 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:31:25.994 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:25.729 106176== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:26.045 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:25.781 106176== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:26.051 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:26.048+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:26.092 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:31:26.092 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:31:26.092 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2023-12-06T18:31:26.092 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2023-12-06T18:31:26.092 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2023-12-06T18:31:26.092 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2023-12-06T18:31:26.152 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:26.148+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:26.157 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:25.887 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:26.193 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:26.193 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:26.205 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:31:26.249 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:25.984 106176== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:26.254 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:26.251+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:26.276 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:26.276 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2023-12-06T18:31:26.276 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:26.355 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:26.351+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:26.360 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:26.360 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:26.370 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:31:26.455 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:26.451+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:26.459 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is set 2023-12-06T18:31:26.528 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:26.528 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:26.540 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:31:26.584 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:26.581+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:26.657 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:26.654+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:26.678 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:26.678 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:26.688 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:31:26.773 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:26.770+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:26.860 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:26.857+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:26.865 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:26.865 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:26.874 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:31:26.961 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:26.958+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:27.053 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:27.053 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:27.062 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:31:27.063 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:27.058+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:27.163 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:27.160+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:27.244 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:27.244 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:27.244 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:31:27.253 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:31:27.264 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:27.261+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:27.366 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:27.363+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:27.447 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:27.448 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:27.457 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:31:27.498 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:27.495+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:27.552 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:31:27.553 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:31:27.567 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:27.565+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:27.651 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:27.652 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:27.661 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:31:27.682 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:31:27.682 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:31:27.706 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:27.702+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:27.770 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:27.767+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:27.774 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:31:27.774 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:31:27.790 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:27.790 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:27.799 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:31:27.870 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:27.868+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:27.898 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:31:27.899 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:31:27.971 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:27.969+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:28.006 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:31:28.006 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:31:28.007 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:28.007 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:28.007 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:28.007 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:28.007 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:28.016 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:31:28.046 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:31:28.090 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:28.087+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:28.101 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:31:28.101 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:31:28.156 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:28.156 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:28.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:28.156 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:28.156 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.156 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:28.156 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:28.156 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 186.66248819500001, 2023-12-06T18:31:28.156 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:31:28.156 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.156 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:28.156 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:28.156 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.157 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.158 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.158 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:31:28.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:31:28.158 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.158 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.158 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.158 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:31:28.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:31:28.158 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.158 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.158 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.158 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:31:28.158 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:31:28.158 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:31:28.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:31:28.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.159 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.159 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:31:28.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:31:28.159 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.159 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.160 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.160 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:31:28.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:31:28.160 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.160 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.160 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.160 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.160 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.160 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:28.166 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:31:28.173 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:28.171+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:28.226 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:31:28.227 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:31:28.275 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:28.272+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:28.296 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:28.296 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:28.296 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:28.296 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:28.305 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:31:28.313 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:31:28.314 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:31:28.375 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:28.373+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:28.435 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:28.435 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:28.435 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:28.435 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:28.435 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:28.445 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:31:28.475 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-12-06T18:31:28.489 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:28.485+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:28.578 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:28.575+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:28.620 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:28.620 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:28.620 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:28.620 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:28.620 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.621 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:31:28.621 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:28.621 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 272.00013714400001, 2023-12-06T18:31:28.621 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:31:28.621 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.621 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.621 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.621 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.621 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.621 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:28.621 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.621 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.621 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.621 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.621 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:28.622 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.622 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.622 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.622 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:31:28.622 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:31:28.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.622 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.622 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.622 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:31:28.622 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:31:28.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.622 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.622 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:31:28.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:31:28.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:31:28.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:31:28.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:31:28.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:31:28.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:31:28.624 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:31:28.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.624 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.624 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.624 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:31:28.624 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:31:28.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.624 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.624 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.624 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:31:28.624 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:31:28.624 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.624 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.624 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.625 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:31:28.625 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:28.625 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 270.97731013499998, 2023-12-06T18:31:28.625 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:31:28.625 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.625 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.625 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.625 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.625 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:28.625 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.625 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.625 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.626 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:28.626 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.626 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.626 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:31:28.626 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:31:28.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.626 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.626 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:31:28.626 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:31:28.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.626 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.627 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:31:28.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:31:28.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.627 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.627 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:31:28.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:31:28.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.627 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.627 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:31:28.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:31:28.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:31:28.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:31:28.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:31:28.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:31:28.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:31:28.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:31:28.629 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.629 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.629 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.629 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:31:28.629 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:28.629 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 270.96526349700002, 2023-12-06T18:31:28.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:31:28.629 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.629 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.629 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.629 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.630 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:28.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.630 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:28.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.630 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:31:28.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:31:28.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:31:28.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:31:28.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:31:28.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:31:28.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:31:28.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:31:28.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:31:28.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:31:28.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.632 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.632 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:31:28.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:31:28.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.632 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.632 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:31:28.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:31:28.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.632 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.632 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:31:28.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:31:28.633 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.633 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.633 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.633 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.633 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:31:28.633 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:28.633 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 270.94386288999999, 2023-12-06T18:31:28.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:31:28.633 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.633 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.633 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.633 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.633 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:28.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:28.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:31:28.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:31:28.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:31:28.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:31:28.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:31:28.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:31:28.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:31:28.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:31:28.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:31:28.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:31:28.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:31:28.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:31:28.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:31:28.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:31:28.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.637 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.637 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:31:28.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:31:28.637 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.637 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.637 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.637 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.637 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:31:28.637 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:28.637 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 270.94268356700002, 2023-12-06T18:31:28.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:31:28.637 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.637 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.637 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.638 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:28.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.638 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:28.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.638 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:31:28.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:31:28.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:31:28.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:31:28.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:31:28.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:31:28.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:31:28.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:31:28.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:31:28.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:31:28.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:31:28.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:31:28.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:31:28.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:31:28.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:31:28.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:31:28.641 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.641 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.641 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.641 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:31:28.641 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:28.641 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 270.94211158600001, 2023-12-06T18:31:28.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:31:28.641 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.641 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.641 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:28.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:28.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:31:28.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:31:28.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.643 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:31:28.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:31:28.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.643 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:31:28.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:31:28.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.643 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:31:28.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:31:28.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:31:28.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:31:28.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:31:28.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:31:28.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:31:28.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:31:28.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:31:28.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:31:28.645 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.645 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.645 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.645 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:31:28.645 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:28.645 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 270.49627611599999, 2023-12-06T18:31:28.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:31:28.645 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.645 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.645 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:28.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:28.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:31:28.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:31:28.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:31:28.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:31:28.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:31:28.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:31:28.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:31:28.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:31:28.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.648 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:31:28.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:31:28.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.648 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:31:28.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:31:28.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.648 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:31:28.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:31:28.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.649 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.649 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:31:28.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:31:28.649 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.649 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.649 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.649 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.649 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.649 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:31:28.649 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:28.649 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 270.49467626000001, 2023-12-06T18:31:28.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:31:28.649 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.649 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.649 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.649 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.650 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:28.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.650 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:28.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.650 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:31:28.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:31:28.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:31:28.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:31:28.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:31:28.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:31:28.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:31:28.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:31:28.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:31:28.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:31:28.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:31:28.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:31:28.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:31:28.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:31:28.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:31:28.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:31:28.653 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.653 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.653 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.653 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:31:28.653 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:28.653 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 270.11547064400003, 2023-12-06T18:31:28.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:31:28.653 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.653 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.653 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.654 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:28.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.654 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:28.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.654 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:31:28.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:31:28.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:31:28.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:31:28.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:31:28.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:31:28.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:31:28.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:31:28.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:31:28.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:31:28.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:31:28.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:31:28.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:31:28.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:31:28.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:31:28.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:31:28.657 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.657 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.657 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.657 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:31:28.657 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:28.657 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 269.87796846200001, 2023-12-06T18:31:28.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:31:28.657 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.657 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.657 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:28.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:28.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:31:28.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:31:28.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:31:28.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:31:28.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:31:28.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:31:28.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:31:28.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:31:28.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:31:28.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:31:28.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:31:28.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:31:28.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:31:28.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:31:28.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:31:28.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:31:28.661 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.661 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.661 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.662 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:31:28.662 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:28.662 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 269.87724046300002, 2023-12-06T18:31:28.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:31:28.662 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.662 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.662 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:28.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:28.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:31:28.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:31:28.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:31:28.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:31:28.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:31:28.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:31:28.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:31:28.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:31:28.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:31:28.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:31:28.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:31:28.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:31:28.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.665 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.665 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:31:28.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:31:28.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.665 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.665 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:31:28.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:31:28.665 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.665 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.665 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.665 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:31:28.666 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:28.666 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 269.87222805300001, 2023-12-06T18:31:28.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:31:28.666 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.666 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.666 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:28.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:28.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:31:28.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:31:28.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:31:28.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:31:28.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:31:28.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:31:28.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:31:28.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:31:28.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:31:28.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:31:28.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:31:28.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:31:28.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:31:28.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:31:28.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:31:28.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:31:28.669 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.669 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.669 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.669 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:31:28.669 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:28.669 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 269.86670097400003, 2023-12-06T18:31:28.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:31:28.670 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.670 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.670 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:28.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:28.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:31:28.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:31:28.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:31:28.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:31:28.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:31:28.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:31:28.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:31:28.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:31:28.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:31:28.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:31:28.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:31:28.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:31:28.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:31:28.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:31:28.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:31:28.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:31:28.673 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.673 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.673 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.673 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:31:28.673 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:28.673 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 269.86619013699999, 2023-12-06T18:31:28.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:31:28.673 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.674 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.674 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:28.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:28.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:31:28.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:31:28.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:31:28.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:31:28.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:31:28.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:31:28.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:31:28.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:31:28.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:31:28.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:31:28.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:31:28.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:31:28.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:31:28.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:31:28.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.677 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.677 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.677 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:31:28.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:31:28.677 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.677 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.677 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.677 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.677 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.677 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:31:28.677 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:28.677 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 269.86573087599999, 2023-12-06T18:31:28.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:31:28.677 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.677 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.678 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:28.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:28.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:31:28.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:31:28.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:31:28.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:31:28.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:31:28.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:31:28.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:31:28.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:31:28.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:31:28.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:31:28.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:31:28.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:31:28.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:31:28.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:31:28.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.681 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.681 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:31:28.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:31:28.681 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.681 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.681 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.681 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:31:28.681 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:28.681 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 263.28581506900002, 2023-12-06T18:31:28.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:31:28.681 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.681 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.682 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:28.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:28.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:31:28.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:31:28.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:31:28.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:31:28.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:31:28.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:31:28.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:31:28.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:31:28.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:31:28.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:31:28.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:31:28.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:31:28.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:31:28.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:31:28.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:31:28.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:31:28.685 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.685 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.685 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.685 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:31:28.685 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:28.685 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 263.27957726400001, 2023-12-06T18:31:28.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:31:28.685 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.685 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.685 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.686 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.686 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.686 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:28.686 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.686 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.686 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.686 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:28.686 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.686 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.686 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.687 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:31:28.687 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:31:28.687 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.687 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.687 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.687 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:31:28.687 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:31:28.687 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.687 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.687 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.687 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:31:28.687 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:31:28.687 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.687 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.688 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.688 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:31:28.688 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:31:28.688 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.688 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.688 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.688 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:31:28.688 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:31:28.688 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.688 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.688 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.688 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:31:28.688 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:31:28.688 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.688 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.688 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:31:28.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:31:28.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.689 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:31:28.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:31:28.689 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.689 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.689 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.689 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:31:28.689 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:28.690 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 263.27351418500001, 2023-12-06T18:31:28.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:31:28.690 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.690 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.690 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:28.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:28.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:31:28.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:31:28.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:31:28.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:31:28.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:31:28.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:31:28.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:31:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:31:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:31:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:31:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:31:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:31:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:31:28.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:31:28.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.693 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:31:28.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:31:28.693 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.693 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.693 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.693 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:31:28.693 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:28.693 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 263.13612710000001, 2023-12-06T18:31:28.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:31:28.694 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.694 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.694 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:28.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:28.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:31:28.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:31:28.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:31:28.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:31:28.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:31:28.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:31:28.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:31:28.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:31:28.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:31:28.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:31:28.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:31:28.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:31:28.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:31:28.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:31:28.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:31:28.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:31:28.697 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.697 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.697 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.697 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:31:28.697 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:28.697 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 263.12945637299998, 2023-12-06T18:31:28.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:31:28.697 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:28.697 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:28.698 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:28.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.698 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:28.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:28.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.698 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:28.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:28.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:28.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.698 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:28.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:31:28.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:31:28.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:28.699 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:31:28.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:31:28.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:28.699 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:31:28.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:31:28.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:28.699 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:31:28.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:31:28.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:28.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:31:28.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:31:28.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:28.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:31:28.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:31:28.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:28.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:31:28.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:31:28.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:28.701 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:28.701 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:28.701 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:31:28.701 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:31:28.701 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.701 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.701 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.701 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:28.701 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:28.701 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:28.702 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:31:28.703 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:28.700+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:28.803 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:28.801+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:28.889 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:28.889 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:28.890 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:28.890 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:28.899 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:31:28.904 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:28.901+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:28.905 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":34,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":11,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:31:29.006 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:29.002+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:29.028 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:29.028 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:29.028 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:29.028 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:29.028 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:29.038 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:31:29.106 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:29.104+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:29.207 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:29.205+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:29.208 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:29.208 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:29.208 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:29.208 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:31:29.208 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:29.217 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:31:29.276 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:08.721 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:29.308 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:29.305+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:29.351 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:29.352 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:29.352 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:29.352 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:29.361 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:31:29.446 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:29.442+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:29.516 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:29.512+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:29.525 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:08.971 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:29.530 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:29.530 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:29.530 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:29.530 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:29.531 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:29.539 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:31:29.572 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is unset 2023-12-06T18:31:29.623 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:29.620+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:29.712 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:29.710+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:29.713 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:29.713 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:29.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:29.713 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:29.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:29.714 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:31:29.714 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:29.714 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 68.051740441000007, 2023-12-06T18:31:29.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:31:29.714 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:29.714 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:29.714 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:29.714 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:29.714 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:29.714 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:31:29.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:29.715 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:29.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:29.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:29.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:29.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:29.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:29.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:29.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:29.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:29.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:29.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:29.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:29.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:29.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:31:29.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:31:29.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:29.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:29.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:29.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:31:29.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:31:29.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:29.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:29.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:29.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:31:29.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:31:29.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:29.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:29.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:29.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:31:29.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:31:29.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:29.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:29.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:29.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:31:29.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:31:29.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:29.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:29.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:29.718 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:31:29.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:31:29.718 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:29.718 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:29.718 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:29.718 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:29.718 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:29.718 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:29.723 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:31:29.814 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:29.811+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:29.863 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:29.863 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:29.863 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:29.863 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:29.872 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:31:29.914 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:29.911+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:30.015 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:30.012+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:30.049 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:30.049 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:30.049 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:30.049 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:30.049 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:30.059 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:31:30.144 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:30.141+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:30.217 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:30.213+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:30.255 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:30.255 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:30.255 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:30.255 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:31:30.255 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:30.264 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:31:30.318 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:30.314+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:30.326 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:31:30.419 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:30.415+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:30.476 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:30.476 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:30.476 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:30.476 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:30.485 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:31:30.529 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:30.526+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:30.610 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:31:30.610 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:31:30.620 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:30.617+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:30.678 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:30.679 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:30.679 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:30.679 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:30.679 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:30.688 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:31:30.731 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:30.727+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:30.766 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:31:30.766 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:31:30.818 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:30.548 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:30.822 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:30.818+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:30.865 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:30.595 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:30.882 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:30.613 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:30.913 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:30.647 106176== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:30.918 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:30.653 106176== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:30.922 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:30.922 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:30.923 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:31:30.923 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:31:30.923 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:30.919+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:30.933 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:30.668 106176== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:30.936 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:31:31.024 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:31.020+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:31.026 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:31:31.026 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:31:31.116 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:31:31.116 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:31:31.124 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:31.120+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:31.127 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:31.127 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:31.137 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:31:31.221 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:31:31.222 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:31:31.226 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:31.222+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:31.279 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:31.279 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:31.288 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:31:31.289 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:31:31.289 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:31:31.289 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2023-12-06T18:31:31.289 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2023-12-06T18:31:31.324 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:31:31.324 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:31:31.366 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:31.362+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:31.426 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:31:31.426 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:31:31.427 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:31.424+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:31.456 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:31.456 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:31.465 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:31:31.528 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:31.524+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:31.589 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-12-06T18:31:31.628 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:31.625+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:31.679 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:31.679 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:31.692 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:31:31.730 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:31.727+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:31.831 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:31.828+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:31.850 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:31.850 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:31.860 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:31:31.919 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:31:31.933 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:31.929+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:32.012 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:32.012 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:32.022 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:31:32.035 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:32.031+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:32.136 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:32.132+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:32.154 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:32.154 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:32.163 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:31:32.237 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:32.233+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:32.241 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:31:32.241 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:32.241 INFO:teuthology.orchestra.run.smithi061.stdout: "epoch": 74, 2023-12-06T18:31:32.241 INFO:teuthology.orchestra.run.smithi061.stdout: "fsid": "497e3283-cc16-40ff-a72c-a7487474fb4b", 2023-12-06T18:31:32.241 INFO:teuthology.orchestra.run.smithi061.stdout: "created": "2023-12-06T18:26:07.122862+0000", 2023-12-06T18:31:32.241 INFO:teuthology.orchestra.run.smithi061.stdout: "modified": "2023-12-06T18:31:29.484913+0000", 2023-12-06T18:31:32.242 INFO:teuthology.orchestra.run.smithi061.stdout: "last_up_change": "2023-12-06T18:30:04.956811+0000", 2023-12-06T18:31:32.242 INFO:teuthology.orchestra.run.smithi061.stdout: "last_in_change": "2023-12-06T18:28:45.621304+0000", 2023-12-06T18:31:32.242 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-06T18:31:32.242 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_num": 5804032, 2023-12-06T18:31:32.242 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_set": [ 2023-12-06T18:31:32.242 INFO:teuthology.orchestra.run.smithi061.stdout: "nodeep-scrub", 2023-12-06T18:31:32.242 INFO:teuthology.orchestra.run.smithi061.stdout: "pglog_hardlimit", 2023-12-06T18:31:32.242 INFO:teuthology.orchestra.run.smithi061.stdout: "purged_snapdirs", 2023-12-06T18:31:32.242 INFO:teuthology.orchestra.run.smithi061.stdout: "recovery_deletes", 2023-12-06T18:31:32.242 INFO:teuthology.orchestra.run.smithi061.stdout: "sortbitwise" 2023-12-06T18:31:32.242 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:31:32.242 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_version": 8, 2023-12-06T18:31:32.243 INFO:teuthology.orchestra.run.smithi061.stdout: "full_ratio": 0.94999998807907104, 2023-12-06T18:31:32.243 INFO:teuthology.orchestra.run.smithi061.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-12-06T18:31:32.243 INFO:teuthology.orchestra.run.smithi061.stdout: "nearfull_ratio": 0.85000002384185791, 2023-12-06T18:31:32.243 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_snapshot": "", 2023-12-06T18:31:32.243 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_max": 2, 2023-12-06T18:31:32.243 INFO:teuthology.orchestra.run.smithi061.stdout: "max_osd": 8, 2023-12-06T18:31:32.243 INFO:teuthology.orchestra.run.smithi061.stdout: "require_min_compat_client": "luminous", 2023-12-06T18:31:32.243 INFO:teuthology.orchestra.run.smithi061.stdout: "min_compat_client": "luminous", 2023-12-06T18:31:32.243 INFO:teuthology.orchestra.run.smithi061.stdout: "require_osd_release": "reef", 2023-12-06T18:31:32.243 INFO:teuthology.orchestra.run.smithi061.stdout: "allow_crimson": false, 2023-12-06T18:31:32.243 INFO:teuthology.orchestra.run.smithi061.stdout: "pools": [ 2023-12-06T18:31:32.243 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.243 INFO:teuthology.orchestra.run.smithi061.stdout: "pool": 1, 2023-12-06T18:31:32.243 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_name": ".mgr", 2023-12-06T18:31:32.244 INFO:teuthology.orchestra.run.smithi061.stdout: "create_time": "2023-12-06T18:26:52.648377+0000", 2023-12-06T18:31:32.244 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-12-06T18:31:32.244 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_names": "hashpspool", 2023-12-06T18:31:32.244 INFO:teuthology.orchestra.run.smithi061.stdout: "type": 1, 2023-12-06T18:31:32.244 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 2, 2023-12-06T18:31:32.244 INFO:teuthology.orchestra.run.smithi061.stdout: "min_size": 1, 2023-12-06T18:31:32.244 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_rule": 0, 2023-12-06T18:31:32.244 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_count": 0, 2023-12-06T18:31:32.244 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_target": 0, 2023-12-06T18:31:32.244 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_barrier": 0, 2023-12-06T18:31:32.244 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-06T18:31:32.244 INFO:teuthology.orchestra.run.smithi061.stdout: "object_hash": 2, 2023-12-06T18:31:32.244 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscale_mode": "off", 2023-12-06T18:31:32.244 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num": 1, 2023-12-06T18:31:32.244 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num": 1, 2023-12-06T18:31:32.245 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num_target": 1, 2023-12-06T18:31:32.245 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_target": 1, 2023-12-06T18:31:32.245 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_pending": 1, 2023-12-06T18:31:32.245 INFO:teuthology.orchestra.run.smithi061.stdout: "last_pg_merge_meta": { 2023-12-06T18:31:32.245 INFO:teuthology.orchestra.run.smithi061.stdout: "source_pgid": "0.0", 2023-12-06T18:31:32.245 INFO:teuthology.orchestra.run.smithi061.stdout: "ready_epoch": 0, 2023-12-06T18:31:32.245 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_started": 0, 2023-12-06T18:31:32.245 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_clean": 0, 2023-12-06T18:31:32.245 INFO:teuthology.orchestra.run.smithi061.stdout: "source_version": "0'0", 2023-12-06T18:31:32.245 INFO:teuthology.orchestra.run.smithi061.stdout: "target_version": "0'0" 2023-12-06T18:31:32.245 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.245 INFO:teuthology.orchestra.run.smithi061.stdout: "last_change": "19", 2023-12-06T18:31:32.245 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend": "0", 2023-12-06T18:31:32.246 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-06T18:31:32.246 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_preluminous": "0", 2023-12-06T18:31:32.246 INFO:teuthology.orchestra.run.smithi061.stdout: "auid": 0, 2023-12-06T18:31:32.246 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_mode": "selfmanaged", 2023-12-06T18:31:32.246 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_seq": 0, 2023-12-06T18:31:32.246 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_epoch": 0, 2023-12-06T18:31:32.246 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_snaps": [], 2023-12-06T18:31:32.246 INFO:teuthology.orchestra.run.smithi061.stdout: "removed_snaps": "[]", 2023-12-06T18:31:32.246 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_bytes": 0, 2023-12-06T18:31:32.246 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_objects": 0, 2023-12-06T18:31:32.247 INFO:teuthology.orchestra.run.smithi061.stdout: "tiers": [], 2023-12-06T18:31:32.247 INFO:teuthology.orchestra.run.smithi061.stdout: "tier_of": -1, 2023-12-06T18:31:32.247 INFO:teuthology.orchestra.run.smithi061.stdout: "read_tier": -1, 2023-12-06T18:31:32.247 INFO:teuthology.orchestra.run.smithi061.stdout: "write_tier": -1, 2023-12-06T18:31:32.247 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_mode": "none", 2023-12-06T18:31:32.247 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_bytes": 0, 2023-12-06T18:31:32.247 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_objects": 0, 2023-12-06T18:31:32.247 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-06T18:31:32.247 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-06T18:31:32.247 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-06T18:31:32.247 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_flush_age": 0, 2023-12-06T18:31:32.247 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_evict_age": 0, 2023-12-06T18:31:32.247 INFO:teuthology.orchestra.run.smithi061.stdout: "erasure_code_profile": "", 2023-12-06T18:31:32.247 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_params": { 2023-12-06T18:31:32.247 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "none" 2023-12-06T18:31:32.248 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.248 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_period": 0, 2023-12-06T18:31:32.248 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_count": 0, 2023-12-06T18:31:32.248 INFO:teuthology.orchestra.run.smithi061.stdout: "use_gmt_hitset": true, 2023-12-06T18:31:32.248 INFO:teuthology.orchestra.run.smithi061.stdout: "min_read_recency_for_promote": 0, 2023-12-06T18:31:32.248 INFO:teuthology.orchestra.run.smithi061.stdout: "min_write_recency_for_promote": 0, 2023-12-06T18:31:32.248 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_grade_decay_rate": 0, 2023-12-06T18:31:32.248 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_search_last_n": 0, 2023-12-06T18:31:32.248 INFO:teuthology.orchestra.run.smithi061.stdout: "grade_table": [], 2023-12-06T18:31:32.248 INFO:teuthology.orchestra.run.smithi061.stdout: "stripe_width": 0, 2023-12-06T18:31:32.248 INFO:teuthology.orchestra.run.smithi061.stdout: "expected_num_objects": 0, 2023-12-06T18:31:32.248 INFO:teuthology.orchestra.run.smithi061.stdout: "fast_read": false, 2023-12-06T18:31:32.248 INFO:teuthology.orchestra.run.smithi061.stdout: "options": { 2023-12-06T18:31:32.248 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_max": 32, 2023-12-06T18:31:32.248 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_min": 1 2023-12-06T18:31:32.249 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.249 INFO:teuthology.orchestra.run.smithi061.stdout: "application_metadata": { 2023-12-06T18:31:32.249 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr": {} 2023-12-06T18:31:32.249 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.249 INFO:teuthology.orchestra.run.smithi061.stdout: "read_balance": { 2023-12-06T18:31:32.249 INFO:teuthology.orchestra.run.smithi061.stdout: "score_acting": 6.9000000953674316, 2023-12-06T18:31:32.249 INFO:teuthology.orchestra.run.smithi061.stdout: "score_stable": 6.9000000953674316, 2023-12-06T18:31:32.249 INFO:teuthology.orchestra.run.smithi061.stdout: "optimal_score": 0.28999999165534973, 2023-12-06T18:31:32.249 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_acting": 2, 2023-12-06T18:31:32.249 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_stable": 2, 2023-12-06T18:31:32.249 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity_weighted": 0.82932090759277344, 2023-12-06T18:31:32.249 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity": 0.94999998807907104, 2023-12-06T18:31:32.249 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2023-12-06T18:31:32.249 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.250 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.250 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.250 INFO:teuthology.orchestra.run.smithi061.stdout: "pool": 2, 2023-12-06T18:31:32.250 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_name": "rbd", 2023-12-06T18:31:32.250 INFO:teuthology.orchestra.run.smithi061.stdout: "create_time": "2023-12-06T18:27:01.712581+0000", 2023-12-06T18:31:32.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-12-06T18:31:32.250 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_names": "hashpspool", 2023-12-06T18:31:32.250 INFO:teuthology.orchestra.run.smithi061.stdout: "type": 1, 2023-12-06T18:31:32.250 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 2, 2023-12-06T18:31:32.250 INFO:teuthology.orchestra.run.smithi061.stdout: "min_size": 1, 2023-12-06T18:31:32.250 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_rule": 0, 2023-12-06T18:31:32.250 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_count": 0, 2023-12-06T18:31:32.250 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_target": 0, 2023-12-06T18:31:32.251 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_barrier": 0, 2023-12-06T18:31:32.251 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-06T18:31:32.251 INFO:teuthology.orchestra.run.smithi061.stdout: "object_hash": 2, 2023-12-06T18:31:32.251 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscale_mode": "off", 2023-12-06T18:31:32.251 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num": 8, 2023-12-06T18:31:32.251 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num": 8, 2023-12-06T18:31:32.251 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num_target": 8, 2023-12-06T18:31:32.251 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_target": 8, 2023-12-06T18:31:32.251 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_pending": 8, 2023-12-06T18:31:32.251 INFO:teuthology.orchestra.run.smithi061.stdout: "last_pg_merge_meta": { 2023-12-06T18:31:32.251 INFO:teuthology.orchestra.run.smithi061.stdout: "source_pgid": "0.0", 2023-12-06T18:31:32.251 INFO:teuthology.orchestra.run.smithi061.stdout: "ready_epoch": 0, 2023-12-06T18:31:32.251 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_started": 0, 2023-12-06T18:31:32.252 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_clean": 0, 2023-12-06T18:31:32.252 INFO:teuthology.orchestra.run.smithi061.stdout: "source_version": "0'0", 2023-12-06T18:31:32.252 INFO:teuthology.orchestra.run.smithi061.stdout: "target_version": "0'0" 2023-12-06T18:31:32.252 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.252 INFO:teuthology.orchestra.run.smithi061.stdout: "last_change": "24", 2023-12-06T18:31:32.252 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend": "0", 2023-12-06T18:31:32.252 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-06T18:31:32.252 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_preluminous": "0", 2023-12-06T18:31:32.252 INFO:teuthology.orchestra.run.smithi061.stdout: "auid": 0, 2023-12-06T18:31:32.252 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_mode": "selfmanaged", 2023-12-06T18:31:32.253 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_seq": 0, 2023-12-06T18:31:32.253 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_epoch": 0, 2023-12-06T18:31:32.253 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_snaps": [], 2023-12-06T18:31:32.253 INFO:teuthology.orchestra.run.smithi061.stdout: "removed_snaps": "[]", 2023-12-06T18:31:32.253 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_bytes": 0, 2023-12-06T18:31:32.253 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_objects": 0, 2023-12-06T18:31:32.253 INFO:teuthology.orchestra.run.smithi061.stdout: "tiers": [], 2023-12-06T18:31:32.253 INFO:teuthology.orchestra.run.smithi061.stdout: "tier_of": -1, 2023-12-06T18:31:32.253 INFO:teuthology.orchestra.run.smithi061.stdout: "read_tier": -1, 2023-12-06T18:31:32.254 INFO:teuthology.orchestra.run.smithi061.stdout: "write_tier": -1, 2023-12-06T18:31:32.254 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_mode": "none", 2023-12-06T18:31:32.254 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_bytes": 0, 2023-12-06T18:31:32.254 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_objects": 0, 2023-12-06T18:31:32.254 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-06T18:31:32.254 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-06T18:31:32.254 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-06T18:31:32.254 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_flush_age": 0, 2023-12-06T18:31:32.254 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_evict_age": 0, 2023-12-06T18:31:32.254 INFO:teuthology.orchestra.run.smithi061.stdout: "erasure_code_profile": "", 2023-12-06T18:31:32.254 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_params": { 2023-12-06T18:31:32.254 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "none" 2023-12-06T18:31:32.254 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.254 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_period": 0, 2023-12-06T18:31:32.255 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_count": 0, 2023-12-06T18:31:32.255 INFO:teuthology.orchestra.run.smithi061.stdout: "use_gmt_hitset": true, 2023-12-06T18:31:32.255 INFO:teuthology.orchestra.run.smithi061.stdout: "min_read_recency_for_promote": 0, 2023-12-06T18:31:32.255 INFO:teuthology.orchestra.run.smithi061.stdout: "min_write_recency_for_promote": 0, 2023-12-06T18:31:32.255 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_grade_decay_rate": 0, 2023-12-06T18:31:32.255 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_search_last_n": 0, 2023-12-06T18:31:32.255 INFO:teuthology.orchestra.run.smithi061.stdout: "grade_table": [], 2023-12-06T18:31:32.255 INFO:teuthology.orchestra.run.smithi061.stdout: "stripe_width": 0, 2023-12-06T18:31:32.255 INFO:teuthology.orchestra.run.smithi061.stdout: "expected_num_objects": 0, 2023-12-06T18:31:32.255 INFO:teuthology.orchestra.run.smithi061.stdout: "fast_read": false, 2023-12-06T18:31:32.255 INFO:teuthology.orchestra.run.smithi061.stdout: "options": {}, 2023-12-06T18:31:32.255 INFO:teuthology.orchestra.run.smithi061.stdout: "application_metadata": { 2023-12-06T18:31:32.255 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd": {} 2023-12-06T18:31:32.255 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.255 INFO:teuthology.orchestra.run.smithi061.stdout: "read_balance": { 2023-12-06T18:31:32.255 INFO:teuthology.orchestra.run.smithi061.stdout: "score_acting": 3.5, 2023-12-06T18:31:32.255 INFO:teuthology.orchestra.run.smithi061.stdout: "score_stable": 3.5, 2023-12-06T18:31:32.256 INFO:teuthology.orchestra.run.smithi061.stdout: "optimal_score": 1, 2023-12-06T18:31:32.256 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_acting": 3.5, 2023-12-06T18:31:32.256 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_stable": 3.5, 2023-12-06T18:31:32.256 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity_weighted": 0.82932090759277344, 2023-12-06T18:31:32.256 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity": 0.94999998807907104, 2023-12-06T18:31:32.256 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2023-12-06T18:31:32.256 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.256 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.256 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:31:32.256 INFO:teuthology.orchestra.run.smithi061.stdout: "osds": [ 2023-12-06T18:31:32.256 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.256 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 0, 2023-12-06T18:31:32.256 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "5e6e17b9-e72e-4e34-a531-0830d659bfe0", 2023-12-06T18:31:32.256 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:31:32.256 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:31:32.257 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:31:32.257 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:31:32.257 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:31:32.257 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:31:32.257 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 16, 2023-12-06T18:31:32.257 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 0, 2023-12-06T18:31:32.257 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:31:32.257 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:31:32.257 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:31:32.257 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.257 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.257 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.257 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6805", 2023-12-06T18:31:32.257 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:31:32.257 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.257 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.258 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.258 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:31:32.258 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.258 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.258 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.258 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6806", 2023-12-06T18:31:32.258 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:31:32.258 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.258 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.258 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.258 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:31:32.258 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.258 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.259 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.259 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6808", 2023-12-06T18:31:32.259 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:31:32.259 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.259 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.259 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.259 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:31:32.259 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.259 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.259 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.259 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6807", 2023-12-06T18:31:32.259 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:31:32.259 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.259 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.259 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.260 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6805/281956997", 2023-12-06T18:31:32.260 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6806/281956997", 2023-12-06T18:31:32.260 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6808/281956997", 2023-12-06T18:31:32.260 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6807/281956997", 2023-12-06T18:31:32.260 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:31:32.260 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:31:32.260 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:31:32.260 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.260 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.260 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.260 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 1, 2023-12-06T18:31:32.260 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "b78c49b0-360d-4351-b44e-65ccaaeb768f", 2023-12-06T18:31:32.260 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:31:32.260 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:31:32.260 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:31:32.260 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:31:32.261 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:31:32.261 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:31:32.261 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 16, 2023-12-06T18:31:32.261 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 44, 2023-12-06T18:31:32.261 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:31:32.261 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:31:32.261 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:31:32.261 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.261 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.262 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.262 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6809", 2023-12-06T18:31:32.262 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:31:32.262 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.262 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.262 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.263 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:31:32.263 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.263 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.263 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.263 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6810", 2023-12-06T18:31:32.263 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:31:32.263 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.263 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.263 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.263 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:31:32.263 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.264 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.264 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.264 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6812", 2023-12-06T18:31:32.264 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:31:32.264 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.264 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.264 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.264 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:31:32.264 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.264 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.264 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.264 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6811", 2023-12-06T18:31:32.264 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:31:32.264 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.265 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.265 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.265 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6809/3306625622", 2023-12-06T18:31:32.265 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6810/3306625622", 2023-12-06T18:31:32.265 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6812/3306625622", 2023-12-06T18:31:32.265 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6811/3306625622", 2023-12-06T18:31:32.265 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:31:32.265 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:31:32.265 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:31:32.265 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.265 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.265 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.265 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 2, 2023-12-06T18:31:32.265 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "a2af3151-a4da-4c34-8fb6-78abf035785b", 2023-12-06T18:31:32.265 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:31:32.265 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:31:32.266 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:31:32.266 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 0.6345672607421875, 2023-12-06T18:31:32.266 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:31:32.266 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:31:32.266 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 16, 2023-12-06T18:31:32.266 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 35, 2023-12-06T18:31:32.266 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:31:32.266 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:31:32.266 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:31:32.266 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.266 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.266 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.266 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6801", 2023-12-06T18:31:32.267 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:31:32.267 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.267 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.267 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.267 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:31:32.267 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.267 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.267 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.267 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6802", 2023-12-06T18:31:32.267 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:31:32.267 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.267 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.267 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.267 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:31:32.267 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.268 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.268 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.268 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6804", 2023-12-06T18:31:32.268 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:31:32.268 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.268 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.268 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.268 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:31:32.268 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.268 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.268 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.268 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6803", 2023-12-06T18:31:32.268 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:31:32.268 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.268 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.269 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6801/3589270120", 2023-12-06T18:31:32.269 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6802/3589270120", 2023-12-06T18:31:32.269 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6804/3589270120", 2023-12-06T18:31:32.269 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6803/3589270120", 2023-12-06T18:31:32.269 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:31:32.269 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:31:32.269 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:31:32.269 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.269 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 3, 2023-12-06T18:31:32.269 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "1c754ed6-c535-454f-b35b-00f62cd6ce5a", 2023-12-06T18:31:32.269 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:31:32.269 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:31:32.270 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:31:32.270 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:31:32.270 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:31:32.270 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:31:32.270 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 18, 2023-12-06T18:31:32.270 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 44, 2023-12-06T18:31:32.270 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:31:32.270 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:31:32.270 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:31:32.270 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.270 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.270 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.270 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6813", 2023-12-06T18:31:32.270 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:31:32.271 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.271 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.271 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.271 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:31:32.271 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.271 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.271 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.271 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6814", 2023-12-06T18:31:32.271 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:31:32.271 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.271 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.271 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.271 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:31:32.271 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.272 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.272 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.272 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6816", 2023-12-06T18:31:32.272 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:31:32.272 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.272 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.272 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.272 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:31:32.272 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.272 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.272 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.272 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6815", 2023-12-06T18:31:32.272 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:31:32.272 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.272 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.273 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.273 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6813/2849920842", 2023-12-06T18:31:32.273 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6814/2849920842", 2023-12-06T18:31:32.273 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6816/2849920842", 2023-12-06T18:31:32.273 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6815/2849920842", 2023-12-06T18:31:32.273 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:31:32.273 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:31:32.273 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:31:32.273 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.273 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.273 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.274 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 4, 2023-12-06T18:31:32.274 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "766572af-b9ca-4f38-927d-36f26101ba51", 2023-12-06T18:31:32.274 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:31:32.274 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:31:32.274 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:31:32.274 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:31:32.274 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:31:32.275 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:31:32.275 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 15, 2023-12-06T18:31:32.275 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 38, 2023-12-06T18:31:32.275 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:31:32.275 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:31:32.275 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:31:32.275 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.276 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.276 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6800", 2023-12-06T18:31:32.276 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:31:32.276 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.276 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.276 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:31:32.276 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.276 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.276 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6801", 2023-12-06T18:31:32.276 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:31:32.276 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.276 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.277 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:31:32.277 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.277 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.277 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.277 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6803", 2023-12-06T18:31:32.277 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:31:32.277 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.277 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.277 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.277 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:31:32.277 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.277 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.277 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.277 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6802", 2023-12-06T18:31:32.277 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:31:32.278 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.278 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.278 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.278 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6800/2778770069", 2023-12-06T18:31:32.278 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6801/2778770069", 2023-12-06T18:31:32.278 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6803/2778770069", 2023-12-06T18:31:32.278 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6802/2778770069", 2023-12-06T18:31:32.278 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:31:32.278 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:31:32.278 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:31:32.278 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.278 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.278 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.279 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 5, 2023-12-06T18:31:32.279 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "7a566b60-1543-41d7-894c-e2fbb63e48dc", 2023-12-06T18:31:32.279 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:31:32.279 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:31:32.279 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:31:32.279 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:31:32.279 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:31:32.279 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:31:32.279 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 15, 2023-12-06T18:31:32.279 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 61, 2023-12-06T18:31:32.279 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:31:32.279 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:31:32.279 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:31:32.279 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.280 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.280 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6809", 2023-12-06T18:31:32.280 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:31:32.280 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.280 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.280 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:31:32.280 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.280 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.280 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6810", 2023-12-06T18:31:32.280 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:31:32.280 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.280 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.281 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:31:32.281 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.281 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.281 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6814", 2023-12-06T18:31:32.281 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:31:32.281 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.281 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.281 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:31:32.281 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.281 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.281 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6812", 2023-12-06T18:31:32.281 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:31:32.282 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.282 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.282 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6809/263671335", 2023-12-06T18:31:32.282 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6810/263671335", 2023-12-06T18:31:32.282 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6814/263671335", 2023-12-06T18:31:32.282 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6812/263671335", 2023-12-06T18:31:32.282 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:31:32.282 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:31:32.282 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:31:32.282 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.283 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 6, 2023-12-06T18:31:32.283 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "28cc2358-4ac4-40b2-91a4-20edfe625a73", 2023-12-06T18:31:32.283 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:31:32.283 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:31:32.283 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:31:32.283 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 0, 2023-12-06T18:31:32.283 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:31:32.283 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:31:32.283 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 14, 2023-12-06T18:31:32.283 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 25, 2023-12-06T18:31:32.284 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:31:32.284 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:31:32.284 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:31:32.284 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.284 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.284 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6808", 2023-12-06T18:31:32.284 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:31:32.284 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.284 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.284 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:31:32.284 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.284 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.285 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6811", 2023-12-06T18:31:32.285 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:31:32.285 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.285 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.285 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:31:32.285 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.285 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.285 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6815", 2023-12-06T18:31:32.285 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:31:32.286 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.286 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.286 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:31:32.286 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.286 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.286 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6813", 2023-12-06T18:31:32.286 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:31:32.287 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.287 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.287 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6808/4245639079", 2023-12-06T18:31:32.287 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6811/4245639079", 2023-12-06T18:31:32.287 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6815/4245639079", 2023-12-06T18:31:32.287 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6813/4245639079", 2023-12-06T18:31:32.287 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:31:32.287 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:31:32.287 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:31:32.287 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.288 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.288 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 7, 2023-12-06T18:31:32.288 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "8f79279b-d2d0-438e-b3c2-88df1241ff30", 2023-12-06T18:31:32.288 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:31:32.288 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:31:32.288 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:31:32.288 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:31:32.288 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 14, 2023-12-06T18:31:32.288 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 37, 2023-12-06T18:31:32.288 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 61, 2023-12-06T18:31:32.288 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 62, 2023-12-06T18:31:32.288 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 38, 2023-12-06T18:31:32.288 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:31:32.288 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:31:32.289 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.289 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.289 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.289 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6804", 2023-12-06T18:31:32.289 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3075390540 2023-12-06T18:31:32.289 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.289 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.289 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.289 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:31:32.289 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.289 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.289 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.289 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6805", 2023-12-06T18:31:32.289 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3075390540 2023-12-06T18:31:32.289 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.290 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.290 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.290 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:31:32.290 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.290 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.290 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6807", 2023-12-06T18:31:32.290 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3075390540 2023-12-06T18:31:32.290 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.290 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.290 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.290 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:31:32.290 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:32.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.291 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:32.291 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6806", 2023-12-06T18:31:32.291 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3075390540 2023-12-06T18:31:32.291 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.291 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.291 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6804/3075390540", 2023-12-06T18:31:32.291 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6805/3075390540", 2023-12-06T18:31:32.291 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6807/3075390540", 2023-12-06T18:31:32.291 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6806/3075390540", 2023-12-06T18:31:32.291 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:31:32.291 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:31:32.291 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:31:32.291 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.291 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.292 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:31:32.292 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_xinfo": [ 2023-12-06T18:31:32.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.292 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 0, 2023-12-06T18:31:32.292 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:31:32.292 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:31:32.292 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:31:32.292 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:31:32.292 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:31:32.292 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:52.171904+0000", 2023-12-06T18:31:32.292 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:31:32.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.292 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 1, 2023-12-06T18:31:32.293 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:31:32.293 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:31:32.293 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:31:32.293 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:31:32.293 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:31:32.293 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:52.428840+0000", 2023-12-06T18:31:32.293 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:31:32.293 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.293 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.293 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 2, 2023-12-06T18:31:32.293 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:31:32.293 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:31:32.293 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:31:32.293 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:31:32.294 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:31:32.294 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:51.681954+0000", 2023-12-06T18:31:32.294 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:31:32.294 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.294 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.294 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 3, 2023-12-06T18:31:32.294 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:31:32.294 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:31:32.294 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:31:32.294 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:31:32.294 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:31:32.294 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:53.687417+0000", 2023-12-06T18:31:32.295 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:31:32.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.295 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.295 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 4, 2023-12-06T18:31:32.295 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:31:32.295 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:31:32.295 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:31:32.295 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:31:32.295 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:31:32.295 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:50.158682+0000", 2023-12-06T18:31:32.295 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:31:32.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.295 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.295 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 5, 2023-12-06T18:31:32.295 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:31:32.295 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:31:32.296 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:31:32.296 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:31:32.296 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:31:32.296 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:50.443807+0000", 2023-12-06T18:31:32.296 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:31:32.296 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.296 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.296 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 6, 2023-12-06T18:31:32.296 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:31:32.296 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:31:32.296 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:31:32.297 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:31:32.297 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:31:32.297 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:49.750276+0000", 2023-12-06T18:31:32.297 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:31:32.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.297 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.297 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 7, 2023-12-06T18:31:32.297 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:31:32.297 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:31:32.297 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:31:32.297 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:31:32.297 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:31:32.297 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:49.770987+0000", 2023-12-06T18:31:32.297 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 38 2023-12-06T18:31:32.297 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.297 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:31:32.297 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_upmap": [ 2023-12-06T18:31:32.298 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:32.298 INFO:teuthology.orchestra.run.smithi061.stdout: "pgid": "2.7", 2023-12-06T18:31:32.298 INFO:teuthology.orchestra.run.smithi061.stdout: "osds": [ 2023-12-06T18:31:32.298 INFO:teuthology.orchestra.run.smithi061.stdout: 5, 2023-12-06T18:31:32.298 INFO:teuthology.orchestra.run.smithi061.stdout: 3 2023-12-06T18:31:32.298 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:32.298 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.298 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:31:32.299 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_upmap_items": [], 2023-12-06T18:31:32.299 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_upmap_primaries": [], 2023-12-06T18:31:32.299 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_temp": [], 2023-12-06T18:31:32.299 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_temp": [], 2023-12-06T18:31:32.299 INFO:teuthology.orchestra.run.smithi061.stdout: "blocklist": {}, 2023-12-06T18:31:32.299 INFO:teuthology.orchestra.run.smithi061.stdout: "range_blocklist": {}, 2023-12-06T18:31:32.300 INFO:teuthology.orchestra.run.smithi061.stdout: "erasure_code_profiles": { 2023-12-06T18:31:32.300 INFO:teuthology.orchestra.run.smithi061.stdout: "default": { 2023-12-06T18:31:32.300 INFO:teuthology.orchestra.run.smithi061.stdout: "crush-failure-domain": "osd", 2023-12-06T18:31:32.300 INFO:teuthology.orchestra.run.smithi061.stdout: "k": "2", 2023-12-06T18:31:32.300 INFO:teuthology.orchestra.run.smithi061.stdout: "m": "1", 2023-12-06T18:31:32.300 INFO:teuthology.orchestra.run.smithi061.stdout: "plugin": "jerasure", 2023-12-06T18:31:32.300 INFO:teuthology.orchestra.run.smithi061.stdout: "technique": "reed_sol_van" 2023-12-06T18:31:32.301 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.301 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:32.301 INFO:teuthology.orchestra.run.smithi061.stdout: "removed_snaps_queue": [], 2023-12-06T18:31:32.301 INFO:teuthology.orchestra.run.smithi061.stdout: "new_removed_snaps": [], 2023-12-06T18:31:32.301 INFO:teuthology.orchestra.run.smithi061.stdout: "new_purged_snaps": [], 2023-12-06T18:31:32.301 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_node_flags": {}, 2023-12-06T18:31:32.301 INFO:teuthology.orchestra.run.smithi061.stdout: "device_class_flags": {}, 2023-12-06T18:31:32.301 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode": { 2023-12-06T18:31:32.302 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode_enabled": false, 2023-12-06T18:31:32.302 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_bucket_count": 0, 2023-12-06T18:31:32.302 INFO:teuthology.orchestra.run.smithi061.stdout: "degraded_stretch_mode": 0, 2023-12-06T18:31:32.302 INFO:teuthology.orchestra.run.smithi061.stdout: "recovering_stretch_mode": 0, 2023-12-06T18:31:32.302 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode_bucket": 0 2023-12-06T18:31:32.302 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:32.302 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:32.303 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:32.304 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:32.305 INFO:tasks.thrashosds.thrasher:j is {'epoch': 74, 'fsid': '497e3283-cc16-40ff-a72c-a7487474fb4b', 'created': '2023-12-06T18:26:07.122862+0000', 'modified': '2023-12-06T18:31:29.484913+0000', 'last_up_change': '2023-12-06T18:30:04.956811+0000', 'last_in_change': '2023-12-06T18:28:45.621304+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 8, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-06T18:26:52.648377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8293209075927734, 'average_primary_affinity': 0.949999988079071, 'average_primary_affinity_weighted': 0.8299999833106995}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-12-06T18:27:01.712581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 3.5, 'score_stable': 3.5, 'optimal_score': 1, 'raw_score_acting': 3.5, 'raw_score_stable': 3.5, 'primary_affinity_weighted': 0.8293209075927734, 'average_primary_affinity': 0.949999988079071, 'average_primary_affinity_weighted': 0.8299999833106995}}], 'osds': [{'osd': 0, 'uuid': '5e6e17b9-e72e-4e34-a531-0830d659bfe0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6805', 'nonce': 281956997}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6806', 'nonce': 281956997}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6808', 'nonce': 281956997}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6807', 'nonce': 281956997}]}, 'public_addr': '172.21.15.61:6805/281956997', 'cluster_addr': '172.21.15.61:6806/281956997', 'heartbeat_back_addr': '172.21.15.61:6808/281956997', 'heartbeat_front_addr': '172.21.15.61:6807/281956997', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b78c49b0-360d-4351-b44e-65ccaaeb768f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6809', 'nonce': 3306625622}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6810', 'nonce': 3306625622}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6812', 'nonce': 3306625622}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6811', 'nonce': 3306625622}]}, 'public_addr': '172.21.15.61:6809/3306625622', 'cluster_addr': '172.21.15.61:6810/3306625622', 'heartbeat_back_addr': '172.21.15.61:6812/3306625622', 'heartbeat_front_addr': '172.21.15.61:6811/3306625622', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'a2af3151-a4da-4c34-8fb6-78abf035785b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.6345672607421875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 35, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6801', 'nonce': 3589270120}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6802', 'nonce': 3589270120}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6804', 'nonce': 3589270120}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6803', 'nonce': 3589270120}]}, 'public_addr': '172.21.15.61:6801/3589270120', 'cluster_addr': '172.21.15.61:6802/3589270120', 'heartbeat_back_addr': '172.21.15.61:6804/3589270120', 'heartbeat_front_addr': '172.21.15.61:6803/3589270120', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '1c754ed6-c535-454f-b35b-00f62cd6ce5a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6813', 'nonce': 2849920842}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6814', 'nonce': 2849920842}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6816', 'nonce': 2849920842}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6815', 'nonce': 2849920842}]}, 'public_addr': '172.21.15.61:6813/2849920842', 'cluster_addr': '172.21.15.61:6814/2849920842', 'heartbeat_back_addr': '172.21.15.61:6816/2849920842', 'heartbeat_front_addr': '172.21.15.61:6815/2849920842', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '766572af-b9ca-4f38-927d-36f26101ba51', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 38, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6800', 'nonce': 2778770069}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6801', 'nonce': 2778770069}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6803', 'nonce': 2778770069}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6802', 'nonce': 2778770069}]}, 'public_addr': '172.21.15.152:6800/2778770069', 'cluster_addr': '172.21.15.152:6801/2778770069', 'heartbeat_back_addr': '172.21.15.152:6803/2778770069', 'heartbeat_front_addr': '172.21.15.152:6802/2778770069', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7a566b60-1543-41d7-894c-e2fbb63e48dc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 61, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6809', 'nonce': 263671335}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6810', 'nonce': 263671335}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6814', 'nonce': 263671335}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6812', 'nonce': 263671335}]}, 'public_addr': '172.21.15.152:6809/263671335', 'cluster_addr': '172.21.15.152:6810/263671335', 'heartbeat_back_addr': '172.21.15.152:6814/263671335', 'heartbeat_front_addr': '172.21.15.152:6812/263671335', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '28cc2358-4ac4-40b2-91a4-20edfe625a73', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6808', 'nonce': 4245639079}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6811', 'nonce': 4245639079}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6815', 'nonce': 4245639079}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6813', 'nonce': 4245639079}]}, 'public_addr': '172.21.15.152:6808/4245639079', 'cluster_addr': '172.21.15.152:6811/4245639079', 'heartbeat_back_addr': '172.21.15.152:6815/4245639079', 'heartbeat_front_addr': '172.21.15.152:6813/4245639079', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '8f79279b-d2d0-438e-b3c2-88df1241ff30', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 14, 'last_clean_end': 37, 'up_from': 61, 'up_thru': 62, 'down_at': 38, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6804', 'nonce': 3075390540}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6805', 'nonce': 3075390540}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6807', 'nonce': 3075390540}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6806', 'nonce': 3075390540}]}, 'public_addr': '172.21.15.152:6804/3075390540', 'cluster_addr': '172.21.15.152:6805/3075390540', 'heartbeat_back_addr': '172.21.15.152:6807/3075390540', 'heartbeat_front_addr': '172.21.15.152:6806/3075390540', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:52.171904+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:52.428840+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:51.681954+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:53.687417+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:50.158682+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:50.443807+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:49.750276+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:49.770987+0000', 'dead_epoch': 38}], 'pg_upmap': [{'pgid': '2.7', 'osds': [5, 3]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2023-12-06T18:31:32.306 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:31:32.307 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:31:32.346 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:32.342+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:32.436 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:32.436 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:32.439 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:32.435+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:32.446 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:31:32.540 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:32.536+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:32.622 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:32.622 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:32.631 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:31:32.641 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:32.637+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:32.742 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:32.739+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:32.760 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:32.761 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:32.770 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:31:32.844 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:32.840+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:32.852 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":34,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"c","quorum_age":15,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:31:32.901 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:32.901 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:32.910 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:31:32.945 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:32.941+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:32.985 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:31:32.985 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:31:32.997 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2023-12-06T18:31:32.261992+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227004,"kb_used_data":5656,"kb_used_omap":15,"kb_used_meta":217008,"kb_avail":754747716,"statfs":{"total":773094113280,"available":772861661184,"internally_reserved":0,"allocated":5791744,"data_stored":2939166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15557,"internal_metadata":222217019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":67,"apply_latency_ms":67,"commit_latency_ns":67000000,"apply_latency_ns":67000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001226"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-12-06T18:31:22.771236+0000","last_change":"2023-12-06T18:29:09.051293+0000","last_active":"2023-12-06T18:31:22.771236+0000","last_peered":"2023-12-06T18:31:22.771236+0000","last_clean":"2023-12-06T18:31:22.771236+0000","last_became_active":"2023-12-06T18:29:09.045331+0000","last_became_peered":"2023-12-06T18:29:09.045331+0000","last_unstale":"2023-12-06T18:31:22.771236+0000","last_undegraded":"2023-12-06T18:31:22.771236+0000","last_fullsized":"2023-12-06T18:31:22.771236+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T02:46:01.854508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":58,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-12-06T18:31:24.505699+0000","last_change":"2023-12-06T18:27:04.883840+0000","last_active":"2023-12-06T18:31:24.505699+0000","last_peered":"2023-12-06T18:31:24.505699+0000","last_clean":"2023-12-06T18:31:24.505699+0000","last_became_active":"2023-12-06T18:27:04.858001+0000","last_became_peered":"2023-12-06T18:27:04.858001+0000","last_unstale":"2023-12-06T18:31:24.505699+0000","last_undegraded":"2023-12-06T18:31:24.505699+0000","last_fullsized":"2023-12-06T18:31:24.505699+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:44:02.916244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":61,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-12-06T18:31:26.389211+0000","last_change":"2023-12-06T18:27:04.336778+0000","last_active":"2023-12-06T18:31:26.389211+0000","last_peered":"2023-12-06T18:31:26.389211+0000","last_clean":"2023-12-06T18:31:26.389211+0000","last_became_active":"2023-12-06T18:27:04.321269+0000","last_became_peered":"2023-12-06T18:27:04.321269+0000","last_unstale":"2023-12-06T18:31:26.389211+0000","last_undegraded":"2023-12-06T18:31:26.389211+0000","last_fullsized":"2023-12-06T18:31:26.389211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T22:25:15.703579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-12-06T18:31:22.768661+0000","last_change":"2023-12-06T18:28:47.898016+0000","last_active":"2023-12-06T18:31:22.768661+0000","last_peered":"2023-12-06T18:31:22.768661+0000","last_clean":"2023-12-06T18:31:22.768661+0000","last_became_active":"2023-12-06T18:28:47.891814+0000","last_became_peered":"2023-12-06T18:28:47.891814+0000","last_unstale":"2023-12-06T18:31:22.768661+0000","last_undegraded":"2023-12-06T18:31:22.768661+0000","last_fullsized":"2023-12-06T18:31:22.768661+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T00:49:06.937273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":61,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-12-06T18:31:26.383951+0000","last_change":"2023-12-06T18:27:04.408081+0000","last_active":"2023-12-06T18:31:26.383951+0000","last_peered":"2023-12-06T18:31:26.383951+0000","last_clean":"2023-12-06T18:31:26.383951+0000","last_became_active":"2023-12-06T18:27:04.399765+0000","last_became_peered":"2023-12-06T18:27:04.399765+0000","last_unstale":"2023-12-06T18:31:26.383951+0000","last_undegraded":"2023-12-06T18:31:26.383951+0000","last_fullsized":"2023-12-06T18:31:26.383951+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:40:33.570114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-12-06T18:31:22.850936+0000","last_change":"2023-12-06T18:30:08.762262+0000","last_active":"2023-12-06T18:31:22.850936+0000","last_peered":"2023-12-06T18:31:22.850936+0000","last_clean":"2023-12-06T18:31:22.850936+0000","last_became_active":"2023-12-06T18:30:08.733421+0000","last_became_peered":"2023-12-06T18:30:08.733421+0000","last_unstale":"2023-12-06T18:31:22.850936+0000","last_undegraded":"2023-12-06T18:31:22.850936+0000","last_fullsized":"2023-12-06T18:31:22.850936+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:43:31.796397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":75,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-12-06T18:31:26.381345+0000","last_change":"2023-12-06T18:28:47.876556+0000","last_active":"2023-12-06T18:31:26.381345+0000","last_peered":"2023-12-06T18:31:26.381345+0000","last_clean":"2023-12-06T18:31:26.381345+0000","last_became_active":"2023-12-06T18:28:47.865169+0000","last_became_peered":"2023-12-06T18:28:47.865169+0000","last_unstale":"2023-12-06T18:31:26.381345+0000","last_undegraded":"2023-12-06T18:31:26.381345+0000","last_fullsized":"2023-12-06T18:31:26.381345+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:16:40.095542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":61,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-12-06T18:31:26.387373+0000","last_change":"2023-12-06T18:27:04.609381+0000","last_active":"2023-12-06T18:31:26.387373+0000","last_peered":"2023-12-06T18:31:26.387373+0000","last_clean":"2023-12-06T18:31:26.387373+0000","last_became_active":"2023-12-06T18:27:04.598701+0000","last_became_peered":"2023-12-06T18:27:04.598701+0000","last_unstale":"2023-12-06T18:31:26.387373+0000","last_undegraded":"2023-12-06T18:31:26.387373+0000","last_fullsized":"2023-12-06T18:31:26.387373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:29:58.962692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":193,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-12-06T18:31:22.639665+0000","last_change":"2023-12-06T18:28:48.207134+0000","last_active":"2023-12-06T18:31:22.639665+0000","last_peered":"2023-12-06T18:31:22.639665+0000","last_clean":"2023-12-06T18:31:22.639665+0000","last_became_active":"2023-12-06T18:28:47.875258+0000","last_became_peered":"2023-12-06T18:28:47.875258+0000","last_unstale":"2023-12-06T18:31:22.639665+0000","last_undegraded":"2023-12-06T18:31:22.639665+0000","last_fullsized":"2023-12-06T18:31:22.639665+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:14:32.667039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1204224,"data_stored":1193520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411386,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":634880,"data_stored":271778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":10.073,"5min":10.574,"15min":10.494999999999999},"min":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"max":{"1min":21.175000000000001,"5min":108.57599999999999,"15min":108.57599999999999},"last":10.832000000000001},{"interface":"front","average":{"1min":10.366,"5min":11.858000000000001,"15min":12.459},"min":{"1min":5.3440000000000003,"5min":3.7200000000000002,"15min":3.7200000000000002},"max":{"1min":18.559999999999999,"5min":128.09399999999999,"15min":128.09399999999999},"last":17.038}]},{"osd":1,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":11.718,"5min":12.234,"15min":12.782},"min":{"1min":7.8460000000000001,"5min":6.2539999999999996,"15min":6.2539999999999996},"max":{"1min":25.940999999999999,"5min":70.072000000000003,"15min":70.072000000000003},"last":20.196999999999999},{"interface":"front","average":{"1min":11.935,"5min":13.823,"15min":15.397},"min":{"1min":6.9640000000000004,"5min":6.444,"15min":6.444},"max":{"1min":18.939,"5min":32.920000000000002,"15min":32.920000000000002},"last":14.654}]},{"osd":2,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":14.039999999999999,"5min":14.664,"15min":14.712999999999999},"min":{"1min":8.5079999999999991,"5min":7.5149999999999997,"15min":7.5149999999999997},"max":{"1min":24.446000000000002,"5min":75.125,"15min":75.125},"last":53.505000000000003},{"interface":"front","average":{"1min":13.414999999999999,"5min":16.634,"15min":17.728000000000002},"min":{"1min":7.5419999999999998,"5min":7.5419999999999998,"15min":7.5419999999999998},"max":{"1min":25.609000000000002,"5min":69.403999999999996,"15min":69.403999999999996},"last":19.353999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:31:04 2023","interfaces":[{"interface":"back","average":{"1min":15.379,"5min":12.092000000000001,"15min":11.932},"min":{"1min":8.7910000000000004,"5min":5.8879999999999999,"15min":5.8879999999999999},"max":{"1min":27.518000000000001,"5min":27.518000000000001,"15min":27.518000000000001},"last":11.345000000000001},{"interface":"front","average":{"1min":12.076000000000001,"5min":15.141999999999999,"15min":15.778},"min":{"1min":7.8949999999999996,"5min":6.3099999999999996,"15min":6.3099999999999996},"max":{"1min":19.82,"5min":71.718000000000004,"15min":71.718000000000004},"last":9.6630000000000003}]},{"osd":5,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":18.186,"5min":18.327999999999999,"15min":17.914000000000001},"min":{"1min":10.417999999999999,"5min":8.3179999999999996,"15min":8.3179999999999996},"max":{"1min":33.335999999999999,"5min":75.751999999999995,"15min":75.751999999999995},"last":23.170999999999999},{"interface":"front","average":{"1min":17.651,"5min":17.765999999999998,"15min":18.141999999999999},"min":{"1min":8.4920000000000009,"5min":7.9269999999999996,"15min":7.9269999999999996},"max":{"1min":34.899999999999999,"5min":70.933000000000007,"15min":70.933000000000007},"last":17.646999999999998}]},{"osd":6,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":14.339,"5min":15.102,"15min":16.004000000000001},"min":{"1min":10.676,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":19.245999999999999,"5min":29.245000000000001,"15min":29.245000000000001},"last":13.446999999999999},{"interface":"front","average":{"1min":18.861000000000001,"5min":20.219999999999999,"15min":21.138999999999999},"min":{"1min":11.279,"5min":10.973000000000001,"15min":10.973000000000001},"max":{"1min":36.704999999999998,"5min":74.224000000000004,"15min":74.224000000000004},"last":16.356000000000002}]},{"osd":7,"last update":"Wed Dec 6 18:31:08 2023","interfaces":[{"interface":"back","average":{"1min":21.957999999999998,"5min":21.957999999999998,"15min":21.957999999999998},"min":{"1min":11.506,"5min":11.506,"15min":11.506},"max":{"1min":122.93000000000001,"5min":122.93000000000001,"15min":122.93000000000001},"last":12.766},{"interface":"front","average":{"1min":23.734000000000002,"5min":23.734000000000002,"15min":23.734000000000002},"min":{"1min":13.196,"5min":13.196,"15min":13.196},"max":{"1min":124.473,"5min":124.473,"15min":124.473},"last":21.239999999999998}]}]},{"osd":2,"up_from":16,"seq":68719476793,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":267427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:31:04 2023","interfaces":[{"interface":"back","average":{"1min":9.3260000000000005,"5min":9.8870000000000005,"15min":10.974},"min":{"1min":5.4409999999999998,"5min":4.609,"15min":4.609},"max":{"1min":20.771000000000001,"5min":38.740000000000002,"15min":38.740000000000002},"last":7.6550000000000002},{"interface":"front","average":{"1min":10.164999999999999,"5min":10.585000000000001,"15min":11.113},"min":{"1min":4.3280000000000003,"5min":4.3280000000000003,"15min":4.3280000000000003},"max":{"1min":17.048999999999999,"5min":33.387,"15min":33.387},"last":5.4699999999999998}]},{"osd":1,"last update":"Wed Dec 6 18:31:04 2023","interfaces":[{"interface":"back","average":{"1min":10.449999999999999,"5min":11.323,"15min":11.528},"min":{"1min":7.2229999999999999,"5min":5.9139999999999997,"15min":5.9139999999999997},"max":{"1min":20.832000000000001,"5min":42.920999999999999,"15min":42.920999999999999},"last":11.577},{"interface":"front","average":{"1min":13.028,"5min":13.336,"15min":13.786},"min":{"1min":6.1479999999999997,"5min":6.1479999999999997,"15min":6.1479999999999997},"max":{"1min":21.481999999999999,"5min":34.607999999999997,"15min":34.607999999999997},"last":14.217000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:31:04 2023","interfaces":[{"interface":"back","average":{"1min":12.215,"5min":16.128,"15min":18.634},"min":{"1min":7.2549999999999999,"5min":6.1260000000000003,"15min":6.1260000000000003},"max":{"1min":17.878,"5min":112.129,"15min":112.129},"last":12.853999999999999},{"interface":"front","average":{"1min":15.457000000000001,"5min":19.382000000000001,"15min":23.277999999999999},"min":{"1min":11.069000000000001,"5min":7.5510000000000002,"15min":7.5510000000000002},"max":{"1min":19.986999999999998,"5min":181.601,"15min":181.601},"last":13.496}]},{"osd":4,"last update":"Wed Dec 6 18:31:04 2023","interfaces":[{"interface":"back","average":{"1min":14.536,"5min":15.851000000000001,"15min":17.577999999999999},"min":{"1min":8.1349999999999998,"5min":6.609,"15min":6.609},"max":{"1min":23.422000000000001,"5min":97.088999999999999,"15min":97.088999999999999},"last":12.215},{"interface":"front","average":{"1min":14.007999999999999,"5min":17.056999999999999,"15min":20.344999999999999},"min":{"1min":6.0720000000000001,"5min":6.0720000000000001,"15min":6.0720000000000001},"max":{"1min":43.198,"5min":98.069000000000003,"15min":98.069000000000003},"last":10.74}]},{"osd":5,"last update":"Wed Dec 6 18:31:04 2023","interfaces":[{"interface":"back","average":{"1min":17.363,"5min":17.454999999999998,"15min":18.015000000000001},"min":{"1min":8.2100000000000009,"5min":7.8170000000000002,"15min":7.8170000000000002},"max":{"1min":22.111000000000001,"5min":57.628999999999998,"15min":57.628999999999998},"last":10.382999999999999},{"interface":"front","average":{"1min":14.59,"5min":15.462999999999999,"15min":15.714},"min":{"1min":8.5990000000000002,"5min":8.5990000000000002,"15min":8.5990000000000002},"max":{"1min":21.459,"5min":47.749000000000002,"15min":47.749000000000002},"last":15.031000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:31:04 2023","interfaces":[{"interface":"back","average":{"1min":16.934999999999999,"5min":29.489000000000001,"15min":41.509999999999998},"min":{"1min":11.621,"5min":8.7289999999999992,"15min":8.7289999999999992},"max":{"1min":26.984999999999999,"5min":498.38499999999999,"15min":498.38499999999999},"last":19.573},{"interface":"front","average":{"1min":18.338000000000001,"5min":30.812000000000001,"15min":43.351999999999997},"min":{"1min":9.7010000000000005,"5min":9.7010000000000005,"15min":9.7010000000000005},"max":{"1min":25.861000000000001,"5min":496.17399999999998,"15min":496.17399999999998},"last":18.649999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:31:10 2023","interfaces":[{"interface":"back","average":{"1min":18.922999999999998,"5min":18.922999999999998,"15min":18.922999999999998},"min":{"1min":11.933999999999999,"5min":11.933999999999999,"15min":11.933999999999999},"max":{"1min":49.456000000000003,"5min":49.456000000000003,"15min":49.456000000000003},"last":16.145},{"interface":"front","average":{"1min":20.155999999999999,"5min":20.155999999999999,"15min":20.155999999999999},"min":{"1min":13.901,"5min":13.901,"15min":13.901},"max":{"1min":40.656999999999996,"5min":40.656999999999996,"15min":40.656999999999996},"last":20.513999999999999}]}]},{"osd":1,"up_from":16,"seq":68719476794,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1024000,"data_stored":665267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":10.913,"5min":15.702,"15min":21.475999999999999},"min":{"1min":5.0890000000000004,"5min":5.0890000000000004,"15min":5.0890000000000004},"max":{"1min":18.282,"5min":238.87100000000001,"15min":238.87100000000001},"last":34.286000000000001},{"interface":"front","average":{"1min":9.7639999999999993,"5min":26.013999999999999,"15min":42.161999999999999},"min":{"1min":6.0350000000000001,"5min":4.4269999999999996,"15min":4.4269999999999996},"max":{"1min":16.062999999999999,"5min":661.33600000000001,"15min":661.33600000000001},"last":9.7129999999999992}]},{"osd":2,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":18.408999999999999,"5min":22.713000000000001,"15min":32.253},"min":{"1min":7.1559999999999997,"5min":6.6340000000000003,"15min":6.6340000000000003},"max":{"1min":93.168000000000006,"5min":384.315,"15min":384.315},"last":22.949999999999999},{"interface":"front","average":{"1min":22.268999999999998,"5min":31.728000000000002,"15min":46.054000000000002},"min":{"1min":12.404999999999999,"5min":5.7009999999999996,"15min":5.7009999999999996},"max":{"1min":94.230000000000004,"5min":656.98699999999997,"15min":656.98699999999997},"last":35.524999999999999}]},{"osd":3,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":11.784000000000001,"5min":11.923,"15min":13.041},"min":{"1min":6.6619999999999999,"5min":5.548,"15min":5.548},"max":{"1min":31.16,"5min":34.057000000000002,"15min":34.057000000000002},"last":24.472000000000001},{"interface":"front","average":{"1min":14.317,"5min":14.416,"15min":15.952999999999999},"min":{"1min":7.9299999999999997,"5min":6.5730000000000004,"15min":6.5730000000000004},"max":{"1min":38.133000000000003,"5min":38.133000000000003,"15min":38.133000000000003},"last":8.5679999999999996}]},{"osd":4,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":12.744999999999999,"5min":25.795000000000002,"15min":39.542000000000002},"min":{"1min":7.0839999999999996,"5min":6.2930000000000001,"15min":6.2930000000000001},"max":{"1min":29.344999999999999,"5min":447.67200000000003,"15min":447.67200000000003},"last":21.841000000000001},{"interface":"front","average":{"1min":18.292999999999999,"5min":31.774999999999999,"15min":47.042999999999999},"min":{"1min":12.065,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":44.043999999999997,"5min":630.572,"15min":630.572},"last":32.219999999999999}]},{"osd":5,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":16.238,"5min":29.34,"15min":41.911999999999999},"min":{"1min":8.0069999999999997,"5min":8.0069999999999997,"15min":8.0069999999999997},"max":{"1min":45.750999999999998,"5min":389.55000000000001,"15min":389.55000000000001},"last":7.0540000000000003},{"interface":"front","average":{"1min":17.428999999999998,"5min":34.850999999999999,"15min":54.871000000000002},"min":{"1min":7.5869999999999997,"5min":7.0410000000000004,"15min":7.0410000000000004},"max":{"1min":37.744,"5min":658.48599999999999,"15min":658.48599999999999},"last":26.841000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":14.976000000000001,"5min":24.675999999999998,"15min":34.963999999999999},"min":{"1min":8.5229999999999997,"5min":8.4710000000000001,"15min":8.4710000000000001},"max":{"1min":54.383000000000003,"5min":439.56400000000002,"15min":439.56400000000002},"last":30.204000000000001},{"interface":"front","average":{"1min":20.981999999999999,"5min":27.957000000000001,"15min":36.582999999999998},"min":{"1min":13.901999999999999,"5min":10.641999999999999,"15min":10.641999999999999},"max":{"1min":55.423999999999999,"5min":386.71499999999997,"15min":386.71499999999997},"last":25.501999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":18.789000000000001,"5min":18.789000000000001,"15min":18.789000000000001},"min":{"1min":11.512,"5min":11.512,"15min":11.512},"max":{"1min":97.561999999999998,"5min":97.561999999999998,"15min":97.561999999999998},"last":28.106000000000002},{"interface":"front","average":{"1min":17.646000000000001,"5min":17.646000000000001,"15min":17.646000000000001},"min":{"1min":10.603,"5min":10.603,"15min":10.603},"max":{"1min":45.402999999999999,"5min":45.402999999999999,"15min":45.402999999999999},"last":33.448}]}]},{"osd":0,"up_from":16,"seq":68719476795,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":634880,"data_stored":271778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 6 18:31:03 2023","interfaces":[{"interface":"back","average":{"1min":11.746,"5min":13.705,"15min":16.933},"min":{"1min":6.8760000000000003,"5min":3.2360000000000002,"15min":3.2360000000000002},"max":{"1min":25.324999999999999,"5min":116.593,"15min":116.593},"last":7.1459999999999999},{"interface":"front","average":{"1min":11.741,"5min":15.111000000000001,"15min":17.638999999999999},"min":{"1min":6.6879999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"max":{"1min":26.614999999999998,"5min":117.896,"15min":117.896},"last":10.715}]},{"osd":2,"last update":"Wed Dec 6 18:31:03 2023","interfaces":[{"interface":"back","average":{"1min":12.739000000000001,"5min":13.983000000000001,"15min":13.587},"min":{"1min":7.9240000000000004,"5min":5.694,"15min":5.694},"max":{"1min":17.562999999999999,"5min":43.752000000000002,"15min":43.752000000000002},"last":7.7939999999999996},{"interface":"front","average":{"1min":10.945,"5min":12.004,"15min":11.316000000000001},"min":{"1min":6.6210000000000004,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":17.666,"5min":46.387999999999998,"15min":46.387999999999998},"last":16.268999999999998}]},{"osd":3,"last update":"Wed Dec 6 18:31:08 2023","interfaces":[{"interface":"back","average":{"1min":14.369999999999999,"5min":14.734,"15min":16.140999999999998},"min":{"1min":9.2289999999999992,"5min":6.1379999999999999,"15min":6.1379999999999999},"max":{"1min":20.352,"5min":46.689999999999998,"15min":46.689999999999998},"last":28.835999999999999},{"interface":"front","average":{"1min":14.148,"5min":15.906000000000001,"15min":16.155000000000001},"min":{"1min":11.037000000000001,"5min":8.0329999999999995,"15min":8.0329999999999995},"max":{"1min":20.064,"5min":50.545999999999999,"15min":50.545999999999999},"last":28.042999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:31:03 2023","interfaces":[{"interface":"back","average":{"1min":10.169,"5min":15.519,"15min":19.427},"min":{"1min":6.641,"5min":5.9349999999999996,"15min":5.9349999999999996},"max":{"1min":22.158000000000001,"5min":82.432000000000002,"15min":82.432000000000002},"last":13.720000000000001},{"interface":"front","average":{"1min":18.053000000000001,"5min":18.855,"15min":20.155000000000001},"min":{"1min":12.497,"5min":6.2629999999999999,"15min":6.2629999999999999},"max":{"1min":23.114000000000001,"5min":81.239999999999995,"15min":81.239999999999995},"last":11.347}]},{"osd":5,"last update":"Wed Dec 6 18:31:03 2023","interfaces":[{"interface":"back","average":{"1min":15.563000000000001,"5min":16.891999999999999,"15min":17.385000000000002},"min":{"1min":8.6479999999999997,"5min":8.0099999999999998,"15min":8.0099999999999998},"max":{"1min":27.327999999999999,"5min":71.367000000000004,"15min":71.367000000000004},"last":8.9459999999999997},{"interface":"front","average":{"1min":14.071999999999999,"5min":16.843,"15min":17.75},"min":{"1min":8.9320000000000004,"5min":8.5299999999999994,"15min":8.5299999999999994},"max":{"1min":29.286000000000001,"5min":72.307000000000002,"15min":72.307000000000002},"last":15.102}]},{"osd":6,"last update":"Wed Dec 6 18:31:03 2023","interfaces":[{"interface":"back","average":{"1min":18.530000000000001,"5min":19.073,"15min":19.029},"min":{"1min":11.763,"5min":7.7560000000000002,"15min":7.7560000000000002},"max":{"1min":23.082999999999998,"5min":62.932000000000002,"15min":62.932000000000002},"last":12.647},{"interface":"front","average":{"1min":15.797000000000001,"5min":17.73,"15min":18.940000000000001},"min":{"1min":11.856999999999999,"5min":9.6940000000000008,"15min":9.6940000000000008},"max":{"1min":24.016999999999999,"5min":52.627000000000002,"15min":52.627000000000002},"last":17.324000000000002}]},{"osd":7,"last update":"Wed Dec 6 18:31:08 2023","interfaces":[{"interface":"back","average":{"1min":19.120999999999999,"5min":19.120999999999999,"15min":19.120999999999999},"min":{"1min":13.513,"5min":13.513,"15min":13.513},"max":{"1min":24.071999999999999,"5min":24.071999999999999,"15min":24.071999999999999},"last":17.963999999999999},{"interface":"front","average":{"1min":17.896999999999998,"5min":17.896999999999998,"15min":17.896999999999998},"min":{"1min":13.669,"5min":13.669,"15min":13.669},"max":{"1min":23.620000000000001,"5min":23.620000000000001,"15min":23.620000000000001},"last":20.167000000000002}]}]},{"osd":5,"up_from":15,"seq":64424509499,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1024000,"data_stored":665267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":8.1989999999999998,"5min":11.412000000000001,"15min":14.845000000000001},"min":{"1min":4.7389999999999999,"5min":3.3660000000000001,"15min":3.3660000000000001},"max":{"1min":15.797000000000001,"5min":208.291,"15min":208.291},"last":4.2110000000000003},{"interface":"front","average":{"1min":10.285,"5min":14.279,"15min":18.748999999999999},"min":{"1min":5.1429999999999998,"5min":4.4299999999999997,"15min":4.4299999999999997},"max":{"1min":15.239000000000001,"5min":199.94900000000001,"15min":199.94900000000001},"last":9.8360000000000003}]},{"osd":1,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":13.602,"5min":34.588000000000001,"15min":55.927},"min":{"1min":6.9450000000000003,"5min":6.9450000000000003,"15min":6.9450000000000003},"max":{"1min":19.356999999999999,"5min":1211.6010000000001,"15min":1211.6010000000001},"last":10.568},{"interface":"front","average":{"1min":8.9770000000000003,"5min":30.417999999999999,"15min":51.085000000000001},"min":{"1min":4.7199999999999998,"5min":4.7199999999999998,"15min":4.7199999999999998},"max":{"1min":14.074999999999999,"5min":1189.3869999999999,"15min":1189.3869999999999},"last":8.9480000000000004}]},{"osd":2,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":13.939,"5min":18.715,"15min":23.940000000000001},"min":{"1min":9.0090000000000003,"5min":6.5129999999999999,"15min":6.5129999999999999},"max":{"1min":19.02,"5min":283.46800000000002,"15min":283.46800000000002},"last":11.183999999999999},{"interface":"front","average":{"1min":16.815999999999999,"5min":22.056999999999999,"15min":27.766999999999999},"min":{"1min":10.054,"5min":8.9350000000000005,"15min":8.9350000000000005},"max":{"1min":22.969999999999999,"5min":296.47800000000001,"15min":296.47800000000001},"last":15.237}]},{"osd":3,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":17.452999999999999,"5min":17.550000000000001,"15min":18.931000000000001},"min":{"1min":12.442,"5min":9.1739999999999995,"15min":9.1739999999999995},"max":{"1min":23.66,"5min":47.926000000000002,"15min":47.926000000000002},"last":8.1500000000000004},{"interface":"front","average":{"1min":12.148,"5min":13.238,"15min":14.881},"min":{"1min":7.5810000000000004,"5min":5.3979999999999997,"15min":5.3979999999999997},"max":{"1min":17.27,"5min":66.176000000000002,"15min":66.176000000000002},"last":18.294}]},{"osd":4,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":18.606000000000002,"5min":18.021999999999998,"15min":19.501999999999999},"min":{"1min":12.250999999999999,"5min":10.286,"15min":10.286},"max":{"1min":29.350999999999999,"5min":67.248000000000005,"15min":67.248000000000005},"last":11.747},{"interface":"front","average":{"1min":10.641,"5min":11.377000000000001,"15min":12.823},"min":{"1min":6.1020000000000003,"5min":5.3300000000000001,"15min":5.3300000000000001},"max":{"1min":28.155999999999999,"5min":77.266999999999996,"15min":77.266999999999996},"last":13.923}]},{"osd":6,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":12.084,"5min":13.314,"15min":15.378},"min":{"1min":7.8390000000000004,"5min":5.915,"15min":5.915},"max":{"1min":17.699000000000002,"5min":76.751999999999995,"15min":76.751999999999995},"last":14.597},{"interface":"front","average":{"1min":19.111999999999998,"5min":19.358000000000001,"15min":20.786999999999999},"min":{"1min":15.135999999999999,"5min":8.8390000000000004,"15min":8.8390000000000004},"max":{"1min":24.015000000000001,"5min":57.101999999999997,"15min":57.101999999999997},"last":12.337}]},{"osd":7,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":14.211,"5min":14.211,"15min":14.211},"min":{"1min":7.5519999999999996,"5min":7.5519999999999996,"15min":7.5519999999999996},"max":{"1min":21.047000000000001,"5min":21.047000000000001,"15min":21.047000000000001},"last":16.643999999999998},{"interface":"front","average":{"1min":11.888,"5min":11.888,"15min":11.888},"min":{"1min":8.2829999999999995,"5min":8.2829999999999995,"15min":8.2829999999999995},"max":{"1min":15.411,"5min":15.411,"15min":15.411},"last":15.923999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509500,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":618496,"data_stored":267146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":10.843,"5min":11.773,"15min":13.387},"min":{"1min":5.3300000000000001,"5min":3.7429999999999999,"15min":3.7429999999999999},"max":{"1min":35.863,"5min":53.298999999999999,"15min":53.298999999999999},"last":5.2539999999999996},{"interface":"front","average":{"1min":10.867000000000001,"5min":10.327999999999999,"15min":10.82},"min":{"1min":5.9139999999999997,"5min":3.9670000000000001,"15min":3.9670000000000001},"max":{"1min":35.146000000000001,"5min":52.567,"15min":52.567},"last":9.1170000000000009}]},{"osd":1,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":11.565,"5min":12.291,"15min":13.744999999999999},"min":{"1min":5.8390000000000004,"5min":5.835,"15min":5.835},"max":{"1min":20.745000000000001,"5min":49.183,"15min":49.183},"last":9.6449999999999996},{"interface":"front","average":{"1min":11.513,"5min":13.619,"15min":15.367000000000001},"min":{"1min":7.4690000000000003,"5min":5.96,"15min":5.96},"max":{"1min":19.991,"5min":45.412999999999997,"15min":45.412999999999997},"last":11.625}]},{"osd":2,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":10.635999999999999,"5min":16.286000000000001,"15min":21.515999999999998},"min":{"1min":7.3319999999999999,"5min":5.6390000000000002,"15min":5.6390000000000002},"max":{"1min":14.907999999999999,"5min":85.394000000000005,"15min":85.394000000000005},"last":11.025},{"interface":"front","average":{"1min":14.616,"5min":16.981999999999999,"15min":20.102},"min":{"1min":8.1720000000000006,"5min":6.3639999999999999,"15min":6.3639999999999999},"max":{"1min":19.039000000000001,"5min":69.530000000000001,"15min":69.530000000000001},"last":14.75}]},{"osd":3,"last update":"Wed Dec 6 18:31:07 2023","interfaces":[{"interface":"back","average":{"1min":15.746,"5min":14.6,"15min":14.467000000000001},"min":{"1min":10.022,"5min":7.008,"15min":7.008},"max":{"1min":20.195,"5min":26.219000000000001,"15min":26.219000000000001},"last":16.053000000000001},{"interface":"front","average":{"1min":11.138999999999999,"5min":13.771000000000001,"15min":14.977},"min":{"1min":6.7370000000000001,"5min":5.5549999999999997,"15min":5.5549999999999997},"max":{"1min":18.253,"5min":29.902999999999999,"15min":29.902999999999999},"last":12.805999999999999}]},{"osd":5,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":13.227,"5min":14.890000000000001,"15min":16.747},"min":{"1min":9.3089999999999993,"5min":6.125,"15min":6.125},"max":{"1min":15.754,"5min":78.798000000000002,"15min":78.798000000000002},"last":53.840000000000003},{"interface":"front","average":{"1min":17.539000000000001,"5min":17.667999999999999,"15min":18.658000000000001},"min":{"1min":12.125999999999999,"5min":6.4450000000000003,"15min":6.4450000000000003},"max":{"1min":23.238,"5min":55.814999999999998,"15min":55.814999999999998},"last":52.781999999999996}]},{"osd":6,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":18.984999999999999,"5min":17.715,"15min":17.992999999999999},"min":{"1min":16.178000000000001,"5min":7.7850000000000001,"15min":7.7850000000000001},"max":{"1min":22.629000000000001,"5min":51.149000000000001,"15min":51.149000000000001},"last":17.422999999999998},{"interface":"front","average":{"1min":13.962999999999999,"5min":15.597,"15min":16.484999999999999},"min":{"1min":9.7680000000000007,"5min":7.8700000000000001,"15min":7.8700000000000001},"max":{"1min":18.120999999999999,"5min":40.426000000000002,"15min":40.426000000000002},"last":13.782}]},{"osd":7,"last update":"Wed Dec 6 18:31:07 2023","interfaces":[{"interface":"back","average":{"1min":18.751000000000001,"5min":18.751000000000001,"15min":18.751000000000001},"min":{"1min":10.289,"5min":10.289,"15min":10.289},"max":{"1min":60.390000000000001,"5min":60.390000000000001,"15min":60.390000000000001},"last":18.123999999999999},{"interface":"front","average":{"1min":22.809000000000001,"5min":22.809000000000001,"15min":22.809000000000001},"min":{"1min":17.552,"5min":17.552,"15min":17.552},"max":{"1min":57.606999999999999,"5min":57.606999999999999,"15min":57.606999999999999},"last":16.864999999999998}]}]},{"osd":7,"up_from":61,"seq":261993005074,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":608,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":622592,"data_stored":267427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4395,"internal_metadata":28634837},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":10.141,"5min":10.141,"15min":10.141},"min":{"1min":4.1059999999999999,"5min":4.1059999999999999,"15min":4.1059999999999999},"max":{"1min":32.523000000000003,"5min":32.523000000000003,"15min":32.523000000000003},"last":7.819},{"interface":"front","average":{"1min":11.015000000000001,"5min":11.015000000000001,"15min":11.015000000000001},"min":{"1min":4.2530000000000001,"5min":4.2530000000000001,"15min":4.2530000000000001},"max":{"1min":33.530999999999999,"5min":33.530999999999999,"15min":33.530999999999999},"last":6.4720000000000004}]},{"osd":1,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":17.460000000000001,"5min":17.460000000000001,"15min":17.460000000000001},"min":{"1min":7.3789999999999996,"5min":7.3789999999999996,"15min":7.3789999999999996},"max":{"1min":64.765000000000001,"5min":64.765000000000001,"15min":64.765000000000001},"last":5.4909999999999997},{"interface":"front","average":{"1min":18.109999999999999,"5min":18.109999999999999,"15min":18.109999999999999},"min":{"1min":7.0739999999999998,"5min":7.0739999999999998,"15min":7.0739999999999998},"max":{"1min":65.994,"5min":65.994,"15min":65.994},"last":10.82}]},{"osd":2,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":18.016999999999999,"5min":18.016999999999999,"15min":18.016999999999999},"min":{"1min":8.0619999999999994,"5min":8.0619999999999994,"15min":8.0619999999999994},"max":{"1min":46.478000000000002,"5min":46.478000000000002,"15min":46.478000000000002},"last":16.513999999999999},{"interface":"front","average":{"1min":14.119,"5min":14.119,"15min":14.119},"min":{"1min":6.7039999999999997,"5min":6.7039999999999997,"15min":6.7039999999999997},"max":{"1min":41.603999999999999,"5min":41.603999999999999,"15min":41.603999999999999},"last":11.172000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":17.262,"5min":17.262,"15min":17.262},"min":{"1min":8.8629999999999995,"5min":8.8629999999999995,"15min":8.8629999999999995},"max":{"1min":60.463999999999999,"5min":60.463999999999999,"15min":60.463999999999999},"last":11.913},{"interface":"front","average":{"1min":20.349,"5min":20.349,"15min":20.349},"min":{"1min":11.082000000000001,"5min":11.082000000000001,"15min":11.082000000000001},"max":{"1min":56.286000000000001,"5min":56.286000000000001,"15min":56.286000000000001},"last":14.664}]},{"osd":4,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":21.329000000000001,"5min":21.329000000000001,"15min":21.329000000000001},"min":{"1min":12.099,"5min":12.099,"15min":12.099},"max":{"1min":57.716999999999999,"5min":57.716999999999999,"15min":57.716999999999999},"last":20.341000000000001},{"interface":"front","average":{"1min":18.623999999999999,"5min":18.623999999999999,"15min":18.623999999999999},"min":{"1min":8.2729999999999997,"5min":8.2729999999999997,"15min":8.2729999999999997},"max":{"1min":67.159000000000006,"5min":67.159000000000006,"15min":67.159000000000006},"last":18.196000000000002}]},{"osd":5,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":14.739000000000001,"5min":14.739000000000001,"15min":14.739000000000001},"min":{"1min":6.3399999999999999,"5min":6.3399999999999999,"15min":6.3399999999999999},"max":{"1min":36.317999999999998,"5min":36.317999999999998,"15min":36.317999999999998},"last":9.0619999999999994},{"interface":"front","average":{"1min":19.93,"5min":19.93,"15min":19.93},"min":{"1min":10.111000000000001,"5min":10.111000000000001,"15min":10.111000000000001},"max":{"1min":70.900000000000006,"5min":70.900000000000006,"15min":70.900000000000006},"last":13.352}]},{"osd":6,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":16.776,"5min":16.776,"15min":16.776},"min":{"1min":8.7170000000000005,"5min":8.7170000000000005,"15min":8.7170000000000005},"max":{"1min":54.393000000000001,"5min":54.393000000000001,"15min":54.393000000000001},"last":12.568},{"interface":"front","average":{"1min":20.928999999999998,"5min":20.928999999999998,"15min":20.928999999999998},"min":{"1min":12.134,"5min":12.134,"15min":12.134},"max":{"1min":69.263000000000005,"5min":69.263000000000005,"15min":69.263000000000005},"last":13.956}]}]},{"osd":6,"up_from":14,"seq":60129542201,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":610304,"data_stored":263076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:31:00 2023","interfaces":[{"interface":"back","average":{"1min":9.8420000000000005,"5min":9.5800000000000001,"15min":9.9199999999999999},"min":{"1min":4.1260000000000003,"5min":3.8519999999999999,"15min":3.8519999999999999},"max":{"1min":22.094999999999999,"5min":46.947000000000003,"15min":46.947000000000003},"last":8.0990000000000002},{"interface":"front","average":{"1min":11.103,"5min":10.573,"15min":10.211},"min":{"1min":4.609,"5min":4.0650000000000004,"15min":4.0650000000000004},"max":{"1min":19.335000000000001,"5min":45.953000000000003,"15min":45.953000000000003},"last":5.0499999999999998}]},{"osd":1,"last update":"Wed Dec 6 18:31:00 2023","interfaces":[{"interface":"back","average":{"1min":9.1880000000000006,"5min":10.928000000000001,"15min":12.289999999999999},"min":{"1min":4.5650000000000004,"5min":4.3760000000000003,"15min":4.3760000000000003},"max":{"1min":15.869,"5min":66.313000000000002,"15min":66.313000000000002},"last":7.077},{"interface":"front","average":{"1min":11.144,"5min":13.48,"15min":14.359999999999999},"min":{"1min":6.0170000000000003,"5min":5.8540000000000001,"15min":5.8540000000000001},"max":{"1min":16.629000000000001,"5min":67.533000000000001,"15min":67.533000000000001},"last":16.042000000000002}]},{"osd":2,"last update":"Wed Dec 6 18:31:00 2023","interfaces":[{"interface":"back","average":{"1min":12.227,"5min":15.805999999999999,"15min":19.670000000000002},"min":{"1min":6.0640000000000001,"5min":6.0640000000000001,"15min":6.0640000000000001},"max":{"1min":16.919,"5min":200.88499999999999,"15min":200.88499999999999},"last":12.917999999999999},{"interface":"front","average":{"1min":16.376000000000001,"5min":16.141999999999999,"15min":16.404},"min":{"1min":7.8129999999999997,"5min":7.4119999999999999,"15min":7.4119999999999999},"max":{"1min":30.908999999999999,"5min":48.005000000000003,"15min":48.005000000000003},"last":15.347}]},{"osd":3,"last update":"Wed Dec 6 18:31:05 2023","interfaces":[{"interface":"back","average":{"1min":14.098000000000001,"5min":13.124000000000001,"15min":13.737},"min":{"1min":7.3250000000000002,"5min":7.298,"15min":7.298},"max":{"1min":36.283999999999999,"5min":36.283999999999999,"15min":36.283999999999999},"last":11.6},{"interface":"front","average":{"1min":16.347000000000001,"5min":15.859,"15min":16.193000000000001},"min":{"1min":11.247999999999999,"5min":7.6079999999999997,"15min":7.6079999999999997},"max":{"1min":34.317,"5min":47.006999999999998,"15min":47.006999999999998},"last":13.525}]},{"osd":4,"last update":"Wed Dec 6 18:31:00 2023","interfaces":[{"interface":"back","average":{"1min":15.241,"5min":14.584,"15min":15.077999999999999},"min":{"1min":6.7999999999999998,"5min":6.1859999999999999,"15min":6.1859999999999999},"max":{"1min":49.170000000000002,"5min":49.170000000000002,"15min":49.170000000000002},"last":9.1690000000000005},{"interface":"front","average":{"1min":17.789000000000001,"5min":16.899000000000001,"15min":16.725000000000001},"min":{"1min":8.9019999999999992,"5min":7.7169999999999996,"15min":7.7169999999999996},"max":{"1min":38.085000000000001,"5min":48.988999999999997,"15min":48.988999999999997},"last":14.137}]},{"osd":5,"last update":"Wed Dec 6 18:31:00 2023","interfaces":[{"interface":"back","average":{"1min":17.669,"5min":18.295999999999999,"15min":18.401},"min":{"1min":10.053000000000001,"5min":7.8879999999999999,"15min":7.8879999999999999},"max":{"1min":25.257999999999999,"5min":57.308,"15min":57.308},"last":14.743},{"interface":"front","average":{"1min":9.8829999999999991,"5min":12.920999999999999,"15min":14.550000000000001},"min":{"1min":6.5199999999999996,"5min":6.0129999999999999,"15min":6.0129999999999999},"max":{"1min":14.032999999999999,"5min":68.463999999999999,"15min":68.463999999999999},"last":10.337}]},{"osd":7,"last update":"Wed Dec 6 18:31:10 2023","interfaces":[{"interface":"back","average":{"1min":15.401,"5min":15.401,"15min":15.401},"min":{"1min":8.5310000000000006,"5min":8.5310000000000006,"15min":8.5310000000000006},"max":{"1min":28.393999999999998,"5min":28.393999999999998,"15min":28.393999999999998},"last":22.363},{"interface":"front","average":{"1min":18.766999999999999,"5min":18.766999999999999,"15min":18.766999999999999},"min":{"1min":13.574,"5min":13.574,"15min":13.574},"max":{"1min":27.132000000000001,"5min":27.132000000000001,"15min":27.132000000000001},"last":17.134}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:31:32.999 INFO:tasks.thrashosds.thrasher:Setting 2.0 to [0, 3, 2, 1] 2023-12-06T18:31:32.999 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.0', '0', '3', '2', '1'] 2023-12-06T18:31:32.999 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.0 0 3 2 1 2023-12-06T18:31:33.042 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:33.042 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:33.051 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:31:33.082 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:33.078+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:33.166 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:12.611 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:33.169 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:33.166+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:33.194 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:33.194 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:33.206 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:31:33.231 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:12.676 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:33.242 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is unset 2023-12-06T18:31:33.248 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:33.245+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:33.350 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:33.346+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:33.377 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:33.377 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:33.386 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:31:33.440 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:31:33.443 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:12.887 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:31:33.485 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:33.481+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:33.513 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:33.513 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:33.521 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:31:33.566 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:33.562+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:33.653 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:33.653 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:33.653 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:33.649+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:33.662 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:31:33.704 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:31:33.704 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:31:33.754 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:33.750+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:33.829 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:31:33.830 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:31:33.831 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:33.831 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:33.840 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:31:33.885 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:33.881+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:33.955 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:33.951+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:33.980 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:31:33.981 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:33.981 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:33.981 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:33.981 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:33.981 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:31:33.992 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:31:34.077 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:34.073+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:34.094 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:31:34.094 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:31:34.161 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:34.157+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:34.175 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:34.175 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:34.176 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:34.176 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:34.176 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:34.185 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:31:34.189 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:31:34.189 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:31:34.256 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:34.253+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:34.277 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:31:34.277 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:31:34.342 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:34.343 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:34.343 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:34.343 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:31:34.343 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:34.352 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:31:34.380 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:31:34.380 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:31:34.380 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:34.376+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:34.458 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:34.455+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:34.488 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:31:34.488 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:31:34.504 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:34.505 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:34.505 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:34.505 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:34.514 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:31:34.559 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:34.555+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:34.660 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:34.656+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:34.692 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:34.693 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:34.693 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:34.693 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:34.693 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:34.703 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:31:34.788 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:34.784+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:34.862 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:34.859+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:34.940 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:34.940 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:34.940 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:34.940 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:34.940 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.940 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:31:34.940 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:34.940 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 279.52942658500001, 2023-12-06T18:31:34.940 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:31:34.940 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:34.941 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:34.941 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:34.941 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:34.941 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:34.941 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:31:34.941 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.941 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:34.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.941 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:34.941 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:34.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.941 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:34.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:34.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:34.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:31:34.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:31:34.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:34.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:31:34.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:31:34.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.943 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:34.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:31:34.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:31:34.943 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.943 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:34.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:31:34.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:31:34.943 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.943 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:34.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:31:34.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:31:34.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.944 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.944 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:31:34.944 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:31:34.944 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:31:34.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.944 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.944 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:34.944 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:31:34.944 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:31:34.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.945 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.945 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:34.945 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:31:34.945 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:31:34.945 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.945 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.945 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:34.945 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:31:34.945 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:31:34.945 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.945 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.945 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:34.945 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:31:34.945 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:31:34.945 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.946 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.946 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:34.946 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:31:34.946 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:31:34.946 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.946 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.946 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:34.946 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:31:34.946 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:31:34.946 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.946 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.946 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:34.946 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:31:34.947 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:31:34.947 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.947 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:34.947 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.947 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.947 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.947 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:34.947 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:34.947 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 277.420853346, 2023-12-06T18:31:34.947 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:31:34.947 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:34.947 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:34.947 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:34.947 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:34.947 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:34.948 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:31:34.948 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.948 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:34.948 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.948 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:34.948 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:34.948 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.948 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.948 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.948 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:34.948 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:34.948 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.948 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.948 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.949 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:34.949 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:31:34.949 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:31:34.949 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.949 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.949 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:34.949 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:31:34.949 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:31:34.949 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.950 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.950 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:34.950 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:31:34.950 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:31:34.950 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.950 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.950 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:34.950 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:31:34.950 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:31:34.950 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.950 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.950 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:34.951 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:31:34.951 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:31:34.951 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.951 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.951 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:34.951 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:31:34.951 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:31:34.951 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.951 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.951 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:34.951 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:31:34.951 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:31:34.951 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.951 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.952 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:34.952 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:31:34.952 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:31:34.952 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.952 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.952 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:34.952 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:31:34.952 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:31:34.952 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.952 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.952 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:34.952 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:31:34.952 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:31:34.952 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.953 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.953 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:34.953 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:31:34.953 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:31:34.953 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.953 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:34.953 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.953 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.953 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.953 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:34.953 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:34.953 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 277.42010811400002, 2023-12-06T18:31:34.953 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:31:34.953 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:34.953 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:34.954 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:34.954 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:34.954 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:34.954 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:31:34.954 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.954 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:34.954 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.954 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:34.954 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:34.954 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.954 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.954 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.954 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:34.954 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:34.955 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.955 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.955 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:34.955 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:31:34.955 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:31:34.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.955 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.955 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:34.955 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:31:34.955 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:31:34.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.955 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.955 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:34.955 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:31:34.955 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:31:34.956 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.956 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.956 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:34.956 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:31:34.956 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:31:34.956 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.956 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.956 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:34.956 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:31:34.956 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:31:34.956 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.956 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.956 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:34.956 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:31:34.956 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:31:34.957 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.957 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.957 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:34.957 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:31:34.957 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:31:34.957 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.957 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.957 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:34.957 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:31:34.957 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:31:34.957 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.957 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.957 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:34.957 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:31:34.957 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:31:34.958 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.958 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.958 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:34.958 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:31:34.958 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:31:34.958 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.958 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.958 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:34.958 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:31:34.958 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:31:34.958 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.958 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:34.958 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.958 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.959 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.959 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:34.959 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:34.959 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 277.419424359, 2023-12-06T18:31:34.959 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:31:34.959 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:34.959 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:34.959 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:34.959 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:34.959 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:34.959 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:31:34.959 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.959 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:34.959 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.960 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:34.960 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:34.960 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.960 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.960 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.960 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:34.960 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:34.960 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.960 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.960 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.960 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:34.960 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:31:34.960 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:31:34.960 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.961 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.961 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:34.961 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:31:34.961 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:31:34.961 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.961 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.961 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:34.961 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:31:34.961 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:31:34.961 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.961 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.961 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:34.961 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:31:34.961 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:31:34.961 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.962 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.962 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:34.962 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:31:34.962 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:31:34.962 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.962 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.962 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:34.962 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:31:34.962 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:31:34.962 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.962 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.962 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:34.962 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:31:34.963 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:31:34.963 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.963 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.963 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:34.963 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:31:34.963 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:31:34.963 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.963 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.963 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:34.963 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:31:34.963 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:31:34.963 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.963 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.963 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:34.964 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:31:34.964 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:31:34.964 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.964 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.964 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:34.964 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:31:34.964 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:31:34.964 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.964 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:34.964 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.964 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.964 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.964 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:34.965 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:34.965 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 277.413851104, 2023-12-06T18:31:34.965 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:31:34.965 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:34.965 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:34.965 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:34.965 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:34.965 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:34.965 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:31:34.965 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.965 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:34.965 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.965 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:34.965 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:34.965 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.966 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.966 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:34.966 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:34.966 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.966 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.966 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:34.966 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:31:34.966 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:31:34.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.966 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.966 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:34.966 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:31:34.967 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:31:34.967 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.967 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.967 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:34.967 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:31:34.967 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:31:34.967 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.967 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.967 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:34.967 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:31:34.967 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:31:34.967 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.967 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.968 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:34.968 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:31:34.968 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:31:34.968 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.968 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.968 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:34.968 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:31:34.968 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:31:34.968 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.968 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.968 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:34.968 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:31:34.968 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:31:34.968 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.969 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.969 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:34.969 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:31:34.969 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:31:34.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.969 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.969 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:34.969 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:31:34.969 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:31:34.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.969 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.969 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:34.969 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:31:34.969 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:31:34.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.969 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.970 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:34.970 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:31:34.970 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:31:34.970 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.970 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:34.970 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.970 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.970 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.970 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:34.970 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:34.970 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 277.413177464, 2023-12-06T18:31:34.970 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:31:34.970 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:34.970 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:34.971 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:34.971 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:34.971 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:34.971 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:31:34.971 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.971 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:34.971 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.971 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:34.971 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:34.971 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.971 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.971 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.971 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:34.971 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:34.971 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.971 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.972 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.972 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:34.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:31:34.972 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:31:34.972 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.972 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.972 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:34.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:31:34.972 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:31:34.972 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.972 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.972 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:34.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:31:34.972 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:31:34.972 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.973 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.973 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:34.973 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:31:34.973 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:31:34.973 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.973 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.973 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:34.973 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:31:34.973 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:31:34.973 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.973 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.973 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:34.973 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:31:34.973 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:31:34.973 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.974 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.974 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:34.974 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:31:34.974 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:31:34.974 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.974 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.974 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:34.974 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:31:34.974 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:31:34.974 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.974 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.974 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:34.974 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:31:34.974 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:31:34.975 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.975 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.975 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:34.975 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:31:34.975 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:31:34.975 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.975 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.975 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:34.975 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:31:34.975 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:31:34.975 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.975 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:34.975 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.975 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.975 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.976 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:34.976 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:34.976 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 277.41251526500002, 2023-12-06T18:31:34.976 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:31:34.976 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:34.976 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:34.976 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:34.976 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:34.976 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:34.976 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:31:34.976 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.976 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:34.976 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.977 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:34.977 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:34.977 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.977 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.977 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:34.977 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:34.977 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.977 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.977 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:34.977 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:31:34.977 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:31:34.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.977 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.978 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:34.978 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:31:34.978 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:31:34.978 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.978 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.978 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:34.978 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:31:34.978 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:31:34.978 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.978 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.978 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:34.978 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:31:34.978 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:31:34.978 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.979 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.979 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:34.979 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:31:34.979 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:31:34.979 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.979 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.979 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:34.979 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:31:34.979 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:31:34.979 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.979 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.979 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:34.979 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:31:34.979 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:31:34.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.980 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:34.980 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:31:34.980 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:31:34.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.980 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:34.980 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:31:34.980 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:31:34.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.980 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:34.980 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:31:34.981 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:31:34.981 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.981 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.981 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:34.981 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:31:34.981 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:31:34.981 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.981 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:34.981 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.981 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.981 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.981 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:34.981 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:34.981 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 277.33980582999999, 2023-12-06T18:31:34.981 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:31:34.982 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:34.982 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:34.982 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:34.982 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:34.982 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:34.982 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:31:34.982 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.982 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:34.982 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.982 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:34.982 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:34.982 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.982 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.983 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.983 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:34.983 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:34.983 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.983 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.983 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.983 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:34.983 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:31:34.983 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:31:34.983 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.983 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.983 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:34.983 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:31:34.983 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:31:34.984 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.984 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.984 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:34.984 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:31:34.984 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:31:34.984 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.984 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.984 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:34.984 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:31:34.984 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:31:34.984 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.984 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.984 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:34.984 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:31:34.984 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:31:34.985 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.985 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.985 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:34.985 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:31:34.985 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:31:34.985 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.985 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.985 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:34.985 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:31:34.985 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:31:34.985 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.985 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.985 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:34.985 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:31:34.985 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:31:34.986 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.986 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.986 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:34.986 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:31:34.986 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:31:34.986 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.986 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.986 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:34.986 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:31:34.986 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:31:34.986 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.986 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.986 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:34.986 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:31:34.987 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:31:34.987 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.987 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:34.987 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.987 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.987 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.987 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:34.987 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:34.987 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 277.339124121, 2023-12-06T18:31:34.987 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:31:34.987 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:34.987 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:34.987 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:34.987 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:34.987 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:34.988 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:31:34.988 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.988 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:34.988 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.988 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:34.988 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:34.988 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.988 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.988 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.988 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:34.988 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:34.988 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.988 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.988 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.988 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:34.989 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:31:34.989 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:31:34.989 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.989 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.989 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:34.989 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:31:34.989 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:31:34.989 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.989 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.989 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:34.989 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:31:34.989 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:31:34.989 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.989 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.989 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:34.990 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:31:34.990 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:31:34.990 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.990 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.990 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:34.990 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:31:34.990 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:31:34.990 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.990 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.990 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:34.990 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:31:34.990 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:31:34.990 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.990 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.991 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:34.991 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:31:34.991 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:31:34.991 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.991 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.991 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:34.991 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:31:34.991 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:31:34.991 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.991 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.991 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:34.991 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:31:34.991 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:31:34.991 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.991 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.991 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:34.992 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:31:34.992 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:31:34.992 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.992 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.992 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:34.992 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:31:34.992 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:31:34.992 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.992 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:34.992 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.992 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.992 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.992 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:34.992 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:34.992 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 276.26032003300003, 2023-12-06T18:31:34.992 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:31:34.993 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:34.993 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:34.993 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:34.993 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:34.993 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:34.993 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:31:34.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.993 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:34.993 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.993 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:34.993 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:34.993 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.993 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.993 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:34.993 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:34.994 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.994 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.994 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.994 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:34.994 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:31:34.994 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:31:34.994 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.994 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.994 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:34.994 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:31:34.994 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:31:34.994 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.994 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.994 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:34.995 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:31:34.995 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:31:34.995 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.995 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.995 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:34.995 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:31:34.995 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:31:34.995 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.995 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.995 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:34.995 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:31:34.995 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:31:34.995 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.995 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.995 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:34.996 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:31:34.996 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:31:34.996 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.996 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.996 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:34.996 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:31:34.996 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:31:34.996 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.996 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.996 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:34.996 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:31:34.996 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:31:34.996 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.996 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.996 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:34.996 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:31:34.997 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:31:34.997 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.997 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.997 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:34.997 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:31:34.997 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:31:34.997 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.997 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.997 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:34.997 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:31:34.997 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:31:34.997 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.997 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:34.997 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:34.997 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.997 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.998 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:34.998 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:34.998 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 276.25890345900001, 2023-12-06T18:31:34.998 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:31:34.998 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:34.998 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:34.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:34.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:34.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:34.998 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:31:34.998 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.998 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:34.998 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.999 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:34.999 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:34.999 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.999 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.999 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.999 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:34.999 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:34.999 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:34.999 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.999 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.999 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:34.999 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:31:34.999 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:31:34.999 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:34.999 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:34.999 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:34.999 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:31:35.000 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:31:35.000 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.000 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.000 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:35.000 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:31:35.000 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:31:35.000 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.000 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.000 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:35.000 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:31:35.000 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:31:35.000 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.000 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.000 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:35.000 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:31:35.001 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:31:35.001 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.001 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.001 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:35.001 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:31:35.001 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:31:35.001 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.001 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.001 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:35.001 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:31:35.001 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:31:35.001 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.001 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.001 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:35.001 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:31:35.001 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:31:35.002 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.002 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.002 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:35.002 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:31:35.002 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:31:35.002 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.002 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.002 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:35.002 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:31:35.002 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:31:35.002 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.002 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.002 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:35.002 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:31:35.002 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:31:35.003 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.003 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:35.003 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.003 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.003 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.003 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:35.003 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:35.003 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 276.25781833399998, 2023-12-06T18:31:35.003 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:31:35.003 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:35.003 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:35.003 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:35.003 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:35.003 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:35.003 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:31:35.004 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.004 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:35.004 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.004 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:35.004 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:35.004 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.004 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.004 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.004 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:35.004 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:35.004 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.004 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.004 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.004 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:35.004 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:31:35.005 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:31:35.005 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.005 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.005 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:35.005 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:31:35.005 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:31:35.005 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.005 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.005 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:35.005 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:31:35.005 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:31:35.005 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.005 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.005 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:35.005 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:31:35.006 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:31:35.006 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.006 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.006 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:35.006 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:31:35.006 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:31:35.006 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.006 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.006 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:35.006 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:31:35.006 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:31:35.006 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.006 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.006 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:35.007 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:31:35.007 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:31:35.007 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.007 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.007 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:35.007 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:31:35.007 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:31:35.007 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.007 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.007 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:35.007 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:31:35.007 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:31:35.007 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.007 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.007 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:35.007 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:31:35.008 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:31:35.008 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.008 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.008 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:35.008 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:31:35.008 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:31:35.008 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.008 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:35.008 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.008 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.008 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.008 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:35.008 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:35.008 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 276.22266923199999, 2023-12-06T18:31:35.008 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:31:35.009 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:35.009 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:35.009 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:35.009 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:35.009 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:35.009 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:31:35.009 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.009 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:35.009 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.009 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:35.009 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:35.009 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.009 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.009 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.010 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:35.010 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:35.010 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.010 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.010 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.010 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:35.010 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:31:35.010 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:31:35.010 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.010 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.010 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:35.010 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:31:35.010 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:31:35.010 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.011 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.011 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:35.011 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:31:35.011 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:31:35.011 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.011 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.011 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:35.011 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:31:35.011 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:31:35.011 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.011 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.011 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:35.011 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:31:35.011 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:31:35.011 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.012 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.012 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:35.012 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:31:35.012 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:31:35.012 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.012 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.012 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:35.012 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:31:35.012 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:31:35.012 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.012 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.012 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:35.012 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:31:35.012 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:31:35.012 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.012 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.013 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:35.013 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:31:35.013 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:31:35.013 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.013 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.013 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:35.013 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:31:35.013 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:31:35.013 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.013 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.013 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:35.013 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:31:35.014 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:31:35.014 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.014 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:35.014 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.014 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.014 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.014 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:35.014 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:35.014 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 269.67293870700001, 2023-12-06T18:31:35.014 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:31:35.014 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:35.014 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:35.014 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:35.014 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:35.015 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:35.015 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:31:35.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.015 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:35.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:35.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:35.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:35.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:35.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:35.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:31:35.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:31:35.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:35.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:31:35.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:31:35.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:35.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:31:35.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:31:35.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:35.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:31:35.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:31:35.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:35.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:31:35.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:31:35.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:35.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:31:35.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:31:35.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:35.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:31:35.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:31:35.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:35.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:31:35.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:31:35.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:35.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:31:35.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:31:35.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:35.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:31:35.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:31:35.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:35.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:31:35.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:31:35.020 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.020 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:35.020 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.021 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.021 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.021 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:35.021 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:35.021 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 269.67213309900001, 2023-12-06T18:31:35.021 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:31:35.021 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:35.021 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:35.021 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:35.021 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:35.021 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:35.021 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:31:35.021 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.021 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:35.021 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.021 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:35.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:35.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:35.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:35.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:35.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:31:35.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:31:35.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:35.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:31:35.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:31:35.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:35.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:31:35.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:31:35.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:35.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:31:35.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:31:35.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:35.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:31:35.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:31:35.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:35.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:31:35.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:31:35.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:35.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:31:35.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:31:35.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:35.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:31:35.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:31:35.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:35.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:31:35.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:31:35.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:35.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:31:35.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:31:35.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:35.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:31:35.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:31:35.026 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.026 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:35.026 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.026 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.026 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.026 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:35.026 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:35.026 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 269.67144229000002, 2023-12-06T18:31:35.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:31:35.026 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:35.026 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:35.026 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:35.026 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:35.026 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:35.027 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:31:35.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.027 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:35.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.027 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:35.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:35.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.027 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:35.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:35.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.027 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:35.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:31:35.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:31:35.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:35.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:31:35.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:31:35.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:35.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:31:35.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:31:35.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:35.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:31:35.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:31:35.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:35.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:31:35.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:31:35.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:35.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:31:35.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:31:35.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:35.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:31:35.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:31:35.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:35.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:31:35.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:31:35.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:35.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:31:35.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:31:35.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:35.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:31:35.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:31:35.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:35.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:31:35.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:31:35.031 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.031 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:35.031 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.031 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:35.031 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:35.032 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 269.66714478199998, 2023-12-06T18:31:35.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:31:35.032 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:35.032 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:35.032 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:35.032 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:35.032 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:35.032 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:31:35.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.032 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:35.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:35.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:35.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:35.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:35.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:35.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:31:35.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:31:35.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:35.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:31:35.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:31:35.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:35.034 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:31:35.034 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:31:35.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.034 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:35.034 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:31:35.034 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:31:35.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.034 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:35.034 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:31:35.034 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:31:35.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:35.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:31:35.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:31:35.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:35.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:31:35.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:31:35.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:35.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:31:35.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:31:35.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:35.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:31:35.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:31:35.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:35.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:31:35.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:31:35.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:35.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:31:35.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:31:35.036 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.036 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:35.037 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.037 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:35.037 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:35.037 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 269.66647710699999, 2023-12-06T18:31:35.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:31:35.037 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:35.037 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:35.037 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:35.037 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:35.037 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:35.037 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:31:35.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.037 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:35.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.037 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:35.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:35.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:35.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:35.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:35.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:31:35.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:31:35.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:35.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:31:35.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:31:35.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:35.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:31:35.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:31:35.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:35.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:31:35.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:31:35.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:35.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:31:35.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:31:35.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:35.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:31:35.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:31:35.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:35.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:31:35.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:31:35.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:35.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:31:35.041 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:31:35.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.041 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:35.041 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:31:35.041 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:31:35.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.041 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:35.041 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:31:35.041 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:31:35.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.041 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:35.041 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:31:35.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:31:35.042 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.042 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:35.042 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.042 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.042 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.042 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:35.042 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:35.042 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 269.66582167899998, 2023-12-06T18:31:35.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:31:35.042 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:35.042 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:35.042 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:35.042 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:35.042 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:35.043 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:31:35.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.043 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:35.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:35.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:35.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:35.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:35.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:35.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:31:35.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:31:35.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:35.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:31:35.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:31:35.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:35.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:31:35.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:31:35.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:35.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:31:35.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:31:35.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:35.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:31:35.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:31:35.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:35.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:31:35.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:31:35.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.046 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:35.046 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:31:35.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:31:35.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.046 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:35.046 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:31:35.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:31:35.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.046 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:35.046 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:31:35.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:31:35.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:35.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:31:35.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:31:35.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:35.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:31:35.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:31:35.047 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.047 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:35.047 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.048 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:35.048 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:35.048 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 269.625167132, 2023-12-06T18:31:35.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:31:35.048 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:35.048 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:35.048 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:35.048 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:35.048 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:35.048 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:31:35.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.048 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:35.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:35.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:35.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:35.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:35.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:35.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:31:35.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:31:35.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:35.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:31:35.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:31:35.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:35.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:31:35.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:31:35.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:35.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:31:35.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:31:35.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:35.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:31:35.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:31:35.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:35.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:31:35.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:31:35.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:35.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:31:35.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:31:35.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:35.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:31:35.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:31:35.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:35.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:31:35.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:31:35.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:35.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:31:35.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:31:35.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:35.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:31:35.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:31:35.053 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.053 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:35.053 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.053 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.053 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:35.053 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:35.054 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:31:35.055 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:35.051+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:35.155 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:35.152+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:35.251 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:35.252 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:35.252 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:35.252 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:35.256 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:35.253+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:35.258 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-12-06T18:31:35.261 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:31:35.357 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:35.354+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:35.437 INFO:teuthology.orchestra.run.smithi061.stderr:set 2.0 pg_upmap_items mapping to [0->3,2->1] 2023-12-06T18:31:35.460 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:35.460 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:35.460 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:35.460 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:35.460 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:35.461 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:35.457+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:35.469 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:31:35.559 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:35.556+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:35.649 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:35.649 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:35.649 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:35.649 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:35.649 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.649 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:31:35.649 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:35.649 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 253.833147403, 2023-12-06T18:31:35.649 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:31:35.650 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:35.650 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:31:35.650 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:35.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.650 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:35.650 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:35.650 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.650 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.650 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.650 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:35.651 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:35.651 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:35.651 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.651 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.651 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:35.651 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:31:35.651 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:31:35.651 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.651 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.651 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:35.651 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:31:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:31:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:31:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:31:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:31:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:31:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.652 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:35.653 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:31:35.653 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:31:35.653 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.653 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.653 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:35.653 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:31:35.653 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:31:35.653 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:35.653 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:35.653 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:35.653 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:31:35.653 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:31:35.653 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.654 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:35.654 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.654 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:35.654 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:35.654 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:35.659 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:31:35.660 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:35.657+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:35.762 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:35.758+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:35.843 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:35.843 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:35.843 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:35.843 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:35.852 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:31:35.867 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:31:35.897 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:35.893+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:35.963 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:35.960+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:35.992 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:35.992 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:36.001 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:31:36.064 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:36.061+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:36.165 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:36.162+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:36.190 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:36.190 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 135.09067505499999, 2023-12-06T18:31:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:31:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:31:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:36.191 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:31:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:31:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:31:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:31:36.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:31:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:31:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:31:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:31:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:31:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:31:36.193 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:31:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:31:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:31:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 75.072417752999996, 2023-12-06T18:31:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:31:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:36.194 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:31:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:36.195 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:31:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:31:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:31:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:31:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:36.196 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:31:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:31:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:31:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:31:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:31:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:31:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:36.197 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:31:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:31:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:36.198 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:36.198 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:36.200 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:31:36.267 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:36.263+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:36.337 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:36.337 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:36.337 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:36.337 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:36.347 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:31:36.367 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:36.364+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:36.438 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is set 2023-12-06T18:31:36.468 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:36.465+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:36.505 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:31:36.535 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:36.535 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:36.536 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:36.536 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:36.536 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:36.548 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:31:36.569 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:36.566+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:36.603 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:31:36.618 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2023-12-06T18:31:36.618 INFO:tasks.mon_recovery:removing mon b 2023-12-06T18:31:36.618 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-12-06T18:31:36.618 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:31:36.652 INFO:tasks.ceph.mon.b.smithi152.stderr:2023-12-06T18:31:36.636+0000 9f54640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 106174) UID: 0 2023-12-06T18:31:36.652 INFO:tasks.ceph.mon.b.smithi152.stderr:2023-12-06T18:31:36.638+0000 9f54640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2023-12-06T18:31:36.670 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:36.666+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:36.741 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:36.741 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:36.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:36.741 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:36.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:36.741 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:36.741 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:36.741 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 195.247635567, 2023-12-06T18:31:36.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:31:36.741 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:31:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:36.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:31:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:31:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:31:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:31:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:36.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:31:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:31:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:31:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:31:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:31:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:31:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:36.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:31:36.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:31:36.745 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:36.745 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:36.745 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:36.745 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:36.745 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:36.745 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:36.750 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:31:36.793 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:36.788+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:36.871 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:36.868+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:36.924 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:36.924 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:36.924 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:36.924 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:36.933 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:31:36.975 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:36.971+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:37.081 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:37.078+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:37.148 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:37.148 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:37.148 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:37.148 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:37.148 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:37.159 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:31:37.174 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:37.170+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:37.274 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:37.271+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:37.373 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:37.373 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:37.373 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:37.373 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:37.373 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.373 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:31:37.373 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:37.374 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 280.754300048, 2023-12-06T18:31:37.374 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:31:37.374 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.374 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.374 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.374 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.374 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:37.374 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.374 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.374 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.374 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.374 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:37.374 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:31:37.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:31:37.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:31:37.375 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:31:37.375 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.375 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.375 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.375 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:31:37.376 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:31:37.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.376 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.376 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:31:37.376 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:31:37.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.376 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.376 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:31:37.376 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:31:37.376 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.376 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.376 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.376 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:31:37.377 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:31:37.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.377 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.377 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.377 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:31:37.377 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:31:37.377 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.377 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.377 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.377 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:31:37.377 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:31:37.377 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.377 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.377 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.378 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:31:37.378 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:37.378 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 279.73147303899998, 2023-12-06T18:31:37.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:31:37.378 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.378 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.378 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.378 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.378 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:37.378 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.378 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.378 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.379 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.379 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:37.379 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.379 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.379 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.379 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:31:37.379 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:31:37.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.379 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.379 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.379 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:31:37.379 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:31:37.379 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.379 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:31:37.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:31:37.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.380 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:31:37.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:31:37.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.380 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.380 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.380 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:31:37.380 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:31:37.380 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.381 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.381 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.381 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:31:37.381 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:31:37.381 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.381 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.381 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.381 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:31:37.381 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:31:37.381 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.381 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.381 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.381 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:31:37.381 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:31:37.381 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.382 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.382 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.382 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.382 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.382 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:31:37.382 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:37.382 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 279.71942640100002, 2023-12-06T18:31:37.382 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:31:37.382 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.382 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.382 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.382 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.382 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.382 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:37.383 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.383 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.383 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:37.383 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.383 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.383 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:31:37.383 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:31:37.383 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.383 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.383 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.383 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:31:37.384 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:31:37.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.384 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.384 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.384 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:31:37.384 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:31:37.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.384 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.384 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.384 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:31:37.384 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:31:37.384 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.384 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.384 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.384 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:31:37.385 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:31:37.385 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.385 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.385 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.385 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:31:37.385 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:31:37.385 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.385 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.385 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.385 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:31:37.385 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:31:37.385 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.385 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.385 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.385 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:31:37.386 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:31:37.386 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.386 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.386 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.386 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.386 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.386 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:31:37.386 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:37.386 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 279.69802579399999, 2023-12-06T18:31:37.386 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:31:37.386 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.386 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.386 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.386 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.387 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.387 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:37.387 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.387 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.387 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.387 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.387 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:37.387 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.387 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.387 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.387 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.387 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:31:37.387 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:31:37.387 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.388 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.388 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.388 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:31:37.388 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:31:37.388 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.388 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.388 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.388 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:31:37.388 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:31:37.388 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.388 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.388 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.388 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:31:37.388 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:31:37.388 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.389 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.389 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.389 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:31:37.389 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:31:37.389 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.389 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.389 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.389 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:31:37.389 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:31:37.389 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.389 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.389 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.389 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:31:37.389 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:31:37.389 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.390 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.390 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.390 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:31:37.390 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:31:37.390 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.390 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.390 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.390 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.390 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.390 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:31:37.390 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:37.390 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 279.69684647100001, 2023-12-06T18:31:37.390 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:31:37.390 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.391 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.391 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.391 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.391 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.391 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:37.391 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.391 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.391 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.391 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.391 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:37.391 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.391 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.391 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.391 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.391 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:31:37.392 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:31:37.392 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.392 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.392 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.392 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:31:37.392 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:31:37.392 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.392 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.392 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.392 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:31:37.392 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:31:37.392 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.392 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.392 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:31:37.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:31:37.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:31:37.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:31:37.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.393 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:31:37.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:31:37.393 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.393 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.393 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.394 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:31:37.394 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:31:37.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.394 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.394 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.394 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:31:37.394 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:31:37.394 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.394 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.394 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.394 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.394 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:31:37.395 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:37.395 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 279.69627449000001, 2023-12-06T18:31:37.395 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:31:37.395 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.395 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.395 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.395 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.395 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.395 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:37.395 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.395 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.395 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.395 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.395 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:37.395 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.396 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.396 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.396 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:31:37.396 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:31:37.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.396 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.396 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.396 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:31:37.396 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:31:37.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.396 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.396 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.396 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:31:37.397 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:31:37.397 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.397 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.397 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:31:37.397 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:31:37.397 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.397 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.397 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:31:37.397 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:31:37.397 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.397 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.397 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:31:37.398 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:31:37.398 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.398 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.398 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.398 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:31:37.398 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:31:37.398 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.398 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.398 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.398 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:31:37.398 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:31:37.398 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.398 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.399 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.399 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:31:37.399 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:37.399 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 279.25043901999999, 2023-12-06T18:31:37.399 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:31:37.399 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.399 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.399 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.399 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.399 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:37.399 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.399 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:37.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:31:37.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:31:37.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:31:37.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:31:37.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:31:37.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:31:37.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:31:37.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:31:37.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:31:37.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:31:37.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.402 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.402 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:31:37.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:31:37.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.402 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.402 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:31:37.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:31:37.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.402 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.402 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:31:37.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:31:37.403 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.403 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.403 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.403 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:31:37.403 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:37.403 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 279.248839164, 2023-12-06T18:31:37.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:31:37.403 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.403 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.403 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.403 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.404 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:37.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.404 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.404 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:37.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.404 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.404 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:31:37.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:31:37.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.404 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:31:37.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:31:37.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:31:37.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:31:37.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:31:37.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:31:37.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:31:37.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:31:37.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:31:37.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:31:37.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:31:37.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:31:37.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.407 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:31:37.407 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:31:37.407 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.407 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.407 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.407 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.407 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.407 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:31:37.407 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:37.407 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 278.86963354800002, 2023-12-06T18:31:37.407 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:31:37.407 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.407 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.407 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.408 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.408 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.408 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:37.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.408 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.408 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.408 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:37.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.408 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.408 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.408 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:31:37.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:31:37.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:31:37.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:31:37.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:31:37.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:31:37.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:31:37.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:31:37.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:31:37.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:31:37.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:31:37.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:31:37.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:31:37.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:31:37.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:31:37.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:31:37.411 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.411 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.411 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.411 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:31:37.411 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:37.411 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 278.63213136600001, 2023-12-06T18:31:37.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:31:37.411 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.412 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.412 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.412 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.412 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:37.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.412 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.412 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.412 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:37.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.412 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.412 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.412 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:31:37.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:31:37.413 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.413 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:31:37.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:31:37.413 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.413 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:31:37.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:31:37.413 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.413 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:31:37.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:31:37.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:31:37.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:31:37.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:31:37.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:31:37.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:31:37.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:31:37.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:31:37.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:31:37.415 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.415 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.415 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.415 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:31:37.415 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:37.415 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 278.63140336700002, 2023-12-06T18:31:37.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:31:37.416 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.416 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.416 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:37.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:37.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.417 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:31:37.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:31:37.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.417 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:31:37.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:31:37.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.417 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:31:37.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:31:37.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.417 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:31:37.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:31:37.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:31:37.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:31:37.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:31:37.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:31:37.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:31:37.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:31:37.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:31:37.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:31:37.419 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.419 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.419 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.419 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:31:37.419 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:37.419 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 278.62639095700001, 2023-12-06T18:31:37.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:31:37.420 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.420 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.420 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:37.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.420 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:37.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:31:37.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:31:37.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:31:37.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:31:37.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:31:37.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:31:37.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.422 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.422 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:31:37.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:31:37.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.422 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.422 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:31:37.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:31:37.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.422 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.422 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:31:37.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:31:37.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:31:37.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:31:37.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:31:37.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:31:37.423 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.423 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.423 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.423 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:31:37.423 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:37.423 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 278.62086387800002, 2023-12-06T18:31:37.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:31:37.424 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.424 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.424 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.424 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.424 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:37.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.424 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.424 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.424 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:37.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.424 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:31:37.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:31:37.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:31:37.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:31:37.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:31:37.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:31:37.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.426 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.426 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:31:37.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:31:37.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.426 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.426 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:31:37.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:31:37.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.426 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.426 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:31:37.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:31:37.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.427 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.427 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:31:37.427 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:31:37.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.427 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.427 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:31:37.427 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:31:37.427 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.427 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.427 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.428 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:31:37.428 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:37.428 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 278.62035304099999, 2023-12-06T18:31:37.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:31:37.428 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.428 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.428 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.428 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.428 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.428 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:37.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.428 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.428 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.428 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.428 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:37.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:31:37.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:31:37.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:31:37.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:31:37.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:31:37.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:31:37.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:31:37.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:31:37.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:31:37.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:31:37.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:31:37.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:31:37.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:31:37.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:31:37.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:31:37.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:31:37.431 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.431 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.431 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.432 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:31:37.432 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:37.432 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 278.61989377999998, 2023-12-06T18:31:37.432 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:31:37.432 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.432 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.432 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.432 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.432 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:37.432 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:37.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:31:37.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:31:37.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:31:37.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:31:37.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.434 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.434 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:31:37.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:31:37.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.434 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.434 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:31:37.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:31:37.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.434 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.434 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:31:37.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:31:37.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:31:37.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:31:37.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:31:37.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:31:37.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:31:37.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:31:37.435 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.436 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.436 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.436 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.436 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:31:37.436 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:37.436 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 272.03997797300002, 2023-12-06T18:31:37.436 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:31:37.436 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.436 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.436 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.436 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.436 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:37.436 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:37.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:31:37.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:31:37.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:31:37.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:31:37.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:31:37.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:31:37.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:31:37.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:31:37.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:31:37.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:31:37.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:31:37.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:31:37.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:31:37.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:31:37.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:31:37.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:31:37.440 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.440 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.440 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.440 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:31:37.440 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:37.440 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 272.03374016800001, 2023-12-06T18:31:37.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:31:37.440 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.440 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.440 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:37.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:37.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:31:37.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:31:37.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:31:37.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:31:37.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:31:37.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:31:37.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:31:37.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:31:37.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:31:37.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:31:37.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:31:37.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:31:37.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:31:37.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:31:37.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:31:37.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:31:37.444 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.444 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.444 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.444 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:31:37.444 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:37.444 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 272.02767708900001, 2023-12-06T18:31:37.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:31:37.444 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.444 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.444 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:37.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:37.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:31:37.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:31:37.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:31:37.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:31:37.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:31:37.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:31:37.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:31:37.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:31:37.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:31:37.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:31:37.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:31:37.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:31:37.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:31:37.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:31:37.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:31:37.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:31:37.448 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.448 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.448 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.448 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:31:37.448 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:37.448 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 271.89029000400001, 2023-12-06T18:31:37.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:31:37.448 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.448 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.448 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:37.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:37.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:31:37.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:31:37.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:31:37.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:31:37.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:31:37.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:31:37.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:31:37.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:31:37.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:31:37.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:31:37.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:31:37.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:31:37.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:31:37.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:31:37.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:31:37.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:31:37.452 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.452 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.452 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.452 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:31:37.452 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:37.452 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 271.88361927699998, 2023-12-06T18:31:37.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:31:37.452 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:37.452 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:37.452 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:37.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:37.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:37.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:37.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:37.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:37.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:37.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:31:37.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:31:37.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:37.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:31:37.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:31:37.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:37.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:31:37.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:31:37.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:37.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:31:37.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:31:37.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:37.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:31:37.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:31:37.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:37.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:31:37.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:31:37.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:37.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:31:37.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:31:37.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:37.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:37.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:37.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:31:37.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:31:37.456 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.456 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.456 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.456 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:37.456 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:37.456 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:37.458 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:31:37.459 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:37.133 106176== 2023-12-06T18:31:37.459 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:37.133 106176== Exit program on first error (--exit-on-first-error=yes) 2023-12-06T18:31:37.459 INFO:tasks.ceph.mon.b.smithi152.stderr:daemon-helper: command failed with exit status 42 2023-12-06T18:31:37.465 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:37.458+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:37.558 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:37.555+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:37.593 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:37.594 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:37.594 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:37.594 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:37.603 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:31:37.660 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:37.656+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:37.770 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:37.767+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:37.775 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:37.775 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:37.775 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:37.775 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:37.775 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:37.786 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:31:37.861 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:37.858+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:37.913 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:37.913 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:37.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:37.913 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:31:37.913 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:37.922 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:31:37.962 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:37.958+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:38.050 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:38.050 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:38.050 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:38.051 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:38.059 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:31:38.062 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:38.059+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:38.163 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:38.159+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:38.186 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:38.186 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:38.186 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:38.187 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:38.187 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:38.196 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:31:38.264 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:38.260+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:38.365 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:38.362+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:38.365 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:38.366 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:38.366 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:38.366 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:38.366 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:38.366 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:31:38.366 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:38.366 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 76.703787465999994, 2023-12-06T18:31:38.366 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:31:38.366 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:38.366 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:38.366 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:38.366 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:38.367 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:38.367 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:31:38.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:38.367 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:38.367 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:38.367 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:38.367 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:38.367 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:38.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:38.367 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:38.367 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:38.367 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:38.367 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:38.367 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:38.367 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:38.368 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:38.368 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:31:38.368 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:31:38.368 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:38.368 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:38.368 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:38.368 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:31:38.368 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:31:38.368 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:38.368 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:38.368 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:38.368 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:31:38.368 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:31:38.368 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:38.368 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:38.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:38.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:31:38.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:31:38.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:38.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:38.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:38.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:31:38.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:31:38.369 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:38.369 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:38.369 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:38.369 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:31:38.369 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:31:38.369 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:38.369 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:38.369 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:38.370 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:38.370 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:38.370 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:38.374 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:31:38.452 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-12-06T18:31:38.497 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:38.493+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:38.526 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:38.527 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:38.527 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:38.536 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:31:38.579 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:38.574+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:38.669 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:38.665+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:38.714 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:38.714 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:38.714 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:38.714 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:38.714 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:38.723 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:31:38.770 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:38.766+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:38.860 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:38.860 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:38.860 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:38.860 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:31:38.860 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:38.869 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:31:38.871 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:38.867+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:38.972 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:38.968+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:39.051 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:39.051 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:39.051 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:39.051 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:39.061 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:31:39.072 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:39.069+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:39.174 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:39.170+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:39.245 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:39.245 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:39.245 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:39.245 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:39.245 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:39.255 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:31:39.275 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:39.271+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:39.376 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:39.373+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:39.489 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:39.490 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:39.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:39.490 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:39.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.490 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:31:39.490 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:39.490 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 284.079416337, 2023-12-06T18:31:39.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:31:39.490 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.490 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.490 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.490 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.490 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.491 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:31:39.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.491 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:39.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:39.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:31:39.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:31:39.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:31:39.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:31:39.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:31:39.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:31:39.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:31:39.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:31:39.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:31:39.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:31:39.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:31:39.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:31:39.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:31:39.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:31:39.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:31:39.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:31:39.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:31:39.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:31:39.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:31:39.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:31:39.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:31:39.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:31:39.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:31:39.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:31:39.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:31:39.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:31:39.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:31:39.495 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.495 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.495 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.496 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:39.496 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:39.496 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 281.97084309799999, 2023-12-06T18:31:39.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:31:39.496 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.496 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.496 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.496 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.496 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.496 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:31:39.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.496 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:39.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:39.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:31:39.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:31:39.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:31:39.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:31:39.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:31:39.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:31:39.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:31:39.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:31:39.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:31:39.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:31:39.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:31:39.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:31:39.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:31:39.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:31:39.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:31:39.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:31:39.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:31:39.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:31:39.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:31:39.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:31:39.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.501 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.501 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:31:39.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:31:39.501 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.501 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.501 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.501 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:39.501 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:39.501 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 281.970097866, 2023-12-06T18:31:39.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:31:39.501 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.501 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.501 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.502 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.502 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.502 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:31:39.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.502 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:39.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:39.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:31:39.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:31:39.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:31:39.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:31:39.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:31:39.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:31:39.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:31:39.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:31:39.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:31:39.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:31:39.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.504 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:31:39.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:31:39.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:31:39.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:31:39.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:31:39.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:31:39.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:31:39.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:31:39.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:31:39.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:31:39.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:31:39.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:31:39.506 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.506 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.506 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.506 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:39.507 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:39.507 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 281.96941411099999, 2023-12-06T18:31:39.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:31:39.507 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.507 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.507 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.507 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.507 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.507 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:31:39.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.507 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:39.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:39.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:31:39.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:31:39.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:31:39.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:31:39.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:31:39.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:31:39.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:31:39.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:31:39.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:31:39.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:31:39.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.510 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.510 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:31:39.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:31:39.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.510 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.510 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:31:39.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:31:39.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.510 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.510 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:31:39.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:31:39.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:31:39.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:31:39.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:31:39.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:31:39.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:31:39.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:31:39.511 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.512 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.512 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.512 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:39.512 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:39.512 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 281.96384085599999, 2023-12-06T18:31:39.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:31:39.512 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.512 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.512 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.512 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.512 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.512 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:31:39.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.513 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:39.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:39.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:31:39.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:31:39.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:31:39.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:31:39.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:31:39.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:31:39.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:31:39.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:31:39.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:31:39.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:31:39.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:31:39.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:31:39.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:31:39.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:31:39.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:31:39.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:31:39.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:31:39.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:31:39.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:31:39.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:31:39.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:31:39.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:31:39.517 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.517 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.517 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.517 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:39.517 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:39.517 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 281.96316721599999, 2023-12-06T18:31:39.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:31:39.517 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.518 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.518 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.518 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.518 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.518 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:31:39.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.518 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:39.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:39.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:31:39.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:31:39.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:31:39.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:31:39.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:31:39.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:31:39.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:31:39.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:31:39.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:31:39.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:31:39.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:31:39.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:31:39.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:31:39.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:31:39.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:31:39.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:31:39.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:31:39.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:31:39.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.522 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.522 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:31:39.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:31:39.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.522 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.522 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:31:39.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:31:39.522 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.522 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.522 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.523 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:39.523 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:39.523 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 281.96250501700001, 2023-12-06T18:31:39.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:31:39.523 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.523 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.523 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.523 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.523 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.523 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:31:39.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.523 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:39.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:39.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:31:39.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:31:39.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:31:39.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:31:39.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:31:39.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:31:39.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:31:39.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:31:39.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:31:39.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:31:39.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:31:39.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:31:39.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:31:39.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:31:39.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:31:39.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:31:39.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:31:39.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:31:39.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:31:39.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:31:39.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:31:39.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:31:39.528 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.528 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.528 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.528 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:39.528 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:39.528 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 281.88979558199998, 2023-12-06T18:31:39.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:31:39.528 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.528 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.528 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.528 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.528 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.529 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:31:39.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.529 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:39.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:39.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:31:39.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:31:39.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:31:39.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:31:39.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:31:39.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:31:39.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:31:39.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:31:39.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:31:39.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:31:39.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:31:39.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:31:39.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:31:39.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:31:39.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:31:39.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:31:39.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:31:39.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:31:39.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:31:39.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:31:39.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:31:39.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:31:39.533 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.533 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.533 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.533 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:39.533 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:39.533 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 281.88911387299999, 2023-12-06T18:31:39.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:31:39.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.534 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.534 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.534 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.534 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.534 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:31:39.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.534 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:39.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:39.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:31:39.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:31:39.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:31:39.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:31:39.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:31:39.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:31:39.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:31:39.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:31:39.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:31:39.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:31:39.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:31:39.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:31:39.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:31:39.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:31:39.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:31:39.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:31:39.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.538 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:31:39.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:31:39.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:31:39.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:31:39.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:31:39.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:31:39.539 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.539 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.539 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.539 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:39.539 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:39.539 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 280.81030978500002, 2023-12-06T18:31:39.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:31:39.540 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.540 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.540 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.540 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.540 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.540 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:31:39.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.540 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.541 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:39.485+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:39.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:39.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:39.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:31:39.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:31:39.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:31:39.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:31:39.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:31:39.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:31:39.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:31:39.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:31:39.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:31:39.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:31:39.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:31:39.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:31:39.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:31:39.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:31:39.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:31:39.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:31:39.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.545 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.545 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:31:39.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:31:39.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:31:39.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:31:39.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:31:39.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:31:39.546 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.546 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.546 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.546 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:39.546 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:39.547 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 280.808893211, 2023-12-06T18:31:39.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:31:39.547 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.547 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.547 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.547 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.547 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.547 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:31:39.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.547 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:39.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:39.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:31:39.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:31:39.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:31:39.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:31:39.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:31:39.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:31:39.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:31:39.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:31:39.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:31:39.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:31:39.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:31:39.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:31:39.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:31:39.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:31:39.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:31:39.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:31:39.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:31:39.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:31:39.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:31:39.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:31:39.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:31:39.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:31:39.552 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.552 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.552 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.552 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:39.552 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:39.552 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 280.80780808600002, 2023-12-06T18:31:39.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:31:39.552 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.552 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.552 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.552 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.552 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.552 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:31:39.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.553 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:39.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:39.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:31:39.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:31:39.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:31:39.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:31:39.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:31:39.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:31:39.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:31:39.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:31:39.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:31:39.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:31:39.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:31:39.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:31:39.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:31:39.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:31:39.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.556 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.556 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:31:39.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:31:39.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.556 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.556 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:31:39.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:31:39.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.556 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.556 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:31:39.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:31:39.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:31:39.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:31:39.557 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.557 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.557 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.557 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:39.557 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:39.557 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 280.77265898399997, 2023-12-06T18:31:39.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:31:39.557 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.557 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.558 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.558 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.558 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.558 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:31:39.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.558 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:39.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:39.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:31:39.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:31:39.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:31:39.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:31:39.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:31:39.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:31:39.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:31:39.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:31:39.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:31:39.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:31:39.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:31:39.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:31:39.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:31:39.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:31:39.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:31:39.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:31:39.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:31:39.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:31:39.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:31:39.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:31:39.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:31:39.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:31:39.562 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.562 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.562 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.562 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:39.563 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:39.563 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 274.222928459, 2023-12-06T18:31:39.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:31:39.563 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.563 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.563 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.563 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.563 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.563 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:31:39.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.563 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:39.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:39.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:31:39.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:31:39.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:31:39.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:31:39.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:31:39.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:31:39.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:31:39.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:31:39.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:31:39.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:31:39.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:31:39.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:31:39.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:31:39.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:31:39.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:31:39.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:31:39.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.567 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.567 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:31:39.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:31:39.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.567 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.567 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:31:39.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:31:39.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.567 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.567 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:31:39.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:31:39.568 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.568 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.568 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.568 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:39.568 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:39.568 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 274.22212285099999, 2023-12-06T18:31:39.568 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:31:39.568 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.568 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.568 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.568 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.568 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.568 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:31:39.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.569 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:39.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:39.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:31:39.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:31:39.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.570 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.570 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:31:39.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:31:39.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.570 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.570 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:31:39.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:31:39.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.570 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:31:39.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:31:39.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:31:39.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:31:39.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:31:39.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:31:39.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:31:39.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:31:39.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.572 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:31:39.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:31:39.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.572 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:31:39.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:31:39.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.573 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:31:39.573 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:31:39.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.573 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.573 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:31:39.573 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:31:39.573 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.573 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.573 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.573 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:39.574 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:39.574 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 274.221432042, 2023-12-06T18:31:39.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:31:39.574 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.574 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.574 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.574 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.574 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.574 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:31:39.574 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.574 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:39.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.575 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.575 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:39.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.575 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.575 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:31:39.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:31:39.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.575 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.575 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:31:39.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:31:39.576 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.576 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.576 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.576 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:31:39.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:31:39.576 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.576 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.576 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.576 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:31:39.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:31:39.576 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.576 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.576 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.576 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:31:39.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:31:39.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.577 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.577 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:31:39.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:31:39.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.577 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.577 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:31:39.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:31:39.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.577 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.577 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:31:39.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:31:39.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:31:39.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:31:39.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:31:39.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:31:39.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.579 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:31:39.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:31:39.579 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.579 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.579 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.579 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.579 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:39.579 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:39.579 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 274.21713453400002, 2023-12-06T18:31:39.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:31:39.579 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.579 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.579 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.579 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.580 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.580 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:31:39.580 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.580 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.580 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.580 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:39.580 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.580 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.580 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.580 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:39.580 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.580 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.581 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:31:39.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:31:39.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.581 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.581 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:31:39.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:31:39.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.581 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.581 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:31:39.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:31:39.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.581 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.582 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.582 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:31:39.582 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:31:39.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.582 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.582 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:31:39.582 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:31:39.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.582 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.582 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:31:39.582 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:31:39.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.583 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:31:39.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:31:39.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.583 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:31:39.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:31:39.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.583 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:31:39.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:31:39.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.584 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:31:39.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:31:39.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.584 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:31:39.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:31:39.584 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.584 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.584 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.584 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:39.585 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:39.585 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 274.21646685899998, 2023-12-06T18:31:39.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:31:39.585 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.585 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.585 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.585 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.585 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.585 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:31:39.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.585 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.585 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.585 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:39.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:39.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:31:39.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:31:39.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:31:39.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:31:39.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:31:39.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:31:39.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:31:39.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:31:39.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.588 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:31:39.588 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:31:39.588 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.588 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.588 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.588 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:31:39.588 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:31:39.588 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.588 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.588 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.588 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:31:39.588 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:31:39.588 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.588 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.588 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.588 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:31:39.589 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:31:39.589 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.589 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.589 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:31:39.589 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:31:39.589 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.589 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.589 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:31:39.589 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:31:39.589 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.589 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.589 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:31:39.590 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:31:39.590 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.590 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.590 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.590 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.590 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.590 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:39.590 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:39.590 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 274.21581143100002, 2023-12-06T18:31:39.590 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:31:39.590 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.590 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.590 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.590 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.591 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.591 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:31:39.591 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.591 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.591 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.591 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.591 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:39.591 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.591 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.591 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.591 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.591 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:39.591 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.591 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.591 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.592 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.592 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:31:39.592 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:31:39.592 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.592 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.592 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.592 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:31:39.592 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:31:39.592 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.592 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.592 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.592 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:31:39.592 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:31:39.592 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.592 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.593 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:31:39.593 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:31:39.593 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.593 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.593 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:31:39.593 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:31:39.593 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.593 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.593 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.593 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:31:39.593 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:31:39.593 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.593 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.594 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.594 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:31:39.594 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:31:39.594 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.594 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.594 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.594 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:31:39.594 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:31:39.594 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.594 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.594 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.594 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:31:39.594 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:31:39.594 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.595 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.595 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.595 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:31:39.595 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:31:39.595 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.595 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.595 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.595 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:31:39.595 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:31:39.595 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.595 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.595 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.595 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.595 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.595 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:39.595 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:39.596 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 274.17515688399999, 2023-12-06T18:31:39.596 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:31:39.596 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:39.596 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:39.596 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:39.596 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:39.596 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:39.596 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:31:39.596 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.596 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:39.596 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.596 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:39.596 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:39.596 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.596 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.596 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.597 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:39.597 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:39.597 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:39.597 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.597 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.597 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:39.597 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:31:39.597 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:31:39.597 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.597 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.597 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:39.597 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:31:39.597 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:31:39.597 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.597 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:39.598 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:31:39.598 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:31:39.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:39.598 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:31:39.598 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:31:39.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.598 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:39.598 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:31:39.598 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:31:39.598 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.598 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:39.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:31:39.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:31:39.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.599 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:39.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:31:39.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:31:39.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.599 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:39.599 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:31:39.599 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:31:39.599 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.599 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.599 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:39.600 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:31:39.600 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:31:39.600 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.600 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.600 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:39.600 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:31:39.600 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:31:39.600 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:39.600 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:39.600 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:39.600 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:31:39.600 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:31:39.600 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.600 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.600 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.601 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:39.601 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:39.601 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:39.601 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:31:39.602 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-12-06T18:31:39.603 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:39.599+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:39.725 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:39.721+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:39.739 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:39.740 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:39.740 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:39.740 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:39.748 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:31:39.833 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:39.829+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:39.914 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:39.910+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:39.928 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:39.928 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:39.928 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:39.928 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:39.928 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:39.938 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:31:40.007 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:40.003+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:40.108 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:40.104+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:40.118 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:31:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 258.30215911099998, 2023-12-06T18:31:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:31:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:31:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:31:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:31:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:31:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:31:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:31:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:31:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:31:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:31:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:31:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:31:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:31:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:31:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:31:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:31:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:40.122 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:40.127 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:31:40.209 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:40.206+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:40.266 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:40.267 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:40.267 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:40.267 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:40.275 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:31:40.310 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:40.307+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:40.411 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:40.407+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:40.413 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:40.414 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:40.414 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:40.414 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:40.414 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:40.422 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:31:40.451 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:31:40.451 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:31:40.512 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:40.509+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:40.603 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:40.604 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:40.604 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:40.604 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:40.604 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.604 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:40.604 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:40.604 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 139.504086065, 2023-12-06T18:31:40.604 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:31:40.604 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:40.604 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:40.604 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:40.604 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:40.604 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:40.604 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:31:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:40.605 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:31:40.606 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:31:40.606 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.606 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.606 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:40.606 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:31:40.606 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:31:40.606 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.606 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.606 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:40.606 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:31:40.606 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:31:40.606 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.606 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:31:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:31:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:31:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:31:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:31:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:31:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:40.607 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:31:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 79.485828763000001, 2023-12-06T18:31:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:31:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:31:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.608 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:31:40.609 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:31:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:31:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:31:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:31:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:31:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:31:40.610 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:31:40.611 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.611 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.611 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:40.611 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:31:40.611 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:31:40.611 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:40.611 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:40.611 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:40.611 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:31:40.611 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:31:40.611 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:40.611 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:40.611 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:40.611 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:40.611 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:40.612 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:40.613 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:31:40.613 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:40.610+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:40.714 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:40.711+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:40.752 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:40.752 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:40.753 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:40.753 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:40.762 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:31:40.815 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:40.812+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:40.891 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:40.891 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:40.891 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:40.891 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:40.891 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:40.901 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:31:40.941 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:40.938+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:41.017 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:41.014+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:41.074 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:41.074 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 199.57987367300001, 2023-12-06T18:31:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:31:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:31:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.075 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:31:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:31:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.076 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:31:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:31:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:31:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:31:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:31:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:31:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.077 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:31:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:31:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:31:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:31:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.078 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.078 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:41.082 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:31:41.118 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:41.115+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:41.219 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:41.216+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:41.252 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:41.252 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:41.252 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:41.252 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:41.262 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:31:41.346 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:41.342+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:41.422 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:41.418+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:41.429 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:41.429 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:41.429 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:41.429 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:41.429 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:41.438 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:31:41.522 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:41.519+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:41.623 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:41.620+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:41.650 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:41.651 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:41.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:41.651 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:41.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.651 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:31:41.651 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:41.651 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 285.03490543800001, 2023-12-06T18:31:41.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:31:41.651 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.651 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.651 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:41.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:41.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:31:41.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:31:41.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:31:41.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:31:41.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:31:41.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:31:41.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:31:41.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:31:41.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:31:41.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:31:41.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.654 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:31:41.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:31:41.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.654 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:31:41.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:31:41.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.654 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:31:41.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:31:41.655 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.655 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.655 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.655 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:31:41.655 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:41.655 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 284.01207842899998, 2023-12-06T18:31:41.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:31:41.655 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.655 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.655 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:41.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:41.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:31:41.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:31:41.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:31:41.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:31:41.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:31:41.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:31:41.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.657 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:31:41.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:31:41.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:31:41.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:31:41.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:31:41.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:31:41.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:31:41.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:31:41.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:31:41.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:31:41.659 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.659 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.659 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.659 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:31:41.659 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:41.659 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 284.00003179100003, 2023-12-06T18:31:41.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:31:41.659 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.659 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.659 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:41.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:41.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:31:41.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:31:41.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:31:41.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:31:41.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:31:41.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:31:41.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:31:41.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:31:41.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:31:41.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:31:41.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:31:41.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:31:41.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:31:41.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:31:41.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:31:41.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:31:41.663 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.663 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.663 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.663 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:31:41.663 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:41.663 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 283.97863118399999, 2023-12-06T18:31:41.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:31:41.663 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.663 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.663 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:41.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:41.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:31:41.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:31:41.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:31:41.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:31:41.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.665 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.665 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:31:41.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:31:41.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.665 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.665 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:31:41.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:31:41.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.665 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:31:41.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:31:41.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:31:41.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:31:41.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:31:41.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:31:41.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:31:41.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:31:41.667 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.667 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.667 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.667 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:31:41.667 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:41.667 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 283.97745186100002, 2023-12-06T18:31:41.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:31:41.667 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.667 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.667 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:41.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:41.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:31:41.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:31:41.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:31:41.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:31:41.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:31:41.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:31:41.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:31:41.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:31:41.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:31:41.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:31:41.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:31:41.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:31:41.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:31:41.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:31:41.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:31:41.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:31:41.671 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.671 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.671 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.671 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:31:41.671 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:41.671 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 283.97687988000001, 2023-12-06T18:31:41.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:31:41.671 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.671 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.671 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:41.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:41.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:31:41.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:31:41.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:31:41.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:31:41.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:31:41.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:31:41.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:31:41.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:31:41.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:31:41.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:31:41.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:31:41.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:31:41.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:31:41.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:31:41.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:31:41.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:31:41.675 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.675 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.675 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.675 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:31:41.675 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:41.675 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 283.53104440999999, 2023-12-06T18:31:41.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:31:41.675 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.675 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.675 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:41.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:41.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:31:41.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:31:41.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:31:41.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:31:41.677 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.677 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.677 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.677 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:31:41.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:31:41.677 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.677 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.677 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.677 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:31:41.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:31:41.677 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.677 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.677 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.677 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:31:41.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:31:41.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:31:41.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:31:41.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:31:41.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:31:41.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:31:41.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:31:41.679 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.679 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.679 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.679 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:31:41.679 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:41.679 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 283.52944455400001, 2023-12-06T18:31:41.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:31:41.679 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.679 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.679 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:41.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:41.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:31:41.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:31:41.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.681 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:31:41.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:31:41.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.681 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.681 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:31:41.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:31:41.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.681 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.681 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:31:41.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:31:41.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:31:41.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:31:41.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:31:41.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:31:41.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:31:41.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:31:41.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:31:41.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:31:41.683 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.683 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.683 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.683 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:31:41.683 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:41.683 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 283.15023893799997, 2023-12-06T18:31:41.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:31:41.683 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.683 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.683 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:41.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:41.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:31:41.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:31:41.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:31:41.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:31:41.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:31:41.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:31:41.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:31:41.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:31:41.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:31:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:31:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:31:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:31:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:31:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:31:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.686 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.687 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.687 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:31:41.687 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:31:41.687 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.687 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.687 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.687 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.687 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.687 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:31:41.687 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:41.687 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 282.91273675600002, 2023-12-06T18:31:41.687 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:31:41.687 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.687 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.687 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.688 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.688 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.688 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:41.688 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.688 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.688 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.688 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.688 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:41.688 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.688 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.688 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.688 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.688 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:31:41.688 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:31:41.688 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.688 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.689 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:31:41.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:31:41.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.689 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:31:41.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:31:41.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.689 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:31:41.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:31:41.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:31:41.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:31:41.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:31:41.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:31:41.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:31:41.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:31:41.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:31:41.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:31:41.691 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.691 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.691 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.691 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:31:41.691 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:41.691 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 282.91200875700002, 2023-12-06T18:31:41.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:31:41.691 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.691 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.692 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:41.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:41.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:31:41.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:31:41.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.693 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:31:41.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:31:41.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.693 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:31:41.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:31:41.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.693 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:31:41.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:31:41.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:31:41.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:31:41.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:31:41.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:31:41.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:31:41.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:31:41.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:31:41.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:31:41.695 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.695 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.695 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.695 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:31:41.695 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:41.695 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 282.90699634700002, 2023-12-06T18:31:41.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:31:41.695 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.695 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.696 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:41.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:41.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:31:41.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:31:41.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:31:41.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:31:41.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:31:41.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:31:41.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:31:41.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:31:41.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.698 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:31:41.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:31:41.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.698 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:31:41.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:31:41.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.698 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:31:41.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:31:41.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.699 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:31:41.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:31:41.699 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.699 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.699 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.699 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:31:41.699 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:41.699 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 282.90146926800003, 2023-12-06T18:31:41.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:31:41.699 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.699 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.699 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:41.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:41.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:31:41.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:31:41.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.701 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:31:41.701 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:31:41.701 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.701 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.701 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.701 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:31:41.701 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:31:41.701 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.701 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.701 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.701 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:31:41.701 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:31:41.701 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.701 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.701 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.701 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:31:41.702 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:31:41.702 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.702 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.702 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.702 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:31:41.702 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:31:41.702 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.702 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.702 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.702 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:31:41.702 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:31:41.702 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.703 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.703 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.703 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:31:41.703 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:31:41.703 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.703 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.703 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.703 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.703 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.703 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:31:41.703 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:41.703 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 282.90095843099999, 2023-12-06T18:31:41.703 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:31:41.703 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.703 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.704 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:41.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:41.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:31:41.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:31:41.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:31:41.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:31:41.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:31:41.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:31:41.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:31:41.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:31:41.706 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.706 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.706 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.706 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:31:41.706 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:31:41.706 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.706 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.706 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.706 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:31:41.706 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:31:41.706 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.706 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.706 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.706 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:31:41.706 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:31:41.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.707 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.707 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:31:41.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:31:41.707 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.707 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.707 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.707 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:31:41.707 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:41.707 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 282.90049916999999, 2023-12-06T18:31:41.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:31:41.708 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.708 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.708 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:41.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:41.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:31:41.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:31:41.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:31:41.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:31:41.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:31:41.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:31:41.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:31:41.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:31:41.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.710 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.710 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:31:41.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:31:41.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.710 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.710 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:31:41.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:31:41.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.710 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.710 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:31:41.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:31:41.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.711 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.711 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:31:41.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:31:41.711 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.711 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.711 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.711 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:31:41.711 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:41.711 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 276.32058336300003, 2023-12-06T18:31:41.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:31:41.711 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.711 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.712 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:41.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:41.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:31:41.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:31:41.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:31:41.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:31:41.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:31:41.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:31:41.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:31:41.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:31:41.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.714 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.714 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:31:41.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:31:41.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.714 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.714 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:31:41.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:31:41.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.714 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.714 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:31:41.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:31:41.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.715 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:31:41.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:31:41.715 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.715 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.715 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.715 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:31:41.715 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:41.715 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 276.31434555800001, 2023-12-06T18:31:41.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:31:41.715 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.715 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.716 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:41.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:41.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:31:41.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:31:41.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:31:41.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:31:41.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:31:41.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:31:41.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:31:41.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:31:41.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.718 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.718 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:31:41.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:31:41.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.718 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.718 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:31:41.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:31:41.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.718 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.718 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:31:41.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:31:41.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:31:41.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:31:41.719 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.719 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.719 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.719 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:31:41.719 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:41.719 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 276.30828247900001, 2023-12-06T18:31:41.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:31:41.719 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.719 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.719 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:41.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:41.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:31:41.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:31:41.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:31:41.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:31:41.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:31:41.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:31:41.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:31:41.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:31:41.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:31:41.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:31:41.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:31:41.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:31:41.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:31:41.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:31:41.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:31:41.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:31:41.723 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.723 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.723 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.723 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:31:41.723 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:41.723 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 276.17089539400001, 2023-12-06T18:31:41.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:31:41.723 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.723 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.723 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.723 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:41.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:41.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:31:41.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:31:41.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:31:41.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:31:41.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:31:41.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:31:41.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:31:41.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:31:41.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:31:41.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:31:41.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:31:41.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:31:41.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:31:41.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:31:41.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:31:41.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:31:41.727 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.727 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.727 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.727 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:31:41.727 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:41.727 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 276.16422466699998, 2023-12-06T18:31:41.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:31:41.727 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:41.727 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:41.727 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:41.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.727 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:41.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:41.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:41.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:41.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:41.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:41.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:31:41.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:31:41.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:41.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:31:41.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:31:41.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:41.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:31:41.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:31:41.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:41.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:31:41.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:31:41.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:41.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:31:41.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:31:41.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:41.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:31:41.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:31:41.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:41.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:31:41.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:31:41.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:41.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:41.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:41.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:31:41.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:31:41.731 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.731 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.731 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.731 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:41.731 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:41.731 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:41.732 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:31:41.733 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:41.729+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:41.833 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:41.830+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:41.906 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:41.906 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:41.907 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:41.907 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:41.915 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:31:41.935 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:41.931+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:42.035 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:42.032+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:42.046 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:42.046 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:42.046 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:42.046 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:42.046 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:42.056 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:31:42.137 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:42.133+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:42.224 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:42.224 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:42.224 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:42.224 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:31:42.225 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:42.234 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:31:42.237 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:42.233+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:42.338 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:42.334+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:42.362 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:42.362 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:42.362 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:42.363 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:42.372 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:31:42.438 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:42.435+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:42.502 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:42.502 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:42.502 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:42.502 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:42.502 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:42.512 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:31:42.539 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:42.535+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:42.640 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:42.636+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:42.641 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:42.641 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:42.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:42.641 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:42.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:42.641 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:31:42.641 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:42.641 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 80.979364431999997, 2023-12-06T18:31:42.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:31:42.642 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:42.642 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:42.642 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:42.642 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:42.642 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:42.642 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:31:42.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:42.642 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:42.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:42.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:42.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:42.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:42.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:42.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:42.643 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:42.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:42.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:42.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:42.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:42.643 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:42.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:31:42.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:31:42.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:42.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:42.643 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:42.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:31:42.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:31:42.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:42.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:42.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:42.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:31:42.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:31:42.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:42.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:42.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:42.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:31:42.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:31:42.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:42.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:42.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:42.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:31:42.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:31:42.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:42.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:42.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:42.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:31:42.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:31:42.645 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:42.645 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:42.645 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:42.645 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:42.645 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:42.645 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:42.650 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:31:42.720 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-12-06T18:31:42.721 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi152 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b" 2023-12-06T18:31:42.721 INFO:tasks.ceph.mon.b:Stopped 2023-12-06T18:31:42.721 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2023-12-06T18:31:42.721 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:31:42.766 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:42.762+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:42.793 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:42.793 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:42.794 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:42.794 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:42.806 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:31:42.842 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:42.838+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:42.942 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:42.939+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:42.983 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:42.983 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:42.983 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:42.983 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:42.983 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:42.992 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:31:43.044 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:43.040+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:43.128 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:43.128 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:43.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:43.128 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:31:43.128 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:43.137 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:31:43.145 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:43.141+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:43.245 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:43.242+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:43.317 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:43.317 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:43.317 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:43.317 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:43.327 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:31:43.372 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:43.368+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:43.447 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:43.443+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:43.463 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:43.464 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:43.464 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:43.473 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:31:43.558 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:43.554+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:43.649 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:43.645+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:43.713 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:31:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 288.30204517700002, 2023-12-06T18:31:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:31:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.713 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:31:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:43.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:31:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:31:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:31:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:31:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:31:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:31:43.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:31:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:31:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:31:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:31:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:31:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:31:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:31:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:31:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:31:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:31:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:31:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:31:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:31:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:31:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:31:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:31:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:31:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:31:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:31:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:31:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:31:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 286.19347193800002, 2023-12-06T18:31:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:31:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.719 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:31:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:31:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:31:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:31:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:31:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:31:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:31:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.721 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:31:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:31:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:31:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:31:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:31:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:31:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:31:43.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:31:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:31:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:31:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:31:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:31:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:31:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:31:43.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:31:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:31:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 286.19272670599997, 2023-12-06T18:31:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:31:43.724 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:31:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:31:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:31:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:31:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:31:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:31:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:31:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:31:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:31:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:31:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:31:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:31:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:31:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:31:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:31:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:31:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:31:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:31:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:31:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:31:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:31:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:31:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:31:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 286.19204295100002, 2023-12-06T18:31:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:31:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:31:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.730 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:31:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:31:43.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:31:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:31:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:31:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:31:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:31:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:31:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.732 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:31:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:31:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:31:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:31:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:31:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:31:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:31:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:31:43.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:31:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:31:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:31:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:31:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:31:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:31:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 286.18646969600002, 2023-12-06T18:31:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:31:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.735 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:31:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:31:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:31:43.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:31:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:31:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:31:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:31:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:31:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:31:43.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:31:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:31:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:31:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:31:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:31:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:31:43.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:31:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:31:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:31:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:31:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:31:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:31:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:31:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:31:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 286.18579605600002, 2023-12-06T18:31:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:31:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.740 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:31:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:31:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:31:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:31:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:31:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:31:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:31:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:31:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:31:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:31:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:31:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:31:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:31:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:31:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:31:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:31:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:31:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:31:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:31:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:31:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:31:43.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:31:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:31:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 286.18513385699998, 2023-12-06T18:31:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:31:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.745 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:31:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:43.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:31:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:31:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:31:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:31:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:31:43.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:31:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:31:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:31:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:31:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:31:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:31:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:31:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.748 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:31:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:31:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:31:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:31:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:31:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:31:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.749 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:31:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:31:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:31:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:31:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 286.112424422, 2023-12-06T18:31:43.750 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:31:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:31:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:43.751 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:31:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:31:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:31:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:31:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:31:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:31:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:31:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:31:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:31:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:31:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:31:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:31:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.753 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:31:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:31:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:31:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:31:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:31:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:31:43.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:31:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:31:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:31:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:31:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.755 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 286.11174271300001, 2023-12-06T18:31:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:31:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:31:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.756 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:31:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:31:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:31:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:31:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:31:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:31:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:31:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:31:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:31:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:31:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.758 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:31:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:31:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:31:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:31:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:31:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:31:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:31:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:31:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:31:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:31:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:31:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:31:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.760 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 285.03293862499999, 2023-12-06T18:31:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:31:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:31:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:43.761 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:31:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:31:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.762 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:31:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:31:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:31:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:31:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:31:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:31:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.763 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:31:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:31:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:31:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:31:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:31:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:31:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.764 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:31:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:31:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:31:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:31:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:31:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:31:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:31:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:31:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 285.03152205100002, 2023-12-06T18:31:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:31:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.766 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:31:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:31:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:31:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:31:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:31:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:31:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:31:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:31:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:31:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:31:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:31:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:31:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:31:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:31:43.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:31:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:31:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:31:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:31:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:31:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:31:43.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:31:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:31:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:31:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 285.03043692599999, 2023-12-06T18:31:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:31:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.771 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:31:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:31:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:31:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:31:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:31:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:31:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:31:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:31:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:31:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:31:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:31:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:31:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:31:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:31:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:31:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:31:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:31:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:31:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:31:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:31:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:31:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:31:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:31:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:43.776 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 284.995287824, 2023-12-06T18:31:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:31:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:31:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:31:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:31:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:31:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:31:43.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:31:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:31:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:31:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:31:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:31:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:31:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:31:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:31:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:31:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:31:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:31:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:31:43.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:31:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:31:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:31:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:31:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:31:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:31:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.781 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 278.44555729899997, 2023-12-06T18:31:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:31:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:31:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.782 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:31:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:31:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:31:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:31:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:31:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:31:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:31:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:31:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:31:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:31:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:31:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:31:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:31:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:31:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:31:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:31:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:31:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:31:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:31:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:31:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:31:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:31:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 278.44475169100002, 2023-12-06T18:31:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:31:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.787 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:31:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:43.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:31:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:31:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:31:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:31:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:31:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:31:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:31:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:31:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:31:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:31:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:31:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:31:43.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:31:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:31:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:31:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:31:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:31:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:31:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:31:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:31:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:31:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:31:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 278.44406088199997, 2023-12-06T18:31:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:31:43.792 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:31:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:31:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:31:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:31:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:31:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:31:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:31:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:31:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:31:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:31:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:31:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:31:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:31:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:31:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:31:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:31:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:31:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:31:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:31:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:31:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:31:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:31:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:31:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:43.797 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:43.798 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 278.43976337399999, 2023-12-06T18:31:43.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:31:43.798 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.798 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.798 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.798 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.798 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.798 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:31:43.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.798 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:43.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:43.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:31:43.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:31:43.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:31:43.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:31:43.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:31:43.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:31:43.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:31:43.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:31:43.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:31:43.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:31:43.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:31:43.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:31:43.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:31:43.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:31:43.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:31:43.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:31:43.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:31:43.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:31:43.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.802 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:31:43.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:31:43.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.802 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:31:43.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:31:43.802 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.802 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.802 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.803 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:43.803 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:43.803 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 278.43909569900001, 2023-12-06T18:31:43.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:31:43.803 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.803 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.803 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.803 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.803 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.803 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:31:43.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.803 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:43.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:43.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:31:43.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:31:43.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:31:43.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:31:43.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:31:43.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:31:43.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:31:43.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:31:43.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:31:43.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:31:43.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:31:43.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:31:43.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:31:43.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:31:43.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:31:43.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:31:43.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:31:43.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:31:43.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:31:43.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:31:43.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.808 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:31:43.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:31:43.808 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.808 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.808 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.808 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:43.808 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:43.808 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 278.43844027099999, 2023-12-06T18:31:43.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:31:43.808 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.808 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.808 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.808 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.808 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.809 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:31:43.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.809 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:43.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:43.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:31:43.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:31:43.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:31:43.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:31:43.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:31:43.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:31:43.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:31:43.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:31:43.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:31:43.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:31:43.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:31:43.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:31:43.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.811 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.811 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:31:43.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:31:43.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:31:43.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:31:43.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:31:43.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:31:43.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:31:43.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:31:43.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:31:43.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:31:43.813 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.813 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.813 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.813 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:43.813 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:43.813 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 278.39778572400002, 2023-12-06T18:31:43.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:31:43.814 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:43.814 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:43.814 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:43.814 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:43.814 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:43.814 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:31:43.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.814 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:43.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.814 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:43.814 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:43.814 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.814 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:31:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:31:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:31:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:31:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.815 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:31:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:31:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:31:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:31:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:31:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:31:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:31:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:31:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:31:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:31:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:31:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:31:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:43.817 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:31:43.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:31:43.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:43.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:31:43.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:31:43.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:43.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:43.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:43.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:31:43.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:31:43.818 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.818 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.818 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.819 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:43.819 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:43.819 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:43.820 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:31:43.821 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:43.817+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:43.921 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:43.917+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:43.998 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:44.071 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:44.072 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:44.072 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:44.073 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:31:44.074 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:44.070+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:44.175 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:44.171+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:44.210 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:44.211 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:44.211 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:44.211 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:44.211 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:44.220 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:31:44.276 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:44.272+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:44.377 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:44.374+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:44.401 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:44.401 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:44.401 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:44.402 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:44.402 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.402 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:31:44.402 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:44.402 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 262.58545755400002, 2023-12-06T18:31:44.402 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:31:44.402 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:44.402 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:31:44.402 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:44.402 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.402 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:44.402 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:44.402 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:44.402 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.403 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:44.403 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:44.403 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:44.403 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.403 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:44.403 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:31:44.403 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:31:44.403 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.403 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:44.403 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:31:44.403 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:31:44.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.404 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:44.404 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:31:44.404 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:31:44.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.404 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:44.404 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:31:44.404 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:31:44.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.404 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:44.404 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:31:44.404 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:31:44.405 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.405 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.405 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:44.405 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:31:44.405 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:31:44.405 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.405 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.405 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:44.405 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:31:44.405 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:31:44.405 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:44.405 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:44.405 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:44.405 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:44.405 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:44.406 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:44.410 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:31:44.478 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:44.475+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:44.550 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:44.550 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:44.550 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:44.550 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:44.559 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:31:44.600 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:44.596+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:44.681 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:44.677+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:44.735 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:44.736 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:44.736 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:44.736 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:44.736 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:44.746 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:31:44.791 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:44.787+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:44.883 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:44.879+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:44.888 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:44.888 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:44.888 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:44.888 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:44.888 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.888 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:44.888 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:44.888 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 143.78843402499999, 2023-12-06T18:31:44.888 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:31:44.888 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:44.888 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:44.888 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:44.888 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:44.889 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:44.889 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:31:44.889 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.889 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:44.889 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.889 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:44.889 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:44.889 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:44.889 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.889 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.889 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:44.889 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:44.889 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:44.889 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.890 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.890 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:44.890 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:31:44.890 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:31:44.890 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.890 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.890 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:44.890 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:31:44.890 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:31:44.890 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.890 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.890 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:44.890 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:31:44.890 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:31:44.890 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.891 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:44.891 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:31:44.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:31:44.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.891 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.891 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:44.891 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:31:44.891 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:31:44.891 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.892 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:44.892 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:31:44.892 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:31:44.892 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:44.892 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:44.892 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:44.892 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.892 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.892 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:31:44.892 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:44.892 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 83.770176723000006, 2023-12-06T18:31:44.892 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:31:44.892 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:44.892 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:44.893 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:44.893 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:44.893 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:44.893 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:31:44.893 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.893 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:44.893 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.893 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:44.893 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:44.893 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:44.893 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.893 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.893 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:44.893 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:44.893 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:44.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:44.894 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:31:44.894 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:31:44.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:44.894 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:31:44.894 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:31:44.894 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.894 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.894 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:44.895 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:31:44.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:31:44.895 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.895 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.895 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:44.895 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:31:44.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:31:44.895 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.895 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.895 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:44.895 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:31:44.895 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:31:44.895 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:44.895 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:44.896 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:44.896 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:31:44.896 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:31:44.896 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:44.896 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:44.896 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:44.896 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:44.896 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:44.896 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:44.897 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:31:44.984 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:44.980+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:45.035 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:45.035 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:45.035 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:45.035 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:45.044 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:31:45.085 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:45.081+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:45.174 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:45.174 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:45.174 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:45.174 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:45.174 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:45.183 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:31:45.186 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:45.182+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:45.298 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:45.294+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:45.314 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:45.314 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:45.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:45.314 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:45.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.314 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:45.314 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:45.314 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 203.820339214, 2023-12-06T18:31:45.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:31:45.314 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.314 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:45.314 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:45.314 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:45.315 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:45.315 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:31:45.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.315 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:45.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:45.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.316 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:31:45.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:31:45.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.316 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:31:45.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:31:45.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.316 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:31:45.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:31:45.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.317 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.317 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:31:45.317 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:31:45.317 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.317 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.317 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.317 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:31:45.317 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:31:45.317 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.317 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.317 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.317 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:31:45.317 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:31:45.317 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.317 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.318 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.318 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.318 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.318 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:45.323 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:31:45.388 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:45.384+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:45.452 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:31:45.452 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:31:45.453 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2023-12-06T18:31:45.453 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2023-12-06T18:31:45.493 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:45.493 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:45.493 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:45.493 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:45.498 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:45.494+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:45.503 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:31:45.590 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:45.587+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:45.635 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:45.635 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:45.635 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:45.635 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:45.635 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:45.645 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:31:45.691 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:45.688+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:45.799 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:45.795+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:45.809 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-12-06T18:31:45.853 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:45.853 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:45.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:45.853 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:45.853 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.853 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:31:45.853 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:45.853 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 289.23705237000001, 2023-12-06T18:31:45.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:31:45.853 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.854 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.854 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.854 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.854 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:45.854 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.854 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.854 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:45.854 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:31:45.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:31:45.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:31:45.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:31:45.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:31:45.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:31:45.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:31:45.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:31:45.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:31:45.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:31:45.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:31:45.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:31:45.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:31:45.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:31:45.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:31:45.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:31:45.857 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.857 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.857 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.857 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:31:45.857 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:45.858 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 288.21422536099999, 2023-12-06T18:31:45.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:31:45.858 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.858 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.858 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:45.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.858 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:45.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:31:45.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:31:45.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:31:45.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:31:45.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.860 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.860 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:31:45.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:31:45.860 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.860 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.860 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:31:45.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:31:45.860 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.860 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.860 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:31:45.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:31:45.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:31:45.861 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:31:45.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:31:45.861 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:31:45.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:31:45.862 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:31:45.862 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.862 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.862 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.862 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.862 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.862 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:31:45.862 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:45.862 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 288.20217872299997, 2023-12-06T18:31:45.862 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:31:45.862 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.862 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.862 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.862 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:45.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:45.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:31:45.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:31:45.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:31:45.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:31:45.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:31:45.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:31:45.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:31:45.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:31:45.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:31:45.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:31:45.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:31:45.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:31:45.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:31:45.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:31:45.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:31:45.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:31:45.866 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.866 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.866 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.866 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:31:45.866 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:45.866 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 288.180778116, 2023-12-06T18:31:45.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:31:45.866 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.866 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.867 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.867 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:45.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.867 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.867 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:45.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.867 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.867 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:31:45.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:31:45.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:31:45.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:31:45.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:31:45.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:31:45.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:31:45.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:31:45.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:31:45.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:31:45.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:31:45.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:31:45.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.871 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:31:45.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:31:45.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.871 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.871 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:31:45.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:31:45.871 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.871 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.871 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.871 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:31:45.871 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:45.871 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 288.17959879300003, 2023-12-06T18:31:45.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:31:45.872 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.872 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.872 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:45.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:45.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:31:45.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:31:45.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:31:45.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:31:45.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:31:45.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:31:45.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:31:45.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:31:45.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:31:45.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:31:45.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:31:45.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:31:45.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:31:45.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:31:45.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:31:45.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:31:45.875 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.875 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.875 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.876 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:31:45.876 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:45.876 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 288.17902681200002, 2023-12-06T18:31:45.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:31:45.876 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.876 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.876 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:45.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:45.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:31:45.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:31:45.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:31:45.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:31:45.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:31:45.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:31:45.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:31:45.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:31:45.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:31:45.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:31:45.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:31:45.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:31:45.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:31:45.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:31:45.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:31:45.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:31:45.880 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.880 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.880 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.880 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:31:45.880 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:45.880 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 287.733191342, 2023-12-06T18:31:45.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:31:45.880 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.880 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.881 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:45.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:45.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:31:45.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:31:45.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:31:45.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:31:45.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:31:45.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:31:45.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:31:45.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:31:45.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:31:45.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:31:45.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:31:45.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:31:45.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.884 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.884 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.884 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:31:45.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:31:45.884 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.884 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.884 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.884 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:31:45.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:31:45.884 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.884 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.884 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.885 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:31:45.885 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:45.885 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 287.73159148600001, 2023-12-06T18:31:45.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:31:45.885 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.885 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.885 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:45.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:45.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:31:45.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:31:45.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:31:45.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:31:45.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:31:45.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:31:45.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:31:45.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:31:45.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:31:45.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:31:45.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:31:45.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:31:45.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:31:45.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:31:45.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:31:45.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:31:45.889 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.889 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.889 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.889 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.889 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:31:45.889 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:45.889 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 287.35238586999998, 2023-12-06T18:31:45.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:31:45.889 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.889 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.889 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.889 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:45.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:45.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:31:45.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:31:45.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:31:45.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:31:45.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:31:45.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:31:45.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:31:45.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:31:45.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:31:45.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:31:45.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:31:45.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:31:45.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.893 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:31:45.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:31:45.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.893 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.893 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:31:45.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:31:45.893 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.893 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.893 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.893 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:31:45.893 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:45.893 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 287.11488368800002, 2023-12-06T18:31:45.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:31:45.894 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.894 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.894 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:45.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:45.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:31:45.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:31:45.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:31:45.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:31:45.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:31:45.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:31:45.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:31:45.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:31:45.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:31:45.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:31:45.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.897 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:31:45.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:31:45.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.897 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.897 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:31:45.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:31:45.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.897 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.897 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:31:45.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:31:45.897 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.897 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.897 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.898 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:31:45.898 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:45.898 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 287.11415568899997, 2023-12-06T18:31:45.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:31:45.898 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.898 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.898 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:45.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:45.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:31:45.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:31:45.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:31:45.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:31:45.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:31:45.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:31:45.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:31:45.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:31:45.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:31:45.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:31:45.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:31:45.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:31:45.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:31:45.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:31:45.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:31:45.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:31:45.902 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.902 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.902 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.902 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:31:45.902 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:45.902 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 287.10914327900002, 2023-12-06T18:31:45.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:31:45.903 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.903 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.903 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:45.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:45.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:31:45.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:31:45.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:31:45.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:31:45.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:31:45.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:31:45.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:31:45.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:31:45.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:31:45.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:31:45.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:31:45.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:31:45.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:31:45.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:31:45.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:31:45.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:31:45.906 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.906 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.906 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.907 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:31:45.907 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:45.907 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 287.10361619999998, 2023-12-06T18:31:45.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:31:45.907 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.907 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.907 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:45.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:45.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:31:45.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:31:45.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:31:45.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:31:45.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:31:45.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:31:45.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:31:45.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:31:45.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:31:45.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:31:45.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:31:45.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:31:45.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:31:45.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:31:45.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:31:45.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:31:45.911 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.911 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.911 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.911 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:31:45.911 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:45.911 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 287.103105363, 2023-12-06T18:31:45.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:31:45.911 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.912 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.912 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:45.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:45.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:31:45.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:31:45.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:31:45.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:31:45.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:31:45.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:31:45.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:31:45.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:31:45.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:31:45.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:31:45.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:31:45.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:31:45.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:31:45.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:31:45.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:31:45.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:31:45.915 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.915 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.915 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.915 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:31:45.915 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:45.915 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 287.10264610199999, 2023-12-06T18:31:45.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:31:45.916 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.916 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.916 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:45.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:45.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:31:45.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:31:45.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:31:45.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:31:45.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:31:45.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:31:45.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:31:45.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:31:45.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:31:45.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:31:45.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:31:45.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:31:45.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:31:45.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:31:45.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:31:45.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:31:45.919 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.919 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.919 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.919 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:31:45.920 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:45.920 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 280.52273029499997, 2023-12-06T18:31:45.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:31:45.920 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.920 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.920 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:45.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:45.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:31:45.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:31:45.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:31:45.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:31:45.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:31:45.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:31:45.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:31:45.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:31:45.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:31:45.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:31:45.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:31:45.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:31:45.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:31:45.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:31:45.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:31:45.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:31:45.923 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.923 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.923 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.923 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:31:45.924 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:45.924 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 280.51649249000002, 2023-12-06T18:31:45.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:31:45.924 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.924 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.924 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:45.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:45.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:31:45.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:31:45.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:31:45.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:31:45.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:31:45.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:31:45.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:31:45.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:31:45.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:31:45.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:31:45.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:31:45.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:31:45.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:31:45.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:31:45.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:31:45.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:31:45.927 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.927 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.927 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.928 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:31:45.928 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:45.928 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 280.51042941100002, 2023-12-06T18:31:45.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:31:45.928 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.928 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.928 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:45.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:45.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:31:45.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:31:45.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:31:45.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:31:45.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:31:45.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:31:45.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:31:45.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:31:45.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:31:45.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:31:45.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.930 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.930 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:31:45.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:31:45.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:31:45.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:31:45.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:31:45.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:31:45.931 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.931 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.931 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.931 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:31:45.932 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:45.932 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 280.37304232600002, 2023-12-06T18:31:45.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:31:45.932 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.932 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.932 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.932 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:45.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.932 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:45.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:31:45.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:31:45.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:31:45.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:31:45.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:31:45.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:31:45.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:31:45.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:31:45.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:31:45.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:31:45.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.935 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.935 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:31:45.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:31:45.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.935 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.935 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:31:45.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:31:45.935 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.935 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.935 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.935 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:31:45.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:31:45.935 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.935 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.936 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.936 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.936 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.936 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:31:45.936 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:45.936 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 280.36637159899999, 2023-12-06T18:31:45.936 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:31:45.936 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:45.936 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:45.936 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:45.936 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.936 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:45.936 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:45.936 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.936 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:45.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:45.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:45.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:45.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:31:45.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:31:45.937 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.937 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.937 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:45.937 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:31:45.937 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:31:45.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:45.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:31:45.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:31:45.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:45.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:31:45.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:31:45.938 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.938 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.938 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:45.938 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:31:45.938 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:31:45.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.939 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.939 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:45.939 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:31:45.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:31:45.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.939 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.939 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:45.939 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:31:45.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:31:45.939 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:45.939 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:45.939 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:45.939 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:31:45.939 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:31:45.939 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.939 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.940 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.940 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:45.940 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:45.940 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:45.941 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:31:45.941 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:45.938+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:46.043 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:46.039+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:46.109 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:46.109 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:46.109 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:46.109 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:46.118 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:31:46.164 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:46.160+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:46.244 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:46.240+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:46.245 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:46.245 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:46.245 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:46.245 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:46.245 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:46.254 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:31:46.344 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:46.341+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:46.422 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:46.423 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:46.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:46.423 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:31:46.423 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:46.432 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:31:46.435 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":38,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:31:46.445 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:46.442+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:46.547 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:46.543+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:46.562 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:46.562 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:46.563 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:46.563 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:46.572 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:31:46.657 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:46.653+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:46.740 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:46.740 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:46.740 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:46.740 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:46.740 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:46.748 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:46.744+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:46.749 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:31:46.848 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:46.845+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:46.918 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:46.919 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:46.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:46.919 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:46.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:46.919 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:31:46.919 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:46.919 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 85.256623133999994, 2023-12-06T18:31:46.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:31:46.919 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:46.919 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:46.919 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:46.919 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:46.919 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:46.919 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:31:46.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:46.920 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:46.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:46.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:46.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:46.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:46.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:46.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:46.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:46.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:46.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:46.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:46.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:46.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:46.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:31:46.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:31:46.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:46.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:46.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:46.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:31:46.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:31:46.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:46.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:46.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:46.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:31:46.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:31:46.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:46.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:46.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:46.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:31:46.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:31:46.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:46.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:46.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:46.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:31:46.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:31:46.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:46.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:46.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:46.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:31:46.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:31:46.922 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:46.922 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:46.923 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:46.923 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:46.923 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:46.923 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:46.927 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:31:46.972 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:46.968+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:47.050 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:47.046+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:47.063 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:47.064 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:47.064 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:47.064 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:47.074 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:31:47.158 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:47.154+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:47.252 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:47.249+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:47.254 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:47.254 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:47.254 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:47.254 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:47.254 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:47.264 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:31:47.353 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:47.350+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:47.441 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:47.441 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:47.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:47.442 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:31:47.442 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:47.450 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:31:47.454 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:47.451+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:47.555 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:47.552+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:47.628 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:47.628 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:47.628 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:47.628 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:47.637 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:31:47.679 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:47.675+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:47.756 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:47.753+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:47.814 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:47.814 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:47.815 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:47.815 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:47.815 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:47.824 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:31:47.857 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:47.854+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:47.958 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:47.955+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:48.019 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:48.019 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:48.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:48.019 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:48.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.019 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:31:48.019 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 292.609072825, 2023-12-06T18:31:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:31:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:31:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:31:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:31:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:31:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:31:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.021 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:31:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:31:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:31:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:31:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:31:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:31:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:31:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:31:48.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:31:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:31:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:31:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:31:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:31:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:31:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:31:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:31:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:31:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:31:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:31:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:31:48.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:31:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:31:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:31:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 290.50049958599999, 2023-12-06T18:31:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:31:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.025 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:31:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.026 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:31:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:31:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:31:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:31:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:31:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:31:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:31:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:31:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:31:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:31:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:31:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:31:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:31:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:31:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:31:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:31:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:31:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:31:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:31:48.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:31:48.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:31:48.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:31:48.030 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.030 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.030 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.030 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:48.031 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:48.031 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 290.499754354, 2023-12-06T18:31:48.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:31:48.031 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.031 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.031 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.031 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.031 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.031 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:31:48.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.031 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:48.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:48.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:31:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:31:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:31:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:31:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:31:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:31:48.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.034 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.034 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:31:48.034 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:31:48.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.034 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.034 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:31:48.034 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:31:48.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.034 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.034 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:31:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:31:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:31:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:31:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:31:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:31:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:31:48.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:31:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:31:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:31:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:31:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:31:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:48.036 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 290.49907059899999, 2023-12-06T18:31:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:31:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:31:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:48.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:31:48.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:31:48.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:31:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:31:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:31:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:31:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:31:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:31:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:31:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:31:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:31:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:31:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:31:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:31:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:31:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:31:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:31:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:31:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:31:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:31:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.041 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:31:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:31:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 290.49349734399999, 2023-12-06T18:31:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:31:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.042 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:31:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:31:48.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:31:48.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:31:48.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:31:48.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:31:48.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:31:48.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:31:48.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:31:48.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.046 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.046 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:31:48.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:31:48.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.046 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.046 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:31:48.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:31:48.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:31:48.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:31:48.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:31:48.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:31:48.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:31:48.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:31:48.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:31:48.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:31:48.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:31:48.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:31:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 290.49282370399999, 2023-12-06T18:31:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:31:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.049 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:31:48.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.050 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:48.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:48.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:31:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:31:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:31:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:31:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:31:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:31:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:31:48.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:31:48.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:31:48.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:31:48.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:31:48.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:31:48.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:31:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:31:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:31:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:31:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:31:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:31:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:31:48.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:31:48.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.054 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:31:48.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:31:48.054 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.054 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.054 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.054 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.054 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 290.49216150500001, 2023-12-06T18:31:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:31:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:31:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:48.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:31:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:31:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:31:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:31:48.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:31:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:31:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:31:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:31:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:31:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:31:48.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:31:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:31:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:31:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:31:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:31:48.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:31:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:31:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:31:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:31:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:31:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:31:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:31:48.059 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 290.41945206999998, 2023-12-06T18:31:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:31:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.060 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:31:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.061 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:31:48.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:31:48.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:31:48.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:31:48.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:31:48.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:31:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:31:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:31:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:31:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:31:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:31:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:31:48.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:31:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:31:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:31:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:31:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:31:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:31:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:31:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:31:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:31:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:31:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 290.41877036099999, 2023-12-06T18:31:48.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:31:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:31:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.066 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:31:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:31:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:31:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:31:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.067 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:31:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:31:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:31:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:31:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:31:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:31:48.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:31:48.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:31:48.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:31:48.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:31:48.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.070 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:31:48.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:31:48.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.070 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.070 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:31:48.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:31:48.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.070 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.070 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:31:48.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:31:48.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:31:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:31:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 289.33996627300002, 2023-12-06T18:31:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:31:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.071 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:31:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.072 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:31:48.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:31:48.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:31:48.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:31:48.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:31:48.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:31:48.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.074 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:31:48.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:31:48.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:31:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:31:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:31:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:31:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:31:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:31:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.075 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:31:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:31:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:31:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:31:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:31:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:31:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:31:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:31:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 289.338549699, 2023-12-06T18:31:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:31:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.077 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:31:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:31:48.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:31:48.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:31:48.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:31:48.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:31:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:31:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:31:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:31:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:31:48.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:31:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:31:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:31:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:31:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:31:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:31:48.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:31:48.082 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.082 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.082 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.082 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:31:48.082 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:31:48.082 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.082 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.082 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.082 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:31:48.082 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:31:48.082 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.082 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.083 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.083 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:31:48.083 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:31:48.083 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.083 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.083 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.083 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.083 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:48.083 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 289.33746457400002, 2023-12-06T18:31:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:31:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:31:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:48.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:48.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:31:48.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:31:48.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:31:48.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:31:48.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:31:48.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:31:48.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.087 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:31:48.087 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:31:48.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.087 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:31:48.087 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:31:48.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:31:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:31:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:31:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:31:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:31:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:31:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.088 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:31:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:31:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:31:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:31:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:31:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:31:48.089 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.090 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.090 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.090 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:48.090 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:48.090 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 289.30231547199998, 2023-12-06T18:31:48.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:31:48.090 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.091 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.091 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.091 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.091 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.091 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:31:48.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.091 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:48.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:48.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.093 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:31:48.093 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:31:48.093 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.093 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.093 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.093 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:31:48.093 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:31:48.093 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.094 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.094 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.094 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:31:48.094 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:31:48.094 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.094 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.094 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.094 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:31:48.094 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:31:48.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.095 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.095 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.095 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:31:48.095 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:31:48.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.095 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.095 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.095 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:31:48.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:31:48.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.096 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.096 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:31:48.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:31:48.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.096 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.097 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:31:48.097 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:31:48.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.097 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.097 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:31:48.097 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:31:48.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.098 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.098 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:31:48.098 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:31:48.098 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.098 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.098 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.098 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:31:48.098 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:31:48.098 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.099 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.099 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.099 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.099 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:48.099 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:48.099 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 282.752584947, 2023-12-06T18:31:48.099 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:31:48.100 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.100 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.100 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.100 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.100 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.100 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:31:48.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.100 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:48.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:48.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:31:48.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:31:48.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:31:48.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:31:48.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:31:48.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:31:48.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:31:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:31:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:31:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:31:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:31:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:31:48.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.104 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.104 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:31:48.104 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:31:48.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.104 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.104 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:31:48.104 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:31:48.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.105 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.105 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:31:48.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:31:48.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.105 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.105 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:31:48.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:31:48.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.106 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.106 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:31:48.106 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:31:48.106 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.106 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.106 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.106 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:48.106 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:48.106 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 282.751779339, 2023-12-06T18:31:48.106 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:31:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:31:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.108 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.108 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:48.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.108 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.108 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:31:48.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:31:48.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.108 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.108 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:31:48.109 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:31:48.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.109 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.109 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.109 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:31:48.109 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:31:48.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.109 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.109 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.109 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:31:48.109 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:31:48.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.110 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.110 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.110 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:31:48.110 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:31:48.110 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.110 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.110 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.110 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:31:48.110 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:31:48.110 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.110 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.110 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:31:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:31:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:31:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:31:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:31:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:31:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:31:48.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:31:48.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:31:48.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:31:48.112 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.112 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.112 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.113 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:48.113 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:48.113 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 282.75108853, 2023-12-06T18:31:48.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:31:48.113 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.113 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.113 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.113 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.113 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.114 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:31:48.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.114 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.114 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:48.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.114 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:31:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:31:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:31:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:31:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:31:48.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:31:48.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:31:48.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:31:48.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:31:48.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:31:48.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:31:48.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:31:48.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:31:48.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:31:48.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:31:48.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:31:48.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:31:48.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:31:48.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:31:48.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:31:48.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:31:48.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:31:48.122 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.122 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.123 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.123 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:48.123 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:48.123 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 282.74679102200002, 2023-12-06T18:31:48.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:31:48.123 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.124 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.124 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.124 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.124 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.124 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:31:48.124 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.124 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.124 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.124 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.125 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:48.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.125 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.125 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:48.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.126 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.126 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.126 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:31:48.126 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:31:48.126 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.126 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.126 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.126 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:31:48.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:31:48.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.127 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.127 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:31:48.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:31:48.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.127 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.127 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:31:48.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:31:48.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:31:48.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:31:48.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.128 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:31:48.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:31:48.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.128 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:31:48.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:31:48.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.128 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.129 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.129 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:31:48.129 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:31:48.129 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.129 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.129 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.129 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:31:48.129 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:31:48.129 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.129 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.129 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.129 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:31:48.129 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:31:48.129 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.129 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.130 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.130 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:31:48.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:31:48.130 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.130 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.130 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.130 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.130 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:48.130 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:48.130 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 282.74612334699998, 2023-12-06T18:31:48.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:31:48.130 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.131 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.131 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.131 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.131 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.131 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:31:48.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.131 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:48.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:48.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:31:48.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:31:48.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:31:48.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:31:48.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:31:48.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:31:48.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:31:48.134 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:31:48.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:31:48.134 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:31:48.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:31:48.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:31:48.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.135 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.135 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:31:48.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:31:48.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.135 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.135 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:31:48.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:31:48.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.135 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.135 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:31:48.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:31:48.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:31:48.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:31:48.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:31:48.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:31:48.136 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.136 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.136 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.137 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:48.137 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:48.137 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 282.74546791900002, 2023-12-06T18:31:48.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:31:48.137 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.137 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.137 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.137 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.137 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.137 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:31:48.137 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.137 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.137 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:48.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.138 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:48.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.138 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:31:48.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:31:48.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:31:48.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:31:48.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:31:48.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:31:48.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:31:48.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:31:48.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:31:48.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:31:48.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:31:48.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:31:48.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:31:48.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:31:48.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:31:48.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:31:48.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:31:48.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:31:48.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:31:48.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:31:48.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:31:48.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:31:48.142 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.142 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.142 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.143 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:48.143 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:48.143 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 282.70481337199999, 2023-12-06T18:31:48.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:31:48.143 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.143 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:48.143 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:48.143 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:48.143 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:48.143 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:31:48.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.143 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:48.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:48.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:31:48.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:31:48.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:31:48.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:31:48.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:31:48.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:31:48.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:31:48.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:31:48.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:31:48.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:31:48.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:31:48.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:31:48.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:48.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:31:48.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:31:48.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:48.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:31:48.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:31:48.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:48.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:31:48.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:31:48.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:48.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:31:48.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:31:48.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:31:48.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:31:48.148 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.148 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.148 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.148 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.148 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.148 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:48.150 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:31:48.151 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:48.146+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:48.251 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:48.247+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:48.330 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:48.330 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:48.330 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:48.330 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:48.340 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:31:48.352 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:48.348+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:48.452 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:48.449+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:48.516 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:48.516 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:48.516 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:48.516 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:48.516 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:48.525 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:31:48.553 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:48.549+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:48.654 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:48.650+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:48.669 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:48.669 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:48.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:48.669 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:48.669 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.669 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:31:48.669 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:48.669 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 266.85286289599998, 2023-12-06T18:31:48.669 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:31:48.669 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:48.669 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:31:48.669 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:48.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.670 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:48.670 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:48.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.670 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:48.670 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:48.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:48.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.670 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.670 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:48.670 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:31:48.670 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:31:48.670 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:48.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:31:48.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:31:48.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:48.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:31:48.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:31:48.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.671 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.671 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:48.671 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:31:48.671 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:31:48.671 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:48.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:31:48.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:31:48.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:48.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:31:48.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:31:48.672 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:48.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:48.672 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:48.672 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:31:48.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:31:48.672 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.672 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.673 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.673 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:48.673 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:48.673 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:48.678 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:31:48.754 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:48.751+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:48.816 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:48.816 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:48.816 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:48.816 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:48.825 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:31:48.855 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:48.852+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:48.956 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:48.952+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:48.963 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:48.963 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:48.964 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:48.964 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:48.964 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:48.973 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:31:49.057 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:49.053+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:49.155 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 148.05532882, 2023-12-06T18:31:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:31:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:31:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:31:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:31:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:31:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:31:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:31:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:31:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:31:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:31:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:31:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:31:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:31:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:31:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:49.159 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:49.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:49.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:49.159 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:31:49.159 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:49.159 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 88.037071518000005, 2023-12-06T18:31:49.159 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:31:49.159 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:49.159 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:49.159 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:49.159 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:49.159 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:49.159 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:31:49.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:49.159 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:49.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:49.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:49.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:49.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:49.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:49.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:49.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:49.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:49.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:49.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:49.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:49.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:49.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:31:49.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:31:49.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:49.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:49.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:49.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:31:49.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:31:49.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:49.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:49.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:49.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:31:49.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:31:49.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:49.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:49.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:49.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:31:49.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:31:49.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:49.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:49.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:49.162 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:31:49.162 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:31:49.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:49.162 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:49.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:49.162 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:31:49.162 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:31:49.162 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:49.162 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:49.162 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:49.162 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:49.162 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:49.162 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:49.164 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:31:49.164 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:49.160+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:49.265 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:49.261+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:49.304 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:49.305 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:49.305 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:49.305 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:49.315 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:31:49.366 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:49.362+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:49.449 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:31:49.467 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:49.463+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:49.485 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:49.486 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:49.486 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:49.486 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:49.486 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:49.495 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:31:49.579 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:49.576+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:49.668 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:49.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 208.17429071800001, 2023-12-06T18:31:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:31:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:49.669 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:31:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:49.670 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:49.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:31:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:49.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:31:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:31:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:49.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:31:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:31:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:31:49.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:31:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:31:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:31:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:49.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:31:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:31:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:49.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:31:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:31:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:49.677 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:49.678 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:49.678 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:49.665+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:49.680 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:31:49.770 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:49.766+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:49.848 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:49.848 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:49.848 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:49.848 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:49.858 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:31:49.870 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:49.866+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:49.972 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:49.968+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:49.986 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:49.986 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:49.986 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:49.986 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:49.986 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:49.996 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:31:50.080 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:50.077+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:50.174 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:50.171+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:50.204 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:50.204 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:50.204 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:50.204 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:50.204 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.204 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:31:50.204 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:50.204 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 293.58778613499999, 2023-12-06T18:31:50.204 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:31:50.204 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.204 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.204 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.205 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.205 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.205 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:50.205 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.205 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.205 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.205 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.205 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:31:50.205 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.205 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.205 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.205 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.205 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:31:50.206 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:31:50.206 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.206 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.206 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.206 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:31:50.206 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:31:50.206 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.206 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.206 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.206 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:31:50.206 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:31:50.206 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.207 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.207 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.207 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:31:50.207 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:31:50.207 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.207 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.207 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.207 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:31:50.207 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:31:50.207 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.207 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.207 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.207 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:31:50.208 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:31:50.208 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.208 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.208 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.208 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:31:50.208 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:31:50.208 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.208 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.208 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.208 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:31:50.208 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:31:50.208 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.208 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.209 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.209 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.209 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.209 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:31:50.209 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:50.209 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 292.56495912600002, 2023-12-06T18:31:50.209 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:31:50.209 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.209 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.209 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.209 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.209 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.210 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:50.210 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.210 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.210 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.210 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.210 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:31:50.210 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.210 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.210 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.210 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.210 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:31:50.210 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:31:50.210 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.210 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.210 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.211 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:31:50.211 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:31:50.211 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.211 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.211 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.211 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:31:50.211 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:31:50.211 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.211 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.211 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.211 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:31:50.211 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:31:50.211 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.211 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.211 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.212 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:31:50.212 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:31:50.212 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.212 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.212 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.212 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:31:50.212 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:31:50.212 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.212 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.212 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.212 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:31:50.213 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:31:50.213 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.213 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.213 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.213 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:31:50.213 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:31:50.213 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.213 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.213 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.213 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.213 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.214 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:31:50.214 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:50.214 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 292.552912488, 2023-12-06T18:31:50.214 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:31:50.214 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.214 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.214 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.214 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.214 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.214 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:50.214 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.214 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.214 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.214 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.215 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:31:50.215 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.215 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.215 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.215 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.215 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:31:50.215 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:31:50.215 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.215 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.215 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.215 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:31:50.215 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:31:50.215 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.216 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.216 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.216 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:31:50.216 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:31:50.216 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.216 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.216 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.216 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:31:50.216 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:31:50.216 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.216 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.216 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.216 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:31:50.216 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:31:50.216 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.217 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.217 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.217 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:31:50.217 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:31:50.217 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.217 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.217 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.217 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:31:50.217 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:31:50.217 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.217 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.217 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.217 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:31:50.217 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:31:50.217 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.218 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.218 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.218 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.218 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.218 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:31:50.218 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:50.218 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 292.53151188099997, 2023-12-06T18:31:50.218 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:31:50.218 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.218 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.219 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.219 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.219 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.219 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:50.219 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.219 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.219 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.219 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.219 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:31:50.219 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.219 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.219 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.219 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.220 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:31:50.220 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:31:50.220 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.220 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.220 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.220 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:31:50.220 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:31:50.220 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.220 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.220 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.220 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:31:50.220 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:31:50.220 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.220 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.221 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.221 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:31:50.221 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:31:50.221 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.221 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.221 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.221 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:31:50.221 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:31:50.221 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.222 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.222 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.222 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:31:50.222 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:31:50.222 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.222 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.222 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.222 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:31:50.222 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:31:50.222 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.222 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.222 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.222 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:31:50.222 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:31:50.222 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.223 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.223 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.223 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.223 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.223 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:31:50.223 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:50.223 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 292.530332558, 2023-12-06T18:31:50.223 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:31:50.223 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.223 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.223 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.223 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.223 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.223 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:50.223 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.224 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.224 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.224 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.224 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:31:50.224 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.224 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.224 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.224 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.224 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:31:50.224 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:31:50.224 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.224 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.224 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.224 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:31:50.224 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:31:50.224 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.225 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.225 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.225 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:31:50.225 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:31:50.225 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.225 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.225 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.225 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:31:50.225 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:31:50.225 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.225 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.225 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.225 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:31:50.225 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:31:50.226 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.226 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.226 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.226 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:31:50.226 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:31:50.226 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.226 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.226 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.226 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:31:50.226 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:31:50.226 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.226 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.227 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.227 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:31:50.227 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:31:50.227 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.227 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.227 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.227 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.227 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.227 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:31:50.227 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:50.227 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 292.52976057699999, 2023-12-06T18:31:50.227 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:31:50.227 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.227 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.227 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.227 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.228 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.228 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:50.228 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.228 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.228 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.228 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.228 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:31:50.228 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.228 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.228 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.228 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.228 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:31:50.228 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:31:50.228 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.228 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.228 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.228 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:31:50.229 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:31:50.229 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.229 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.229 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.229 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:31:50.229 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:31:50.229 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.229 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.229 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.229 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:31:50.229 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:31:50.229 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.229 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.229 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.229 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:31:50.230 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:31:50.230 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.230 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.230 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.230 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:31:50.230 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:31:50.230 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.230 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.230 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.230 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:31:50.230 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:31:50.230 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.230 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.230 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.231 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:31:50.231 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:31:50.231 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.231 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.231 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.231 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.231 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.231 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:31:50.231 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:50.231 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 292.08392510700003, 2023-12-06T18:31:50.231 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:31:50.231 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.231 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.231 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.231 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.232 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.232 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:50.232 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.232 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.232 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.232 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.232 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:31:50.232 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.232 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.232 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.232 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.232 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:31:50.232 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:31:50.232 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.232 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.233 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.233 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:31:50.233 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:31:50.233 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.233 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.233 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.233 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:31:50.233 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:31:50.233 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.233 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.233 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.233 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:31:50.233 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:31:50.233 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.234 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.234 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.234 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:31:50.234 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:31:50.234 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.234 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.234 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.234 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:31:50.234 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:31:50.234 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.234 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.235 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.235 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:31:50.235 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:31:50.235 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.235 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.235 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.235 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:31:50.235 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:31:50.235 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.235 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.235 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.235 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.235 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.235 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:31:50.235 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:50.236 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 292.08232525099999, 2023-12-06T18:31:50.236 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:31:50.236 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.236 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.236 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.236 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.236 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.236 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:50.236 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.236 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.236 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.236 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.236 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:31:50.236 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.236 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.237 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.237 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.237 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:31:50.237 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:31:50.237 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.237 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.237 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.237 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:31:50.237 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:31:50.237 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.237 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.238 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.238 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:31:50.238 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:31:50.238 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.238 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.238 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.238 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:31:50.238 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:31:50.238 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.238 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.238 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.238 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:31:50.238 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:31:50.238 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:31:50.239 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 291.70311963500001, 2023-12-06T18:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.240 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.241 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:50.241 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.241 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.241 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.241 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:31:50.241 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.241 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.241 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.241 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:31:50.241 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:31:50.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.242 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.242 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.242 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:31:50.242 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:31:50.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.242 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.242 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.242 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:31:50.242 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:31:50.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.242 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.242 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.242 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:31:50.243 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:31:50.243 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.243 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.243 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.243 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:31:50.243 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:31:50.243 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.243 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.243 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.243 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:31:50.243 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:31:50.243 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.243 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.243 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.243 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:31:50.244 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:31:50.244 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.244 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.244 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.244 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:31:50.244 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:31:50.244 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.244 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.244 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.244 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.244 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.244 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:31:50.244 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:50.244 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 291.46561745299999, 2023-12-06T18:31:50.244 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:31:50.245 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.245 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.245 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.245 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.245 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.245 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:50.245 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.245 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.245 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.245 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:31:50.245 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.245 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.246 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.246 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:31:50.246 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:31:50.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.246 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.246 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.246 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:31:50.246 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:31:50.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.246 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.246 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.246 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:31:50.246 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:31:50.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.247 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.247 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.247 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:31:50.247 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:31:50.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.247 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.247 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.247 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:31:50.247 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:31:50.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.247 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.247 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.247 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:31:50.248 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:31:50.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.248 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.248 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.248 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:31:50.248 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:31:50.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.248 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.248 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.248 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:31:50.248 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:31:50.248 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.248 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.248 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.249 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.249 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:31:50.249 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:50.249 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 291.464889454, 2023-12-06T18:31:50.249 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:31:50.249 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.249 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.249 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.249 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.249 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.249 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:50.249 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.249 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.249 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.250 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.250 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:31:50.250 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.250 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.250 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.250 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.250 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:31:50.250 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:31:50.250 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.250 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.250 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.250 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:31:50.250 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:31:50.250 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.250 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.251 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.251 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:31:50.251 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:31:50.251 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.251 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.251 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.251 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:31:50.251 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:31:50.251 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.251 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.251 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.251 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:31:50.251 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:31:50.251 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.252 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.252 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.252 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:31:50.252 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:31:50.252 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.252 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.252 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.252 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:31:50.252 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:31:50.252 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.252 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.252 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.252 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:31:50.253 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:31:50.253 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.253 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.253 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.253 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.253 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.253 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:31:50.253 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:50.253 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 291.459877044, 2023-12-06T18:31:50.253 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:31:50.253 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.253 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.253 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.254 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.254 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.254 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:50.254 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.254 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.254 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.254 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.254 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:31:50.254 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.254 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.254 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.254 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.255 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:31:50.255 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:31:50.255 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.255 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.255 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.255 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:31:50.255 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:31:50.255 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.255 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.255 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.255 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:31:50.255 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:31:50.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.256 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.256 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.256 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:31:50.256 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:31:50.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.256 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.256 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.256 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:31:50.256 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:31:50.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.256 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.256 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.257 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:31:50.257 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:31:50.257 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.257 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.257 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.257 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:31:50.257 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:31:50.257 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.257 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.257 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.257 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:31:50.257 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:31:50.258 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.258 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.258 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.258 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.258 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.258 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:31:50.258 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:50.258 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 291.45434996500001, 2023-12-06T18:31:50.258 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:31:50.258 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.258 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.259 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.259 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.259 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.259 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:50.259 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.259 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.259 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.259 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.259 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:31:50.259 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.259 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.259 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.260 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.260 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:31:50.260 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:31:50.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.260 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.260 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.260 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:31:50.260 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:31:50.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.260 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.260 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.260 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:31:50.260 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:31:50.261 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.261 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.261 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.261 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:31:50.261 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:31:50.261 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.261 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.261 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.261 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:31:50.261 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:31:50.261 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.261 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.261 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.261 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:31:50.262 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:31:50.262 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.262 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.262 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.262 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:31:50.262 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:31:50.262 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.262 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.262 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.262 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:31:50.262 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:31:50.263 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.263 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.263 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.263 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.263 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.263 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:31:50.263 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:50.263 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 291.45383912800003, 2023-12-06T18:31:50.263 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:31:50.264 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.264 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.264 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.264 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.264 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.264 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:50.264 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.264 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.264 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.264 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.264 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:31:50.264 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.265 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.265 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.265 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.265 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:31:50.265 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:31:50.265 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.265 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.265 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.265 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:31:50.265 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:31:50.265 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.265 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.265 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.265 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:31:50.265 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:31:50.265 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.266 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.266 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.266 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:31:50.266 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:31:50.266 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.266 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.266 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.266 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:31:50.266 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:31:50.266 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.266 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.266 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.266 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:31:50.266 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:31:50.267 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.267 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.267 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.267 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:31:50.267 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:31:50.267 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.267 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.267 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.267 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:31:50.267 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:31:50.267 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.267 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.267 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.267 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.267 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.268 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:31:50.268 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:50.268 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 291.45337986700002, 2023-12-06T18:31:50.268 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:31:50.268 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.268 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.268 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.268 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.268 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.268 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:50.268 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.268 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.268 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.268 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.268 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:31:50.269 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.269 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.269 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.269 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.269 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:31:50.269 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:31:50.269 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.269 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.269 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.269 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:31:50.269 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:31:50.269 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.269 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.269 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.269 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:31:50.269 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:31:50.270 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.270 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.270 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.270 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:31:50.270 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:31:50.270 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.270 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.270 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.270 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:31:50.271 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:31:50.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.271 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.271 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.271 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:31:50.271 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:31:50.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.271 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.271 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.271 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:31:50.271 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:31:50.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.271 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.271 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.271 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:31:50.272 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:31:50.272 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.272 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.272 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.272 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.272 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.272 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:31:50.272 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:50.272 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 284.87346406, 2023-12-06T18:31:50.272 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:31:50.272 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.272 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.272 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.272 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.272 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.272 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:50.273 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.273 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.273 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.273 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.273 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:31:50.273 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.273 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.273 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.273 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.273 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:31:50.273 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:31:50.273 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.273 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.273 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.273 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:31:50.274 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:31:50.274 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.274 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.274 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.274 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:31:50.274 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:31:50.274 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.274 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.274 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.274 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:31:50.274 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:31:50.274 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.274 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.275 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.275 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:31:50.275 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:31:50.275 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.275 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.275 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.275 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:31:50.275 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:31:50.275 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.275 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.276 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.276 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:31:50.276 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:31:50.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.276 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.276 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.277 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:31:50.277 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:31:50.277 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.277 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.277 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.277 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.278 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.278 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:31:50.278 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:50.278 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 284.86722625499999, 2023-12-06T18:31:50.278 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:31:50.278 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.279 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.279 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.279 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.279 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:50.279 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.279 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.280 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.280 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:31:50.280 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.281 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:31:50.281 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:31:50.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.281 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:31:50.281 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:31:50.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.282 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:31:50.282 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:31:50.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.282 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.282 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:31:50.283 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:31:50.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.283 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.283 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.283 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:31:50.283 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:31:50.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.283 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.284 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.284 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:31:50.284 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:31:50.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.284 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.284 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.284 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:31:50.285 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:31:50.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.285 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.285 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:31:50.285 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:31:50.285 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.286 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.286 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.286 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:31:50.286 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:50.286 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 284.86116317599999, 2023-12-06T18:31:50.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:31:50.287 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.287 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.287 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:50.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.288 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.288 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.288 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:31:50.288 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.288 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.288 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.289 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:31:50.289 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:31:50.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.289 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.289 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.289 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:31:50.289 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:31:50.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.290 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.290 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:31:50.290 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:31:50.290 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.290 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:31:50.291 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:31:50.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:31:50.291 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:31:50.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.292 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.292 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:31:50.292 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:31:50.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.292 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.292 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.292 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:31:50.292 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:31:50.293 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.293 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.293 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.293 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:31:50.293 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:31:50.293 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.293 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.293 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.294 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.294 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.294 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:31:50.294 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:50.294 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 284.72377609099999, 2023-12-06T18:31:50.294 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:31:50.294 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.294 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.295 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.295 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.295 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.295 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:50.295 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.295 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.295 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.296 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:31:50.296 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.296 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.296 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.296 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:31:50.296 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:31:50.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.296 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.296 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.296 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:31:50.297 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:31:50.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.297 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.297 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.297 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:31:50.297 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:31:50.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.297 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.298 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.298 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:31:50.298 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:31:50.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.298 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.298 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.299 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:31:50.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:31:50.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.299 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.299 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:31:50.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:31:50.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:31:50.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:31:50.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:31:50.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:31:50.300 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.300 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.300 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.300 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:31:50.300 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:50.301 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 284.71710536400002, 2023-12-06T18:31:50.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:31:50.301 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:50.301 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:50.301 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:50.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.301 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:50.301 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:50.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.301 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:50.301 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:31:50.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:50.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.302 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:50.302 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:31:50.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:31:50.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.302 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:50.302 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:31:50.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:31:50.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.302 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:50.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:31:50.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:31:50.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:50.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:31:50.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:31:50.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:50.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:31:50.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:31:50.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:31:50.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:31:50.304 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:31:50.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.304 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:31:50.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:31:50.304 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:31:50.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:50.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:50.304 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:50.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:31:50.304 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:31:50.304 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.304 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.304 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.304 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:50.304 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:50.305 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:50.306 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:31:50.307 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:50.303+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:50.408 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:50.404+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:50.474 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:50.475 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:50.475 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:50.475 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:50.484 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:31:50.528 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:50.524+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:50.609 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:50.605+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:50.611 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:50.611 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:50.611 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:50.612 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:50.612 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:50.621 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:31:50.709 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:50.706+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:50.790 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:50.790 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:50.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:50.790 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:31:50.790 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:50.799 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:31:50.810 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:50.806+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:50.911 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:50.907+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:50.966 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:50.967 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:50.967 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:50.967 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:50.976 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:31:51.020 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:51.016+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:51.101 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:51.101 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:51.102 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:51.102 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:51.102 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:51.110 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:31:51.151 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:51.148+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:51.213 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:51.210+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:51.283 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:31:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:31:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:31:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:31:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 89.621821128999997, 2023-12-06T18:31:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:31:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:31:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:31:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:31:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:31:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:51.284 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:31:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:31:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:31:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:31:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:31:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:31:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:31:51.285 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:31:51.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:31:51.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:51.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:51.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:31:51.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:31:51.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:31:51.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:51.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:51.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:31:51.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:31:51.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:31:51.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:51.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:51.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:31:51.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:31:51.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:31:51.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:51.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:51.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:31:51.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:31:51.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:31:51.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:31:51.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:31:51.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:31:51.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:31:51.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:31:51.287 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:51.287 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:51.287 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:51.288 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:31:51.288 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:31:51.288 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:51.292 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:31:51.338 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:51.334+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:51.415 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:51.411+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:51.437 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:51.437 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:51.437 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:51.437 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:51.446 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:31:51.531 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:51.528+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:51.616 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:51.613+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:51.622 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:51.622 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:51.622 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:51.622 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:51.622 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:51.631 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:31:51.717 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:51.714+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:51.818 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:51.815+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:51.831 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:51.831 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:51.831 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:51.831 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:31:51.831 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:51.840 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:31:51.919 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:51.915+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:51.985 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:51.985 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:51.985 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:51.985 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:51.994 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:31:52.020 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:52.017+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:52.121 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:52.117+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:52.174 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:52.174 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:52.174 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:52.174 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:52.174 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:52.184 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:31:52.185 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:31:52.185 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:52.185 INFO:teuthology.orchestra.run.smithi061.stdout: "epoch": 77, 2023-12-06T18:31:52.185 INFO:teuthology.orchestra.run.smithi061.stdout: "fsid": "497e3283-cc16-40ff-a72c-a7487474fb4b", 2023-12-06T18:31:52.185 INFO:teuthology.orchestra.run.smithi061.stdout: "created": "2023-12-06T18:26:07.122862+0000", 2023-12-06T18:31:52.185 INFO:teuthology.orchestra.run.smithi061.stdout: "modified": "2023-12-06T18:31:36.306793+0000", 2023-12-06T18:31:52.185 INFO:teuthology.orchestra.run.smithi061.stdout: "last_up_change": "2023-12-06T18:30:04.956811+0000", 2023-12-06T18:31:52.186 INFO:teuthology.orchestra.run.smithi061.stdout: "last_in_change": "2023-12-06T18:28:45.621304+0000", 2023-12-06T18:31:52.186 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-06T18:31:52.186 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_num": 5801984, 2023-12-06T18:31:52.186 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_set": [ 2023-12-06T18:31:52.186 INFO:teuthology.orchestra.run.smithi061.stdout: "noscrub", 2023-12-06T18:31:52.186 INFO:teuthology.orchestra.run.smithi061.stdout: "pglog_hardlimit", 2023-12-06T18:31:52.186 INFO:teuthology.orchestra.run.smithi061.stdout: "purged_snapdirs", 2023-12-06T18:31:52.186 INFO:teuthology.orchestra.run.smithi061.stdout: "recovery_deletes", 2023-12-06T18:31:52.186 INFO:teuthology.orchestra.run.smithi061.stdout: "sortbitwise" 2023-12-06T18:31:52.186 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:31:52.186 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_version": 8, 2023-12-06T18:31:52.186 INFO:teuthology.orchestra.run.smithi061.stdout: "full_ratio": 0.94999998807907104, 2023-12-06T18:31:52.186 INFO:teuthology.orchestra.run.smithi061.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-12-06T18:31:52.187 INFO:teuthology.orchestra.run.smithi061.stdout: "nearfull_ratio": 0.85000002384185791, 2023-12-06T18:31:52.187 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_snapshot": "", 2023-12-06T18:31:52.187 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_max": 2, 2023-12-06T18:31:52.187 INFO:teuthology.orchestra.run.smithi061.stdout: "max_osd": 8, 2023-12-06T18:31:52.187 INFO:teuthology.orchestra.run.smithi061.stdout: "require_min_compat_client": "luminous", 2023-12-06T18:31:52.187 INFO:teuthology.orchestra.run.smithi061.stdout: "min_compat_client": "luminous", 2023-12-06T18:31:52.187 INFO:teuthology.orchestra.run.smithi061.stdout: "require_osd_release": "reef", 2023-12-06T18:31:52.187 INFO:teuthology.orchestra.run.smithi061.stdout: "allow_crimson": false, 2023-12-06T18:31:52.187 INFO:teuthology.orchestra.run.smithi061.stdout: "pools": [ 2023-12-06T18:31:52.187 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.187 INFO:teuthology.orchestra.run.smithi061.stdout: "pool": 1, 2023-12-06T18:31:52.187 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_name": ".mgr", 2023-12-06T18:31:52.187 INFO:teuthology.orchestra.run.smithi061.stdout: "create_time": "2023-12-06T18:26:52.648377+0000", 2023-12-06T18:31:52.187 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-12-06T18:31:52.188 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_names": "hashpspool", 2023-12-06T18:31:52.188 INFO:teuthology.orchestra.run.smithi061.stdout: "type": 1, 2023-12-06T18:31:52.188 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 2, 2023-12-06T18:31:52.188 INFO:teuthology.orchestra.run.smithi061.stdout: "min_size": 1, 2023-12-06T18:31:52.188 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_rule": 0, 2023-12-06T18:31:52.188 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_count": 0, 2023-12-06T18:31:52.188 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_target": 0, 2023-12-06T18:31:52.188 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_barrier": 0, 2023-12-06T18:31:52.188 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-06T18:31:52.188 INFO:teuthology.orchestra.run.smithi061.stdout: "object_hash": 2, 2023-12-06T18:31:52.188 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscale_mode": "off", 2023-12-06T18:31:52.188 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num": 1, 2023-12-06T18:31:52.188 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num": 1, 2023-12-06T18:31:52.188 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num_target": 1, 2023-12-06T18:31:52.188 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_target": 1, 2023-12-06T18:31:52.189 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_pending": 1, 2023-12-06T18:31:52.189 INFO:teuthology.orchestra.run.smithi061.stdout: "last_pg_merge_meta": { 2023-12-06T18:31:52.189 INFO:teuthology.orchestra.run.smithi061.stdout: "source_pgid": "0.0", 2023-12-06T18:31:52.189 INFO:teuthology.orchestra.run.smithi061.stdout: "ready_epoch": 0, 2023-12-06T18:31:52.189 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_started": 0, 2023-12-06T18:31:52.189 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_clean": 0, 2023-12-06T18:31:52.189 INFO:teuthology.orchestra.run.smithi061.stdout: "source_version": "0'0", 2023-12-06T18:31:52.189 INFO:teuthology.orchestra.run.smithi061.stdout: "target_version": "0'0" 2023-12-06T18:31:52.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.189 INFO:teuthology.orchestra.run.smithi061.stdout: "last_change": "19", 2023-12-06T18:31:52.189 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend": "0", 2023-12-06T18:31:52.189 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-06T18:31:52.189 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_preluminous": "0", 2023-12-06T18:31:52.189 INFO:teuthology.orchestra.run.smithi061.stdout: "auid": 0, 2023-12-06T18:31:52.189 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_mode": "selfmanaged", 2023-12-06T18:31:52.189 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_seq": 0, 2023-12-06T18:31:52.190 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_epoch": 0, 2023-12-06T18:31:52.190 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_snaps": [], 2023-12-06T18:31:52.190 INFO:teuthology.orchestra.run.smithi061.stdout: "removed_snaps": "[]", 2023-12-06T18:31:52.190 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_bytes": 0, 2023-12-06T18:31:52.190 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_objects": 0, 2023-12-06T18:31:52.190 INFO:teuthology.orchestra.run.smithi061.stdout: "tiers": [], 2023-12-06T18:31:52.190 INFO:teuthology.orchestra.run.smithi061.stdout: "tier_of": -1, 2023-12-06T18:31:52.190 INFO:teuthology.orchestra.run.smithi061.stdout: "read_tier": -1, 2023-12-06T18:31:52.190 INFO:teuthology.orchestra.run.smithi061.stdout: "write_tier": -1, 2023-12-06T18:31:52.190 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_mode": "none", 2023-12-06T18:31:52.190 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_bytes": 0, 2023-12-06T18:31:52.190 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_objects": 0, 2023-12-06T18:31:52.190 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-06T18:31:52.191 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-06T18:31:52.191 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-06T18:31:52.191 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_flush_age": 0, 2023-12-06T18:31:52.191 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_evict_age": 0, 2023-12-06T18:31:52.191 INFO:teuthology.orchestra.run.smithi061.stdout: "erasure_code_profile": "", 2023-12-06T18:31:52.191 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_params": { 2023-12-06T18:31:52.191 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "none" 2023-12-06T18:31:52.191 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.191 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_period": 0, 2023-12-06T18:31:52.191 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_count": 0, 2023-12-06T18:31:52.191 INFO:teuthology.orchestra.run.smithi061.stdout: "use_gmt_hitset": true, 2023-12-06T18:31:52.191 INFO:teuthology.orchestra.run.smithi061.stdout: "min_read_recency_for_promote": 0, 2023-12-06T18:31:52.191 INFO:teuthology.orchestra.run.smithi061.stdout: "min_write_recency_for_promote": 0, 2023-12-06T18:31:52.191 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_grade_decay_rate": 0, 2023-12-06T18:31:52.192 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_search_last_n": 0, 2023-12-06T18:31:52.192 INFO:teuthology.orchestra.run.smithi061.stdout: "grade_table": [], 2023-12-06T18:31:52.192 INFO:teuthology.orchestra.run.smithi061.stdout: "stripe_width": 0, 2023-12-06T18:31:52.192 INFO:teuthology.orchestra.run.smithi061.stdout: "expected_num_objects": 0, 2023-12-06T18:31:52.192 INFO:teuthology.orchestra.run.smithi061.stdout: "fast_read": false, 2023-12-06T18:31:52.192 INFO:teuthology.orchestra.run.smithi061.stdout: "options": { 2023-12-06T18:31:52.192 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_max": 32, 2023-12-06T18:31:52.192 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_min": 1 2023-12-06T18:31:52.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.192 INFO:teuthology.orchestra.run.smithi061.stdout: "application_metadata": { 2023-12-06T18:31:52.192 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr": {} 2023-12-06T18:31:52.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.192 INFO:teuthology.orchestra.run.smithi061.stdout: "read_balance": { 2023-12-06T18:31:52.192 INFO:teuthology.orchestra.run.smithi061.stdout: "score_acting": 6.9000000953674316, 2023-12-06T18:31:52.193 INFO:teuthology.orchestra.run.smithi061.stdout: "score_stable": 6.9000000953674316, 2023-12-06T18:31:52.193 INFO:teuthology.orchestra.run.smithi061.stdout: "optimal_score": 0.28999999165534973, 2023-12-06T18:31:52.193 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_acting": 2, 2023-12-06T18:31:52.193 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_stable": 2, 2023-12-06T18:31:52.193 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity_weighted": 0.82932090759277344, 2023-12-06T18:31:52.193 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity": 0.94999998807907104, 2023-12-06T18:31:52.193 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2023-12-06T18:31:52.193 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.193 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.193 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.193 INFO:teuthology.orchestra.run.smithi061.stdout: "pool": 2, 2023-12-06T18:31:52.193 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_name": "rbd", 2023-12-06T18:31:52.193 INFO:teuthology.orchestra.run.smithi061.stdout: "create_time": "2023-12-06T18:27:01.712581+0000", 2023-12-06T18:31:52.193 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-12-06T18:31:52.193 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_names": "hashpspool", 2023-12-06T18:31:52.193 INFO:teuthology.orchestra.run.smithi061.stdout: "type": 1, 2023-12-06T18:31:52.194 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 2, 2023-12-06T18:31:52.194 INFO:teuthology.orchestra.run.smithi061.stdout: "min_size": 1, 2023-12-06T18:31:52.194 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_rule": 0, 2023-12-06T18:31:52.194 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_count": 0, 2023-12-06T18:31:52.194 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_target": 0, 2023-12-06T18:31:52.194 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_barrier": 0, 2023-12-06T18:31:52.194 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-06T18:31:52.194 INFO:teuthology.orchestra.run.smithi061.stdout: "object_hash": 2, 2023-12-06T18:31:52.194 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscale_mode": "off", 2023-12-06T18:31:52.194 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num": 8, 2023-12-06T18:31:52.194 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num": 8, 2023-12-06T18:31:52.194 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num_target": 8, 2023-12-06T18:31:52.194 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_target": 8, 2023-12-06T18:31:52.194 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_pending": 8, 2023-12-06T18:31:52.194 INFO:teuthology.orchestra.run.smithi061.stdout: "last_pg_merge_meta": { 2023-12-06T18:31:52.195 INFO:teuthology.orchestra.run.smithi061.stdout: "source_pgid": "0.0", 2023-12-06T18:31:52.195 INFO:teuthology.orchestra.run.smithi061.stdout: "ready_epoch": 0, 2023-12-06T18:31:52.195 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_started": 0, 2023-12-06T18:31:52.195 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_clean": 0, 2023-12-06T18:31:52.195 INFO:teuthology.orchestra.run.smithi061.stdout: "source_version": "0'0", 2023-12-06T18:31:52.195 INFO:teuthology.orchestra.run.smithi061.stdout: "target_version": "0'0" 2023-12-06T18:31:52.195 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.195 INFO:teuthology.orchestra.run.smithi061.stdout: "last_change": "24", 2023-12-06T18:31:52.195 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend": "0", 2023-12-06T18:31:52.195 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-06T18:31:52.195 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_preluminous": "0", 2023-12-06T18:31:52.195 INFO:teuthology.orchestra.run.smithi061.stdout: "auid": 0, 2023-12-06T18:31:52.195 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_mode": "selfmanaged", 2023-12-06T18:31:52.195 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_seq": 0, 2023-12-06T18:31:52.195 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_epoch": 0, 2023-12-06T18:31:52.195 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_snaps": [], 2023-12-06T18:31:52.196 INFO:teuthology.orchestra.run.smithi061.stdout: "removed_snaps": "[]", 2023-12-06T18:31:52.196 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_bytes": 0, 2023-12-06T18:31:52.196 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_objects": 0, 2023-12-06T18:31:52.196 INFO:teuthology.orchestra.run.smithi061.stdout: "tiers": [], 2023-12-06T18:31:52.196 INFO:teuthology.orchestra.run.smithi061.stdout: "tier_of": -1, 2023-12-06T18:31:52.196 INFO:teuthology.orchestra.run.smithi061.stdout: "read_tier": -1, 2023-12-06T18:31:52.196 INFO:teuthology.orchestra.run.smithi061.stdout: "write_tier": -1, 2023-12-06T18:31:52.196 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_mode": "none", 2023-12-06T18:31:52.196 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_bytes": 0, 2023-12-06T18:31:52.196 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_objects": 0, 2023-12-06T18:31:52.196 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-06T18:31:52.196 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-06T18:31:52.196 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-06T18:31:52.196 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_flush_age": 0, 2023-12-06T18:31:52.197 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_evict_age": 0, 2023-12-06T18:31:52.197 INFO:teuthology.orchestra.run.smithi061.stdout: "erasure_code_profile": "", 2023-12-06T18:31:52.197 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_params": { 2023-12-06T18:31:52.197 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "none" 2023-12-06T18:31:52.197 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.197 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_period": 0, 2023-12-06T18:31:52.197 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_count": 0, 2023-12-06T18:31:52.197 INFO:teuthology.orchestra.run.smithi061.stdout: "use_gmt_hitset": true, 2023-12-06T18:31:52.197 INFO:teuthology.orchestra.run.smithi061.stdout: "min_read_recency_for_promote": 0, 2023-12-06T18:31:52.197 INFO:teuthology.orchestra.run.smithi061.stdout: "min_write_recency_for_promote": 0, 2023-12-06T18:31:52.197 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_grade_decay_rate": 0, 2023-12-06T18:31:52.197 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_search_last_n": 0, 2023-12-06T18:31:52.197 INFO:teuthology.orchestra.run.smithi061.stdout: "grade_table": [], 2023-12-06T18:31:52.197 INFO:teuthology.orchestra.run.smithi061.stdout: "stripe_width": 0, 2023-12-06T18:31:52.197 INFO:teuthology.orchestra.run.smithi061.stdout: "expected_num_objects": 0, 2023-12-06T18:31:52.198 INFO:teuthology.orchestra.run.smithi061.stdout: "fast_read": false, 2023-12-06T18:31:52.198 INFO:teuthology.orchestra.run.smithi061.stdout: "options": {}, 2023-12-06T18:31:52.198 INFO:teuthology.orchestra.run.smithi061.stdout: "application_metadata": { 2023-12-06T18:31:52.198 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd": {} 2023-12-06T18:31:52.198 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.198 INFO:teuthology.orchestra.run.smithi061.stdout: "read_balance": { 2023-12-06T18:31:52.198 INFO:teuthology.orchestra.run.smithi061.stdout: "score_acting": 3.5, 2023-12-06T18:31:52.198 INFO:teuthology.orchestra.run.smithi061.stdout: "score_stable": 3.5, 2023-12-06T18:31:52.198 INFO:teuthology.orchestra.run.smithi061.stdout: "optimal_score": 1, 2023-12-06T18:31:52.198 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_acting": 3.5, 2023-12-06T18:31:52.198 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_stable": 3.5, 2023-12-06T18:31:52.198 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity_weighted": 0.82932090759277344, 2023-12-06T18:31:52.198 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity": 0.94999998807907104, 2023-12-06T18:31:52.198 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity_weighted": 0.82999998331069946 2023-12-06T18:31:52.198 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.199 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.199 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:31:52.199 INFO:teuthology.orchestra.run.smithi061.stdout: "osds": [ 2023-12-06T18:31:52.199 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.199 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 0, 2023-12-06T18:31:52.199 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "5e6e17b9-e72e-4e34-a531-0830d659bfe0", 2023-12-06T18:31:52.199 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:31:52.199 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:31:52.199 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:31:52.199 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:31:52.199 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:31:52.199 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:31:52.199 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 16, 2023-12-06T18:31:52.199 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 0, 2023-12-06T18:31:52.200 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:31:52.200 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:31:52.200 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:31:52.200 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.200 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.200 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.200 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6805", 2023-12-06T18:31:52.200 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:31:52.200 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.200 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.200 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.200 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:31:52.200 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.200 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.200 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.200 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6806", 2023-12-06T18:31:52.201 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:31:52.201 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.201 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.201 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.201 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:31:52.201 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.201 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.201 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.201 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6808", 2023-12-06T18:31:52.201 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:31:52.201 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.201 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.201 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.201 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:31:52.202 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.202 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.202 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.202 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6807", 2023-12-06T18:31:52.202 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:31:52.202 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.202 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.202 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.202 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6805/281956997", 2023-12-06T18:31:52.202 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6806/281956997", 2023-12-06T18:31:52.202 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6808/281956997", 2023-12-06T18:31:52.202 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6807/281956997", 2023-12-06T18:31:52.202 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:31:52.202 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:31:52.202 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:31:52.203 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.203 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.203 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.203 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 1, 2023-12-06T18:31:52.203 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "b78c49b0-360d-4351-b44e-65ccaaeb768f", 2023-12-06T18:31:52.203 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:31:52.203 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:31:52.203 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:31:52.203 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:31:52.203 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:31:52.203 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:31:52.203 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 16, 2023-12-06T18:31:52.203 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 44, 2023-12-06T18:31:52.203 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:31:52.204 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:31:52.204 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:31:52.204 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.204 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.204 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.204 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6809", 2023-12-06T18:31:52.204 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:31:52.204 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.204 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.204 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.204 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:31:52.204 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.204 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.204 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.204 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6810", 2023-12-06T18:31:52.205 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:31:52.205 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.205 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.205 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.205 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:31:52.205 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.205 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.205 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.205 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6812", 2023-12-06T18:31:52.205 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:31:52.205 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.205 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.205 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.206 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:31:52.206 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.206 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.206 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.206 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6811", 2023-12-06T18:31:52.206 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:31:52.206 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.206 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.206 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.206 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6809/3306625622", 2023-12-06T18:31:52.207 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6810/3306625622", 2023-12-06T18:31:52.207 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6812/3306625622", 2023-12-06T18:31:52.207 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6811/3306625622", 2023-12-06T18:31:52.207 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:31:52.207 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:31:52.207 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:31:52.207 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.207 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.207 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.207 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 2, 2023-12-06T18:31:52.207 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "a2af3151-a4da-4c34-8fb6-78abf035785b", 2023-12-06T18:31:52.207 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:31:52.207 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:31:52.207 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:31:52.207 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 0.6345672607421875, 2023-12-06T18:31:52.207 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:31:52.208 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:31:52.208 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 16, 2023-12-06T18:31:52.208 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 35, 2023-12-06T18:31:52.208 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:31:52.208 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:31:52.208 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:31:52.208 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.208 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.208 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.208 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6801", 2023-12-06T18:31:52.208 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:31:52.208 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.208 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.209 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.209 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:31:52.209 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.209 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.209 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.209 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6802", 2023-12-06T18:31:52.209 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:31:52.209 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.209 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.209 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.209 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:31:52.209 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.209 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.209 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.209 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6804", 2023-12-06T18:31:52.210 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:31:52.210 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.210 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.210 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:31:52.210 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.210 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.210 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.210 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6803", 2023-12-06T18:31:52.210 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:31:52.210 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.210 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.210 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.210 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6801/3589270120", 2023-12-06T18:31:52.210 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6802/3589270120", 2023-12-06T18:31:52.211 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6804/3589270120", 2023-12-06T18:31:52.211 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6803/3589270120", 2023-12-06T18:31:52.211 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:31:52.211 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:31:52.211 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:31:52.211 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.211 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.211 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.211 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 3, 2023-12-06T18:31:52.211 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "1c754ed6-c535-454f-b35b-00f62cd6ce5a", 2023-12-06T18:31:52.211 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:31:52.211 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:31:52.211 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:31:52.211 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:31:52.211 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:31:52.212 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:31:52.212 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 18, 2023-12-06T18:31:52.212 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 44, 2023-12-06T18:31:52.212 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:31:52.212 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:31:52.212 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:31:52.212 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.212 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.212 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.212 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6813", 2023-12-06T18:31:52.212 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:31:52.212 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.212 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.212 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.212 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:31:52.213 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.213 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.213 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.213 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6814", 2023-12-06T18:31:52.213 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:31:52.213 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.213 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.213 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.213 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:31:52.213 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.213 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.213 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.213 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6816", 2023-12-06T18:31:52.213 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:31:52.213 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.214 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.214 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:31:52.214 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.214 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.214 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.214 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6815", 2023-12-06T18:31:52.214 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:31:52.214 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.214 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.214 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.214 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6813/2849920842", 2023-12-06T18:31:52.214 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6814/2849920842", 2023-12-06T18:31:52.214 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6816/2849920842", 2023-12-06T18:31:52.214 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6815/2849920842", 2023-12-06T18:31:52.215 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:31:52.215 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:31:52.215 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:31:52.215 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.215 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.215 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.215 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 4, 2023-12-06T18:31:52.215 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "766572af-b9ca-4f38-927d-36f26101ba51", 2023-12-06T18:31:52.215 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:31:52.215 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:31:52.215 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:31:52.215 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:31:52.215 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:31:52.215 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:31:52.215 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 15, 2023-12-06T18:31:52.215 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 38, 2023-12-06T18:31:52.216 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:31:52.216 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:31:52.216 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:31:52.216 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.216 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.216 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.216 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6800", 2023-12-06T18:31:52.216 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:31:52.216 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.216 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.216 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.216 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:31:52.216 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.216 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.216 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.217 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6801", 2023-12-06T18:31:52.217 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:31:52.217 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.217 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.217 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.217 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:31:52.217 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.217 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.217 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.217 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6803", 2023-12-06T18:31:52.217 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:31:52.217 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.217 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.217 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.217 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:31:52.218 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.218 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.218 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.218 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6802", 2023-12-06T18:31:52.218 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:31:52.218 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.218 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.218 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.218 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6800/2778770069", 2023-12-06T18:31:52.218 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6801/2778770069", 2023-12-06T18:31:52.221 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6803/2778770069", 2023-12-06T18:31:52.221 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6802/2778770069", 2023-12-06T18:31:52.221 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:31:52.221 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:31:52.221 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:31:52.221 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.221 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.221 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.221 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 5, 2023-12-06T18:31:52.221 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "7a566b60-1543-41d7-894c-e2fbb63e48dc", 2023-12-06T18:31:52.221 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:31:52.221 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:31:52.222 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:31:52.222 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:31:52.222 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:31:52.222 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:31:52.222 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 15, 2023-12-06T18:31:52.222 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 61, 2023-12-06T18:31:52.222 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:31:52.222 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:31:52.222 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:31:52.222 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.222 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.222 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.222 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6809", 2023-12-06T18:31:52.223 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:31:52.223 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.223 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.223 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.223 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:31:52.223 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.223 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.223 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.223 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6810", 2023-12-06T18:31:52.223 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:31:52.223 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.223 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.223 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.223 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:31:52.224 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.224 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.224 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.224 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6814", 2023-12-06T18:31:52.224 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:31:52.224 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.224 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.224 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.224 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:31:52.224 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.224 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.224 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.224 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6812", 2023-12-06T18:31:52.224 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:31:52.224 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.225 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.225 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.225 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6809/263671335", 2023-12-06T18:31:52.225 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6810/263671335", 2023-12-06T18:31:52.225 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6814/263671335", 2023-12-06T18:31:52.225 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6812/263671335", 2023-12-06T18:31:52.225 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:31:52.225 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:31:52.225 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:31:52.225 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.225 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.225 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.225 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 6, 2023-12-06T18:31:52.225 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "28cc2358-4ac4-40b2-91a4-20edfe625a73", 2023-12-06T18:31:52.225 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:31:52.226 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:31:52.226 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:31:52.226 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 0, 2023-12-06T18:31:52.226 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:31:52.226 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:31:52.226 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 14, 2023-12-06T18:31:52.226 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 25, 2023-12-06T18:31:52.226 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:31:52.226 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:31:52.226 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:31:52.226 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.226 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.226 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.226 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6808", 2023-12-06T18:31:52.227 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:31:52.227 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.227 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.227 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.227 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:31:52.227 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.227 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.227 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.227 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6811", 2023-12-06T18:31:52.227 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:31:52.227 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.227 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.227 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.227 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:31:52.227 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.227 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.227 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.228 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6815", 2023-12-06T18:31:52.228 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:31:52.228 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.228 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.228 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.228 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:31:52.228 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.228 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.228 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.228 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6813", 2023-12-06T18:31:52.228 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:31:52.228 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.228 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.228 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.228 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6808/4245639079", 2023-12-06T18:31:52.229 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6811/4245639079", 2023-12-06T18:31:52.229 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6815/4245639079", 2023-12-06T18:31:52.229 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6813/4245639079", 2023-12-06T18:31:52.229 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:31:52.229 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:31:52.229 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:31:52.229 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.229 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.229 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.229 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 7, 2023-12-06T18:31:52.229 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "8f79279b-d2d0-438e-b3c2-88df1241ff30", 2023-12-06T18:31:52.229 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:31:52.229 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:31:52.229 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:31:52.229 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:31:52.229 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 14, 2023-12-06T18:31:52.230 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 37, 2023-12-06T18:31:52.230 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 61, 2023-12-06T18:31:52.230 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 62, 2023-12-06T18:31:52.230 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 38, 2023-12-06T18:31:52.230 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:31:52.230 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:31:52.230 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.230 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.230 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.230 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6804", 2023-12-06T18:31:52.230 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3075390540 2023-12-06T18:31:52.230 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.230 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.230 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.230 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:31:52.231 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.231 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.231 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.231 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6805", 2023-12-06T18:31:52.231 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3075390540 2023-12-06T18:31:52.231 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.231 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.231 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.231 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:31:52.231 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.231 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.231 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.231 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6807", 2023-12-06T18:31:52.231 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3075390540 2023-12-06T18:31:52.232 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.232 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.232 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.232 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:31:52.232 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:31:52.232 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.232 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:31:52.232 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6806", 2023-12-06T18:31:52.232 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3075390540 2023-12-06T18:31:52.232 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.232 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.232 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.232 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6804/3075390540", 2023-12-06T18:31:52.232 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6805/3075390540", 2023-12-06T18:31:52.232 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6807/3075390540", 2023-12-06T18:31:52.232 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6806/3075390540", 2023-12-06T18:31:52.233 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:31:52.233 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:31:52.233 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:31:52.233 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.233 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.233 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:31:52.233 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_xinfo": [ 2023-12-06T18:31:52.233 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.233 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 0, 2023-12-06T18:31:52.233 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:31:52.233 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:31:52.233 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:31:52.233 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:31:52.233 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:31:52.233 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:52.171904+0000", 2023-12-06T18:31:52.233 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:31:52.233 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.234 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.234 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 1, 2023-12-06T18:31:52.234 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:31:52.234 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:31:52.234 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:31:52.234 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:31:52.234 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:31:52.234 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:52.428840+0000", 2023-12-06T18:31:52.234 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:31:52.234 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.234 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.234 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 2, 2023-12-06T18:31:52.234 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:31:52.235 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:31:52.235 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:31:52.235 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:31:52.235 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:31:52.235 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:51.681954+0000", 2023-12-06T18:31:52.235 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:31:52.235 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.235 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.235 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 3, 2023-12-06T18:31:52.235 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:31:52.235 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:31:52.235 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:31:52.235 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:31:52.235 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:31:52.235 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:53.687417+0000", 2023-12-06T18:31:52.235 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:31:52.236 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.236 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.236 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 4, 2023-12-06T18:31:52.236 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:31:52.236 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:31:52.236 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:31:52.236 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:31:52.236 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:31:52.236 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:50.158682+0000", 2023-12-06T18:31:52.236 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:31:52.236 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.236 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.236 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 5, 2023-12-06T18:31:52.236 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:31:52.236 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:31:52.236 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:31:52.237 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:31:52.237 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:31:52.237 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:50.443807+0000", 2023-12-06T18:31:52.237 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:31:52.237 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.237 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.237 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 6, 2023-12-06T18:31:52.237 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:31:52.237 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:31:52.237 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:31:52.237 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:31:52.237 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:31:52.237 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:49.750276+0000", 2023-12-06T18:31:52.237 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:31:52.237 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.237 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.238 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 7, 2023-12-06T18:31:52.238 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:31:52.238 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:31:52.238 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:31:52.238 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:31:52.238 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:31:52.238 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:49.770987+0000", 2023-12-06T18:31:52.238 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 38 2023-12-06T18:31:52.238 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.238 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:31:52.238 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_upmap": [ 2023-12-06T18:31:52.238 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:52.238 INFO:teuthology.orchestra.run.smithi061.stdout: "pgid": "2.7", 2023-12-06T18:31:52.238 INFO:teuthology.orchestra.run.smithi061.stdout: "osds": [ 2023-12-06T18:31:52.238 INFO:teuthology.orchestra.run.smithi061.stdout: 5, 2023-12-06T18:31:52.239 INFO:teuthology.orchestra.run.smithi061.stdout: 3 2023-12-06T18:31:52.239 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:52.239 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.239 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:31:52.239 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_upmap_items": [], 2023-12-06T18:31:52.239 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_upmap_primaries": [], 2023-12-06T18:31:52.239 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_temp": [], 2023-12-06T18:31:52.239 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_temp": [], 2023-12-06T18:31:52.239 INFO:teuthology.orchestra.run.smithi061.stdout: "blocklist": {}, 2023-12-06T18:31:52.239 INFO:teuthology.orchestra.run.smithi061.stdout: "range_blocklist": {}, 2023-12-06T18:31:52.239 INFO:teuthology.orchestra.run.smithi061.stdout: "erasure_code_profiles": { 2023-12-06T18:31:52.239 INFO:teuthology.orchestra.run.smithi061.stdout: "default": { 2023-12-06T18:31:52.239 INFO:teuthology.orchestra.run.smithi061.stdout: "crush-failure-domain": "osd", 2023-12-06T18:31:52.239 INFO:teuthology.orchestra.run.smithi061.stdout: "k": "2", 2023-12-06T18:31:52.239 INFO:teuthology.orchestra.run.smithi061.stdout: "m": "1", 2023-12-06T18:31:52.240 INFO:teuthology.orchestra.run.smithi061.stdout: "plugin": "jerasure", 2023-12-06T18:31:52.240 INFO:teuthology.orchestra.run.smithi061.stdout: "technique": "reed_sol_van" 2023-12-06T18:31:52.240 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.240 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:52.240 INFO:teuthology.orchestra.run.smithi061.stdout: "removed_snaps_queue": [], 2023-12-06T18:31:52.240 INFO:teuthology.orchestra.run.smithi061.stdout: "new_removed_snaps": [], 2023-12-06T18:31:52.240 INFO:teuthology.orchestra.run.smithi061.stdout: "new_purged_snaps": [], 2023-12-06T18:31:52.240 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_node_flags": {}, 2023-12-06T18:31:52.240 INFO:teuthology.orchestra.run.smithi061.stdout: "device_class_flags": {}, 2023-12-06T18:31:52.240 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode": { 2023-12-06T18:31:52.240 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode_enabled": false, 2023-12-06T18:31:52.240 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_bucket_count": 0, 2023-12-06T18:31:52.240 INFO:teuthology.orchestra.run.smithi061.stdout: "degraded_stretch_mode": 0, 2023-12-06T18:31:52.240 INFO:teuthology.orchestra.run.smithi061.stdout: "recovering_stretch_mode": 0, 2023-12-06T18:31:52.240 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode_bucket": 0 2023-12-06T18:31:52.241 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:52.241 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:52.243 INFO:tasks.thrashosds.thrasher:j is {'epoch': 77, 'fsid': '497e3283-cc16-40ff-a72c-a7487474fb4b', 'created': '2023-12-06T18:26:07.122862+0000', 'modified': '2023-12-06T18:31:36.306793+0000', 'last_up_change': '2023-12-06T18:30:04.956811+0000', 'last_in_change': '2023-12-06T18:28:45.621304+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 8, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-06T18:26:52.648377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8293209075927734, 'average_primary_affinity': 0.949999988079071, 'average_primary_affinity_weighted': 0.8299999833106995}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-12-06T18:27:01.712581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 3.5, 'score_stable': 3.5, 'optimal_score': 1, 'raw_score_acting': 3.5, 'raw_score_stable': 3.5, 'primary_affinity_weighted': 0.8293209075927734, 'average_primary_affinity': 0.949999988079071, 'average_primary_affinity_weighted': 0.8299999833106995}}], 'osds': [{'osd': 0, 'uuid': '5e6e17b9-e72e-4e34-a531-0830d659bfe0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6805', 'nonce': 281956997}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6806', 'nonce': 281956997}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6808', 'nonce': 281956997}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6807', 'nonce': 281956997}]}, 'public_addr': '172.21.15.61:6805/281956997', 'cluster_addr': '172.21.15.61:6806/281956997', 'heartbeat_back_addr': '172.21.15.61:6808/281956997', 'heartbeat_front_addr': '172.21.15.61:6807/281956997', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b78c49b0-360d-4351-b44e-65ccaaeb768f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6809', 'nonce': 3306625622}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6810', 'nonce': 3306625622}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6812', 'nonce': 3306625622}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6811', 'nonce': 3306625622}]}, 'public_addr': '172.21.15.61:6809/3306625622', 'cluster_addr': '172.21.15.61:6810/3306625622', 'heartbeat_back_addr': '172.21.15.61:6812/3306625622', 'heartbeat_front_addr': '172.21.15.61:6811/3306625622', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'a2af3151-a4da-4c34-8fb6-78abf035785b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.6345672607421875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 35, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6801', 'nonce': 3589270120}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6802', 'nonce': 3589270120}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6804', 'nonce': 3589270120}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6803', 'nonce': 3589270120}]}, 'public_addr': '172.21.15.61:6801/3589270120', 'cluster_addr': '172.21.15.61:6802/3589270120', 'heartbeat_back_addr': '172.21.15.61:6804/3589270120', 'heartbeat_front_addr': '172.21.15.61:6803/3589270120', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '1c754ed6-c535-454f-b35b-00f62cd6ce5a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6813', 'nonce': 2849920842}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6814', 'nonce': 2849920842}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6816', 'nonce': 2849920842}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6815', 'nonce': 2849920842}]}, 'public_addr': '172.21.15.61:6813/2849920842', 'cluster_addr': '172.21.15.61:6814/2849920842', 'heartbeat_back_addr': '172.21.15.61:6816/2849920842', 'heartbeat_front_addr': '172.21.15.61:6815/2849920842', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '766572af-b9ca-4f38-927d-36f26101ba51', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 38, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6800', 'nonce': 2778770069}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6801', 'nonce': 2778770069}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6803', 'nonce': 2778770069}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6802', 'nonce': 2778770069}]}, 'public_addr': '172.21.15.152:6800/2778770069', 'cluster_addr': '172.21.15.152:6801/2778770069', 'heartbeat_back_addr': '172.21.15.152:6803/2778770069', 'heartbeat_front_addr': '172.21.15.152:6802/2778770069', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7a566b60-1543-41d7-894c-e2fbb63e48dc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 61, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6809', 'nonce': 263671335}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6810', 'nonce': 263671335}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6814', 'nonce': 263671335}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6812', 'nonce': 263671335}]}, 'public_addr': '172.21.15.152:6809/263671335', 'cluster_addr': '172.21.15.152:6810/263671335', 'heartbeat_back_addr': '172.21.15.152:6814/263671335', 'heartbeat_front_addr': '172.21.15.152:6812/263671335', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '28cc2358-4ac4-40b2-91a4-20edfe625a73', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6808', 'nonce': 4245639079}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6811', 'nonce': 4245639079}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6815', 'nonce': 4245639079}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6813', 'nonce': 4245639079}]}, 'public_addr': '172.21.15.152:6808/4245639079', 'cluster_addr': '172.21.15.152:6811/4245639079', 'heartbeat_back_addr': '172.21.15.152:6815/4245639079', 'heartbeat_front_addr': '172.21.15.152:6813/4245639079', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '8f79279b-d2d0-438e-b3c2-88df1241ff30', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 14, 'last_clean_end': 37, 'up_from': 61, 'up_thru': 62, 'down_at': 38, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6804', 'nonce': 3075390540}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6805', 'nonce': 3075390540}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6807', 'nonce': 3075390540}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6806', 'nonce': 3075390540}]}, 'public_addr': '172.21.15.152:6804/3075390540', 'cluster_addr': '172.21.15.152:6805/3075390540', 'heartbeat_back_addr': '172.21.15.152:6807/3075390540', 'heartbeat_front_addr': '172.21.15.152:6806/3075390540', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:52.171904+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:52.428840+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:51.681954+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:53.687417+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:50.158682+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:50.443807+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:49.750276+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:49.770987+0000', 'dead_epoch': 38}], 'pg_upmap': [{'pgid': '2.7', 'osds': [5, 3]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2023-12-06T18:31:52.243 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:31:52.243 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:52.239+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:52.305 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:31:52.305 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:31:52.344 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:52.340+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:52.425 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:31:52.425 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:31:52.444 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:52.441+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:52.494 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:52.494 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:52.506 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:31:52.531 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:31:52.531 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:31:52.575 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:52.571+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:52.646 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:52.642+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:52.652 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:31:52.653 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:31:52.680 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:52.680 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:52.689 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:31:52.753 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:31:52.753 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:31:52.754 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:52.750+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:52.842 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:31:52.842 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:31:52.844 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is set 2023-12-06T18:31:52.847 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:52.844+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:52.904 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:52.905 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:52.914 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:31:52.948 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:31:52.948 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:31:52.949 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:52.945+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:52.966 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:31:52.966 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:31:52.984 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2023-12-06T18:31:52.883091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226788,"kb_used_data":5324,"kb_used_omap":15,"kb_used_meta":217008,"kb_avail":754747932,"statfs":{"total":773094113280,"available":772861882368,"internally_reserved":0,"allocated":5451776,"data_stored":2692575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15557,"internal_metadata":222217019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.619220"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":31,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-12-06T18:31:36.092725+0000","last_change":"2023-12-06T18:29:09.051293+0000","last_active":"2023-12-06T18:31:36.092725+0000","last_peered":"2023-12-06T18:31:36.092725+0000","last_clean":"2023-12-06T18:31:36.092725+0000","last_became_active":"2023-12-06T18:29:09.045331+0000","last_became_peered":"2023-12-06T18:29:09.045331+0000","last_unstale":"2023-12-06T18:31:36.092725+0000","last_undegraded":"2023-12-06T18:31:36.092725+0000","last_fullsized":"2023-12-06T18:31:36.092725+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T02:46:01.854508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":62,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-12-06T18:31:36.092906+0000","last_change":"2023-12-06T18:27:04.883840+0000","last_active":"2023-12-06T18:31:36.092906+0000","last_peered":"2023-12-06T18:31:36.092906+0000","last_clean":"2023-12-06T18:31:36.092906+0000","last_became_active":"2023-12-06T18:27:04.858001+0000","last_became_peered":"2023-12-06T18:27:04.858001+0000","last_unstale":"2023-12-06T18:31:36.092906+0000","last_undegraded":"2023-12-06T18:31:36.092906+0000","last_fullsized":"2023-12-06T18:31:36.092906+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:44:02.916244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":64,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-12-06T18:31:36.093494+0000","last_change":"2023-12-06T18:27:04.336778+0000","last_active":"2023-12-06T18:31:36.093494+0000","last_peered":"2023-12-06T18:31:36.093494+0000","last_clean":"2023-12-06T18:31:36.093494+0000","last_became_active":"2023-12-06T18:27:04.321269+0000","last_became_peered":"2023-12-06T18:27:04.321269+0000","last_unstale":"2023-12-06T18:31:36.093494+0000","last_undegraded":"2023-12-06T18:31:36.093494+0000","last_fullsized":"2023-12-06T18:31:36.093494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T22:25:15.703579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-12-06T18:31:36.090942+0000","last_change":"2023-12-06T18:28:47.898016+0000","last_active":"2023-12-06T18:31:36.090942+0000","last_peered":"2023-12-06T18:31:36.090942+0000","last_clean":"2023-12-06T18:31:36.090942+0000","last_became_active":"2023-12-06T18:28:47.891814+0000","last_became_peered":"2023-12-06T18:28:47.891814+0000","last_unstale":"2023-12-06T18:31:36.090942+0000","last_undegraded":"2023-12-06T18:31:36.090942+0000","last_fullsized":"2023-12-06T18:31:36.090942+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T00:49:06.937273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":64,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-12-06T18:31:36.086925+0000","last_change":"2023-12-06T18:27:04.408081+0000","last_active":"2023-12-06T18:31:36.086925+0000","last_peered":"2023-12-06T18:31:36.086925+0000","last_clean":"2023-12-06T18:31:36.086925+0000","last_became_active":"2023-12-06T18:27:04.399765+0000","last_became_peered":"2023-12-06T18:27:04.399765+0000","last_unstale":"2023-12-06T18:31:36.086925+0000","last_undegraded":"2023-12-06T18:31:36.086925+0000","last_fullsized":"2023-12-06T18:31:36.086925+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:40:33.570114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-12-06T18:31:35.347769+0000","last_change":"2023-12-06T18:30:08.762262+0000","last_active":"2023-12-06T18:31:35.347769+0000","last_peered":"2023-12-06T18:31:35.347769+0000","last_clean":"2023-12-06T18:31:35.347769+0000","last_became_active":"2023-12-06T18:30:08.733421+0000","last_became_peered":"2023-12-06T18:30:08.733421+0000","last_unstale":"2023-12-06T18:31:35.347769+0000","last_undegraded":"2023-12-06T18:31:35.347769+0000","last_fullsized":"2023-12-06T18:31:35.347769+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:43:31.796397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":78,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-12-06T18:31:36.082694+0000","last_change":"2023-12-06T18:28:47.876556+0000","last_active":"2023-12-06T18:31:36.082694+0000","last_peered":"2023-12-06T18:31:36.082694+0000","last_clean":"2023-12-06T18:31:36.082694+0000","last_became_active":"2023-12-06T18:28:47.865169+0000","last_became_peered":"2023-12-06T18:28:47.865169+0000","last_unstale":"2023-12-06T18:31:36.082694+0000","last_undegraded":"2023-12-06T18:31:36.082694+0000","last_fullsized":"2023-12-06T18:31:36.082694+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:16:40.095542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":64,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-12-06T18:31:36.090641+0000","last_change":"2023-12-06T18:27:04.609381+0000","last_active":"2023-12-06T18:31:36.090641+0000","last_peered":"2023-12-06T18:31:36.090641+0000","last_clean":"2023-12-06T18:31:36.090641+0000","last_became_active":"2023-12-06T18:27:04.598701+0000","last_became_peered":"2023-12-06T18:27:04.598701+0000","last_unstale":"2023-12-06T18:31:36.090641+0000","last_undegraded":"2023-12-06T18:31:36.090641+0000","last_fullsized":"2023-12-06T18:31:36.090641+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:29:58.962692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":197,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-12-06T18:31:36.072672+0000","last_change":"2023-12-06T18:28:48.207134+0000","last_active":"2023-12-06T18:31:36.072672+0000","last_peered":"2023-12-06T18:31:36.072672+0000","last_clean":"2023-12-06T18:31:36.072672+0000","last_became_active":"2023-12-06T18:28:47.875258+0000","last_became_peered":"2023-12-06T18:28:47.875258+0000","last_unstale":"2023-12-06T18:31:36.072672+0000","last_undegraded":"2023-12-06T18:31:36.072672+0000","last_fullsized":"2023-12-06T18:31:36.072672+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:14:32.667039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1204224,"data_stored":1193520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411390,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":237147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":10.073,"5min":10.574,"15min":10.494999999999999},"min":{"1min":5.0049999999999999,"5min":5.0049999999999999,"15min":5.0049999999999999},"max":{"1min":21.175000000000001,"5min":108.57599999999999,"15min":108.57599999999999},"last":8.5990000000000002},{"interface":"front","average":{"1min":10.366,"5min":11.858000000000001,"15min":12.459},"min":{"1min":5.3440000000000003,"5min":3.7200000000000002,"15min":3.7200000000000002},"max":{"1min":18.559999999999999,"5min":128.09399999999999,"15min":128.09399999999999},"last":8.9760000000000009}]},{"osd":1,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":11.718,"5min":12.234,"15min":12.782},"min":{"1min":7.8460000000000001,"5min":6.2539999999999996,"15min":6.2539999999999996},"max":{"1min":25.940999999999999,"5min":70.072000000000003,"15min":70.072000000000003},"last":10.505000000000001},{"interface":"front","average":{"1min":11.935,"5min":13.823,"15min":15.397},"min":{"1min":6.9640000000000004,"5min":6.444,"15min":6.444},"max":{"1min":18.939,"5min":32.920000000000002,"15min":32.920000000000002},"last":9.7129999999999992}]},{"osd":2,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":14.039999999999999,"5min":14.664,"15min":14.712999999999999},"min":{"1min":8.5079999999999991,"5min":7.5149999999999997,"15min":7.5149999999999997},"max":{"1min":24.446000000000002,"5min":75.125,"15min":75.125},"last":11.218},{"interface":"front","average":{"1min":13.414999999999999,"5min":16.634,"15min":17.728000000000002},"min":{"1min":7.5419999999999998,"5min":7.5419999999999998,"15min":7.5419999999999998},"max":{"1min":25.609000000000002,"5min":69.403999999999996,"15min":69.403999999999996},"last":14.157}]},{"osd":4,"last update":"Wed Dec 6 18:31:04 2023","interfaces":[{"interface":"back","average":{"1min":15.379,"5min":12.092000000000001,"15min":11.932},"min":{"1min":8.7910000000000004,"5min":5.8879999999999999,"15min":5.8879999999999999},"max":{"1min":27.518000000000001,"5min":27.518000000000001,"15min":27.518000000000001},"last":18.053000000000001},{"interface":"front","average":{"1min":12.076000000000001,"5min":15.141999999999999,"15min":15.778},"min":{"1min":7.8949999999999996,"5min":6.3099999999999996,"15min":6.3099999999999996},"max":{"1min":19.82,"5min":71.718000000000004,"15min":71.718000000000004},"last":12.004}]},{"osd":5,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":18.186,"5min":18.327999999999999,"15min":17.914000000000001},"min":{"1min":10.417999999999999,"5min":8.3179999999999996,"15min":8.3179999999999996},"max":{"1min":33.335999999999999,"5min":75.751999999999995,"15min":75.751999999999995},"last":18.959},{"interface":"front","average":{"1min":17.651,"5min":17.765999999999998,"15min":18.141999999999999},"min":{"1min":8.4920000000000009,"5min":7.9269999999999996,"15min":7.9269999999999996},"max":{"1min":34.899999999999999,"5min":70.933000000000007,"15min":70.933000000000007},"last":14.824}]},{"osd":6,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":14.339,"5min":15.102,"15min":16.004000000000001},"min":{"1min":10.676,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":19.245999999999999,"5min":29.245000000000001,"15min":29.245000000000001},"last":12.778},{"interface":"front","average":{"1min":18.861000000000001,"5min":20.219999999999999,"15min":21.138999999999999},"min":{"1min":11.279,"5min":10.973000000000001,"15min":10.973000000000001},"max":{"1min":36.704999999999998,"5min":74.224000000000004,"15min":74.224000000000004},"last":20.126000000000001}]},{"osd":7,"last update":"Wed Dec 6 18:31:08 2023","interfaces":[{"interface":"back","average":{"1min":21.957999999999998,"5min":21.957999999999998,"15min":21.957999999999998},"min":{"1min":11.506,"5min":11.506,"15min":11.506},"max":{"1min":122.93000000000001,"5min":122.93000000000001,"15min":122.93000000000001},"last":13.429},{"interface":"front","average":{"1min":23.734000000000002,"5min":23.734000000000002,"15min":23.734000000000002},"min":{"1min":13.196,"5min":13.196,"15min":13.196},"max":{"1min":124.473,"5min":124.473,"15min":124.473},"last":21.629999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476797,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":237147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:31:04 2023","interfaces":[{"interface":"back","average":{"1min":9.3260000000000005,"5min":9.8870000000000005,"15min":10.974},"min":{"1min":5.4409999999999998,"5min":4.609,"15min":4.609},"max":{"1min":20.771000000000001,"5min":38.740000000000002,"15min":38.740000000000002},"last":7.8499999999999996},{"interface":"front","average":{"1min":10.164999999999999,"5min":10.585000000000001,"15min":11.113},"min":{"1min":4.3280000000000003,"5min":4.3280000000000003,"15min":4.3280000000000003},"max":{"1min":17.048999999999999,"5min":33.387,"15min":33.387},"last":9.4879999999999995}]},{"osd":1,"last update":"Wed Dec 6 18:31:04 2023","interfaces":[{"interface":"back","average":{"1min":10.449999999999999,"5min":11.323,"15min":11.528},"min":{"1min":7.2229999999999999,"5min":5.9139999999999997,"15min":5.9139999999999997},"max":{"1min":20.832000000000001,"5min":42.920999999999999,"15min":42.920999999999999},"last":7.0570000000000004},{"interface":"front","average":{"1min":13.028,"5min":13.336,"15min":13.786},"min":{"1min":6.1479999999999997,"5min":6.1479999999999997,"15min":6.1479999999999997},"max":{"1min":21.481999999999999,"5min":34.607999999999997,"15min":34.607999999999997},"last":10.131}]},{"osd":3,"last update":"Wed Dec 6 18:31:04 2023","interfaces":[{"interface":"back","average":{"1min":12.215,"5min":16.128,"15min":18.634},"min":{"1min":7.2549999999999999,"5min":6.1260000000000003,"15min":6.1260000000000003},"max":{"1min":17.878,"5min":112.129,"15min":112.129},"last":8.875},{"interface":"front","average":{"1min":15.457000000000001,"5min":19.382000000000001,"15min":23.277999999999999},"min":{"1min":11.069000000000001,"5min":7.5510000000000002,"15min":7.5510000000000002},"max":{"1min":19.986999999999998,"5min":181.601,"15min":181.601},"last":16.635000000000002}]},{"osd":4,"last update":"Wed Dec 6 18:31:04 2023","interfaces":[{"interface":"back","average":{"1min":14.536,"5min":15.851000000000001,"15min":17.577999999999999},"min":{"1min":8.1349999999999998,"5min":6.609,"15min":6.609},"max":{"1min":23.422000000000001,"5min":97.088999999999999,"15min":97.088999999999999},"last":12.911},{"interface":"front","average":{"1min":14.007999999999999,"5min":17.056999999999999,"15min":20.344999999999999},"min":{"1min":6.0720000000000001,"5min":6.0720000000000001,"15min":6.0720000000000001},"max":{"1min":43.198,"5min":98.069000000000003,"15min":98.069000000000003},"last":11.551}]},{"osd":5,"last update":"Wed Dec 6 18:31:04 2023","interfaces":[{"interface":"back","average":{"1min":17.363,"5min":17.454999999999998,"15min":18.015000000000001},"min":{"1min":8.2100000000000009,"5min":7.8170000000000002,"15min":7.8170000000000002},"max":{"1min":22.111000000000001,"5min":57.628999999999998,"15min":57.628999999999998},"last":18.513999999999999},{"interface":"front","average":{"1min":14.59,"5min":15.462999999999999,"15min":15.714},"min":{"1min":8.5990000000000002,"5min":8.5990000000000002,"15min":8.5990000000000002},"max":{"1min":21.459,"5min":47.749000000000002,"15min":47.749000000000002},"last":14.151999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:31:04 2023","interfaces":[{"interface":"back","average":{"1min":16.934999999999999,"5min":29.489000000000001,"15min":41.509999999999998},"min":{"1min":11.621,"5min":8.7289999999999992,"15min":8.7289999999999992},"max":{"1min":26.984999999999999,"5min":498.38499999999999,"15min":498.38499999999999},"last":12.285},{"interface":"front","average":{"1min":18.338000000000001,"5min":30.812000000000001,"15min":43.351999999999997},"min":{"1min":9.7010000000000005,"5min":9.7010000000000005,"15min":9.7010000000000005},"max":{"1min":25.861000000000001,"5min":496.17399999999998,"15min":496.17399999999998},"last":17.678000000000001}]},{"osd":7,"last update":"Wed Dec 6 18:31:10 2023","interfaces":[{"interface":"back","average":{"1min":18.922999999999998,"5min":18.922999999999998,"15min":18.922999999999998},"min":{"1min":11.933999999999999,"5min":11.933999999999999,"15min":11.933999999999999},"max":{"1min":49.456000000000003,"5min":49.456000000000003,"15min":49.456000000000003},"last":13.552},{"interface":"front","average":{"1min":20.155999999999999,"5min":20.155999999999999,"15min":20.155999999999999},"min":{"1min":13.901,"5min":13.901,"15min":13.901},"max":{"1min":40.656999999999996,"5min":40.656999999999996,"15min":40.656999999999996},"last":15.885}]}]},{"osd":1,"up_from":16,"seq":68719476798,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":983040,"data_stored":634987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":10.913,"5min":15.702,"15min":21.475999999999999},"min":{"1min":5.0890000000000004,"5min":5.0890000000000004,"15min":5.0890000000000004},"max":{"1min":18.282,"5min":238.87100000000001,"15min":238.87100000000001},"last":7.0170000000000003},{"interface":"front","average":{"1min":9.7639999999999993,"5min":26.013999999999999,"15min":42.161999999999999},"min":{"1min":6.0350000000000001,"5min":4.4269999999999996,"15min":4.4269999999999996},"max":{"1min":16.062999999999999,"5min":661.33600000000001,"15min":661.33600000000001},"last":7.7469999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":18.408999999999999,"5min":22.713000000000001,"15min":32.253},"min":{"1min":7.1559999999999997,"5min":6.6340000000000003,"15min":6.6340000000000003},"max":{"1min":93.168000000000006,"5min":384.315,"15min":384.315},"last":9.5570000000000004},{"interface":"front","average":{"1min":22.268999999999998,"5min":31.728000000000002,"15min":46.054000000000002},"min":{"1min":12.404999999999999,"5min":5.7009999999999996,"15min":5.7009999999999996},"max":{"1min":94.230000000000004,"5min":656.98699999999997,"15min":656.98699999999997},"last":10.724}]},{"osd":3,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":11.784000000000001,"5min":11.923,"15min":13.041},"min":{"1min":6.6619999999999999,"5min":5.548,"15min":5.548},"max":{"1min":31.16,"5min":34.057000000000002,"15min":34.057000000000002},"last":14.305},{"interface":"front","average":{"1min":14.317,"5min":14.416,"15min":15.952999999999999},"min":{"1min":7.9299999999999997,"5min":6.5730000000000004,"15min":6.5730000000000004},"max":{"1min":38.133000000000003,"5min":38.133000000000003,"15min":38.133000000000003},"last":13.012}]},{"osd":4,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":12.744999999999999,"5min":25.795000000000002,"15min":39.542000000000002},"min":{"1min":7.0839999999999996,"5min":6.2930000000000001,"15min":6.2930000000000001},"max":{"1min":29.344999999999999,"5min":447.67200000000003,"15min":447.67200000000003},"last":16.98},{"interface":"front","average":{"1min":18.292999999999999,"5min":31.774999999999999,"15min":47.042999999999999},"min":{"1min":12.065,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":44.043999999999997,"5min":630.572,"15min":630.572},"last":14.032}]},{"osd":5,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":16.238,"5min":29.34,"15min":41.911999999999999},"min":{"1min":8.0069999999999997,"5min":8.0069999999999997,"15min":8.0069999999999997},"max":{"1min":45.750999999999998,"5min":389.55000000000001,"15min":389.55000000000001},"last":12.387},{"interface":"front","average":{"1min":17.428999999999998,"5min":34.850999999999999,"15min":54.871000000000002},"min":{"1min":7.5869999999999997,"5min":7.0410000000000004,"15min":7.0410000000000004},"max":{"1min":37.744,"5min":658.48599999999999,"15min":658.48599999999999},"last":13.272}]},{"osd":6,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":14.976000000000001,"5min":24.675999999999998,"15min":34.963999999999999},"min":{"1min":8.5229999999999997,"5min":8.4710000000000001,"15min":8.4710000000000001},"max":{"1min":54.383000000000003,"5min":439.56400000000002,"15min":439.56400000000002},"last":8.9760000000000009},{"interface":"front","average":{"1min":20.981999999999999,"5min":27.957000000000001,"15min":36.582999999999998},"min":{"1min":13.901999999999999,"5min":10.641999999999999,"15min":10.641999999999999},"max":{"1min":55.423999999999999,"5min":386.71499999999997,"15min":386.71499999999997},"last":20.890000000000001}]},{"osd":7,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":18.789000000000001,"5min":18.789000000000001,"15min":18.789000000000001},"min":{"1min":11.512,"5min":11.512,"15min":11.512},"max":{"1min":97.561999999999998,"5min":97.561999999999998,"15min":97.561999999999998},"last":15.702999999999999},{"interface":"front","average":{"1min":17.646000000000001,"5min":17.646000000000001,"15min":17.646000000000001},"min":{"1min":10.603,"5min":10.603,"15min":10.603},"max":{"1min":45.402999999999999,"5min":45.402999999999999,"15min":45.402999999999999},"last":19.475999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476799,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":581632,"data_stored":237147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 6 18:31:03 2023","interfaces":[{"interface":"back","average":{"1min":11.746,"5min":13.705,"15min":16.933},"min":{"1min":6.8760000000000003,"5min":3.2360000000000002,"15min":3.2360000000000002},"max":{"1min":25.324999999999999,"5min":116.593,"15min":116.593},"last":5.694},{"interface":"front","average":{"1min":11.741,"5min":15.111000000000001,"15min":17.638999999999999},"min":{"1min":6.6879999999999997,"5min":4.9139999999999997,"15min":4.9139999999999997},"max":{"1min":26.614999999999998,"5min":117.896,"15min":117.896},"last":12.307}]},{"osd":2,"last update":"Wed Dec 6 18:31:03 2023","interfaces":[{"interface":"back","average":{"1min":12.739000000000001,"5min":13.983000000000001,"15min":13.587},"min":{"1min":7.9240000000000004,"5min":5.694,"15min":5.694},"max":{"1min":17.562999999999999,"5min":43.752000000000002,"15min":43.752000000000002},"last":12.976000000000001},{"interface":"front","average":{"1min":10.945,"5min":12.004,"15min":11.316000000000001},"min":{"1min":6.6210000000000004,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":17.666,"5min":46.387999999999998,"15min":46.387999999999998},"last":9.3859999999999992}]},{"osd":3,"last update":"Wed Dec 6 18:31:08 2023","interfaces":[{"interface":"back","average":{"1min":14.369999999999999,"5min":14.734,"15min":16.140999999999998},"min":{"1min":9.2289999999999992,"5min":6.1379999999999999,"15min":6.1379999999999999},"max":{"1min":20.352,"5min":46.689999999999998,"15min":46.689999999999998},"last":13.605},{"interface":"front","average":{"1min":14.148,"5min":15.906000000000001,"15min":16.155000000000001},"min":{"1min":11.037000000000001,"5min":8.0329999999999995,"15min":8.0329999999999995},"max":{"1min":20.064,"5min":50.545999999999999,"15min":50.545999999999999},"last":9.9939999999999998}]},{"osd":4,"last update":"Wed Dec 6 18:31:03 2023","interfaces":[{"interface":"back","average":{"1min":10.169,"5min":15.519,"15min":19.427},"min":{"1min":6.641,"5min":5.9349999999999996,"15min":5.9349999999999996},"max":{"1min":22.158000000000001,"5min":82.432000000000002,"15min":82.432000000000002},"last":7.4889999999999999},{"interface":"front","average":{"1min":18.053000000000001,"5min":18.855,"15min":20.155000000000001},"min":{"1min":12.497,"5min":6.2629999999999999,"15min":6.2629999999999999},"max":{"1min":23.114000000000001,"5min":81.239999999999995,"15min":81.239999999999995},"last":14.231}]},{"osd":5,"last update":"Wed Dec 6 18:31:03 2023","interfaces":[{"interface":"back","average":{"1min":15.563000000000001,"5min":16.891999999999999,"15min":17.385000000000002},"min":{"1min":8.6479999999999997,"5min":8.0099999999999998,"15min":8.0099999999999998},"max":{"1min":27.327999999999999,"5min":71.367000000000004,"15min":71.367000000000004},"last":16.248000000000001},{"interface":"front","average":{"1min":14.071999999999999,"5min":16.843,"15min":17.75},"min":{"1min":8.9320000000000004,"5min":8.5299999999999994,"15min":8.5299999999999994},"max":{"1min":29.286000000000001,"5min":72.307000000000002,"15min":72.307000000000002},"last":8.3040000000000003}]},{"osd":6,"last update":"Wed Dec 6 18:31:03 2023","interfaces":[{"interface":"back","average":{"1min":18.530000000000001,"5min":19.073,"15min":19.029},"min":{"1min":11.763,"5min":7.7560000000000002,"15min":7.7560000000000002},"max":{"1min":23.082999999999998,"5min":62.932000000000002,"15min":62.932000000000002},"last":14.852},{"interface":"front","average":{"1min":15.797000000000001,"5min":17.73,"15min":18.940000000000001},"min":{"1min":11.856999999999999,"5min":9.6940000000000008,"15min":9.6940000000000008},"max":{"1min":24.016999999999999,"5min":52.627000000000002,"15min":52.627000000000002},"last":11.679}]},{"osd":7,"last update":"Wed Dec 6 18:31:08 2023","interfaces":[{"interface":"back","average":{"1min":19.120999999999999,"5min":19.120999999999999,"15min":19.120999999999999},"min":{"1min":13.513,"5min":13.513,"15min":13.513},"max":{"1min":24.071999999999999,"5min":24.071999999999999,"15min":24.071999999999999},"last":15.483000000000001},{"interface":"front","average":{"1min":17.896999999999998,"5min":17.896999999999998,"15min":17.896999999999998},"min":{"1min":13.669,"5min":13.669,"15min":13.669},"max":{"1min":23.620000000000001,"5min":23.620000000000001,"15min":23.620000000000001},"last":18.312000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509503,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":983040,"data_stored":634987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":8.1989999999999998,"5min":11.412000000000001,"15min":14.845000000000001},"min":{"1min":4.7389999999999999,"5min":3.3660000000000001,"15min":3.3660000000000001},"max":{"1min":15.797000000000001,"5min":208.291,"15min":208.291},"last":11.129},{"interface":"front","average":{"1min":10.285,"5min":14.279,"15min":18.748999999999999},"min":{"1min":5.1429999999999998,"5min":4.4299999999999997,"15min":4.4299999999999997},"max":{"1min":15.239000000000001,"5min":199.94900000000001,"15min":199.94900000000001},"last":6}]},{"osd":1,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":13.602,"5min":34.588000000000001,"15min":55.927},"min":{"1min":6.9450000000000003,"5min":6.9450000000000003,"15min":6.9450000000000003},"max":{"1min":19.356999999999999,"5min":1211.6010000000001,"15min":1211.6010000000001},"last":11.958},{"interface":"front","average":{"1min":8.9770000000000003,"5min":30.417999999999999,"15min":51.085000000000001},"min":{"1min":4.7199999999999998,"5min":4.7199999999999998,"15min":4.7199999999999998},"max":{"1min":14.074999999999999,"5min":1189.3869999999999,"15min":1189.3869999999999},"last":6.8879999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":13.939,"5min":18.715,"15min":23.940000000000001},"min":{"1min":9.0090000000000003,"5min":6.5129999999999999,"15min":6.5129999999999999},"max":{"1min":19.02,"5min":283.46800000000002,"15min":283.46800000000002},"last":13.599},{"interface":"front","average":{"1min":16.815999999999999,"5min":22.056999999999999,"15min":27.766999999999999},"min":{"1min":10.054,"5min":8.9350000000000005,"15min":8.9350000000000005},"max":{"1min":22.969999999999999,"5min":296.47800000000001,"15min":296.47800000000001},"last":15.243}]},{"osd":3,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":17.452999999999999,"5min":17.550000000000001,"15min":18.931000000000001},"min":{"1min":12.442,"5min":9.1739999999999995,"15min":9.1739999999999995},"max":{"1min":23.66,"5min":47.926000000000002,"15min":47.926000000000002},"last":12.557},{"interface":"front","average":{"1min":12.148,"5min":13.238,"15min":14.881},"min":{"1min":7.5810000000000004,"5min":5.3979999999999997,"15min":5.3979999999999997},"max":{"1min":17.27,"5min":66.176000000000002,"15min":66.176000000000002},"last":14.404}]},{"osd":4,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":18.606000000000002,"5min":18.021999999999998,"15min":19.501999999999999},"min":{"1min":12.250999999999999,"5min":10.286,"15min":10.286},"max":{"1min":29.350999999999999,"5min":67.248000000000005,"15min":67.248000000000005},"last":17.689},{"interface":"front","average":{"1min":10.641,"5min":11.377000000000001,"15min":12.823},"min":{"1min":6.1020000000000003,"5min":5.3300000000000001,"15min":5.3300000000000001},"max":{"1min":28.155999999999999,"5min":77.266999999999996,"15min":77.266999999999996},"last":8.8249999999999993}]},{"osd":6,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":12.084,"5min":13.314,"15min":15.378},"min":{"1min":7.8390000000000004,"5min":5.915,"15min":5.915},"max":{"1min":17.699000000000002,"5min":76.751999999999995,"15min":76.751999999999995},"last":16.815999999999999},{"interface":"front","average":{"1min":19.111999999999998,"5min":19.358000000000001,"15min":20.786999999999999},"min":{"1min":15.135999999999999,"5min":8.8390000000000004,"15min":8.8390000000000004},"max":{"1min":24.015000000000001,"5min":57.101999999999997,"15min":57.101999999999997},"last":18.556999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":14.211,"5min":14.211,"15min":14.211},"min":{"1min":7.5519999999999996,"5min":7.5519999999999996,"15min":7.5519999999999996},"max":{"1min":21.047000000000001,"5min":21.047000000000001,"15min":21.047000000000001},"last":15.843},{"interface":"front","average":{"1min":11.888,"5min":11.888,"15min":11.888},"min":{"1min":8.2829999999999995,"5min":8.2829999999999995,"15min":8.2829999999999995},"max":{"1min":15.411,"5min":15.411,"15min":15.411},"last":9.8759999999999994}]}]},{"osd":4,"up_from":15,"seq":64424509504,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":577536,"data_stored":236866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":10.843,"5min":11.773,"15min":13.387},"min":{"1min":5.3300000000000001,"5min":3.7429999999999999,"15min":3.7429999999999999},"max":{"1min":35.863,"5min":53.298999999999999,"15min":53.298999999999999},"last":6.2729999999999997},{"interface":"front","average":{"1min":10.867000000000001,"5min":10.327999999999999,"15min":10.82},"min":{"1min":5.9139999999999997,"5min":3.9670000000000001,"15min":3.9670000000000001},"max":{"1min":35.146000000000001,"5min":52.567,"15min":52.567},"last":5.524}]},{"osd":1,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":11.565,"5min":12.291,"15min":13.744999999999999},"min":{"1min":5.8390000000000004,"5min":5.835,"15min":5.835},"max":{"1min":20.745000000000001,"5min":49.183,"15min":49.183},"last":7.6189999999999998},{"interface":"front","average":{"1min":11.513,"5min":13.619,"15min":15.367000000000001},"min":{"1min":7.4690000000000003,"5min":5.96,"15min":5.96},"max":{"1min":19.991,"5min":45.412999999999997,"15min":45.412999999999997},"last":12.454000000000001}]},{"osd":2,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":10.635999999999999,"5min":16.286000000000001,"15min":21.515999999999998},"min":{"1min":7.3319999999999999,"5min":5.6390000000000002,"15min":5.6390000000000002},"max":{"1min":14.907999999999999,"5min":85.394000000000005,"15min":85.394000000000005},"last":10.382999999999999},{"interface":"front","average":{"1min":14.616,"5min":16.981999999999999,"15min":20.102},"min":{"1min":8.1720000000000006,"5min":6.3639999999999999,"15min":6.3639999999999999},"max":{"1min":19.039000000000001,"5min":69.530000000000001,"15min":69.530000000000001},"last":15.050000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:31:07 2023","interfaces":[{"interface":"back","average":{"1min":15.746,"5min":14.6,"15min":14.467000000000001},"min":{"1min":10.022,"5min":7.008,"15min":7.008},"max":{"1min":20.195,"5min":26.219000000000001,"15min":26.219000000000001},"last":15.855},{"interface":"front","average":{"1min":11.138999999999999,"5min":13.771000000000001,"15min":14.977},"min":{"1min":6.7370000000000001,"5min":5.5549999999999997,"15min":5.5549999999999997},"max":{"1min":18.253,"5min":29.902999999999999,"15min":29.902999999999999},"last":9.8290000000000006}]},{"osd":5,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":13.227,"5min":14.890000000000001,"15min":16.747},"min":{"1min":9.3089999999999993,"5min":6.125,"15min":6.125},"max":{"1min":15.754,"5min":78.798000000000002,"15min":78.798000000000002},"last":11.069000000000001},{"interface":"front","average":{"1min":17.539000000000001,"5min":17.667999999999999,"15min":18.658000000000001},"min":{"1min":12.125999999999999,"5min":6.4450000000000003,"15min":6.4450000000000003},"max":{"1min":23.238,"5min":55.814999999999998,"15min":55.814999999999998},"last":13.127000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:31:02 2023","interfaces":[{"interface":"back","average":{"1min":18.984999999999999,"5min":17.715,"15min":17.992999999999999},"min":{"1min":16.178000000000001,"5min":7.7850000000000001,"15min":7.7850000000000001},"max":{"1min":22.629000000000001,"5min":51.149000000000001,"15min":51.149000000000001},"last":16.690999999999999},{"interface":"front","average":{"1min":13.962999999999999,"5min":15.597,"15min":16.484999999999999},"min":{"1min":9.7680000000000007,"5min":7.8700000000000001,"15min":7.8700000000000001},"max":{"1min":18.120999999999999,"5min":40.426000000000002,"15min":40.426000000000002},"last":8.8460000000000001}]},{"osd":7,"last update":"Wed Dec 6 18:31:07 2023","interfaces":[{"interface":"back","average":{"1min":18.751000000000001,"5min":18.751000000000001,"15min":18.751000000000001},"min":{"1min":10.289,"5min":10.289,"15min":10.289},"max":{"1min":60.390000000000001,"5min":60.390000000000001,"15min":60.390000000000001},"last":14.416},{"interface":"front","average":{"1min":22.809000000000001,"5min":22.809000000000001,"15min":22.809000000000001},"min":{"1min":17.552,"5min":17.552,"15min":17.552},"max":{"1min":57.606999999999999,"5min":57.606999999999999,"15min":57.606999999999999},"last":17.579999999999998}]}]},{"osd":7,"up_from":61,"seq":261993005078,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32776,"kb_used_data":568,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94339064,"statfs":{"total":96636764160,"available":96603201536,"internally_reserved":0,"allocated":581632,"data_stored":237147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4395,"internal_metadata":28634837},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":10.141,"5min":10.141,"15min":10.141},"min":{"1min":4.1059999999999999,"5min":4.1059999999999999,"15min":4.1059999999999999},"max":{"1min":32.523000000000003,"5min":32.523000000000003,"15min":32.523000000000003},"last":4.3780000000000001},{"interface":"front","average":{"1min":11.015000000000001,"5min":11.015000000000001,"15min":11.015000000000001},"min":{"1min":4.2530000000000001,"5min":4.2530000000000001,"15min":4.2530000000000001},"max":{"1min":33.530999999999999,"5min":33.530999999999999,"15min":33.530999999999999},"last":8.0009999999999994}]},{"osd":1,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":17.460000000000001,"5min":17.460000000000001,"15min":17.460000000000001},"min":{"1min":7.3789999999999996,"5min":7.3789999999999996,"15min":7.3789999999999996},"max":{"1min":64.765000000000001,"5min":64.765000000000001,"15min":64.765000000000001},"last":8.5899999999999999},{"interface":"front","average":{"1min":18.109999999999999,"5min":18.109999999999999,"15min":18.109999999999999},"min":{"1min":7.0739999999999998,"5min":7.0739999999999998,"15min":7.0739999999999998},"max":{"1min":65.994,"5min":65.994,"15min":65.994},"last":10.106}]},{"osd":2,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":18.016999999999999,"5min":18.016999999999999,"15min":18.016999999999999},"min":{"1min":8.0619999999999994,"5min":8.0619999999999994,"15min":8.0619999999999994},"max":{"1min":46.478000000000002,"5min":46.478000000000002,"15min":46.478000000000002},"last":9.0649999999999995},{"interface":"front","average":{"1min":14.119,"5min":14.119,"15min":14.119},"min":{"1min":6.7039999999999997,"5min":6.7039999999999997,"15min":6.7039999999999997},"max":{"1min":41.603999999999999,"5min":41.603999999999999,"15min":41.603999999999999},"last":11.492000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":17.262,"5min":17.262,"15min":17.262},"min":{"1min":8.8629999999999995,"5min":8.8629999999999995,"15min":8.8629999999999995},"max":{"1min":60.463999999999999,"5min":60.463999999999999,"15min":60.463999999999999},"last":13.586},{"interface":"front","average":{"1min":20.349,"5min":20.349,"15min":20.349},"min":{"1min":11.082000000000001,"5min":11.082000000000001,"15min":11.082000000000001},"max":{"1min":56.286000000000001,"5min":56.286000000000001,"15min":56.286000000000001},"last":12.093}]},{"osd":4,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":21.329000000000001,"5min":21.329000000000001,"15min":21.329000000000001},"min":{"1min":12.099,"5min":12.099,"15min":12.099},"max":{"1min":57.716999999999999,"5min":57.716999999999999,"15min":57.716999999999999},"last":10.797000000000001},{"interface":"front","average":{"1min":18.623999999999999,"5min":18.623999999999999,"15min":18.623999999999999},"min":{"1min":8.2729999999999997,"5min":8.2729999999999997,"15min":8.2729999999999997},"max":{"1min":67.159000000000006,"5min":67.159000000000006,"15min":67.159000000000006},"last":15.087}]},{"osd":5,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":14.739000000000001,"5min":14.739000000000001,"15min":14.739000000000001},"min":{"1min":6.3399999999999999,"5min":6.3399999999999999,"15min":6.3399999999999999},"max":{"1min":36.317999999999998,"5min":36.317999999999998,"15min":36.317999999999998},"last":13.923999999999999},{"interface":"front","average":{"1min":19.93,"5min":19.93,"15min":19.93},"min":{"1min":10.111000000000001,"5min":10.111000000000001,"15min":10.111000000000001},"max":{"1min":70.900000000000006,"5min":70.900000000000006,"15min":70.900000000000006},"last":16.533000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:31:06 2023","interfaces":[{"interface":"back","average":{"1min":16.776,"5min":16.776,"15min":16.776},"min":{"1min":8.7170000000000005,"5min":8.7170000000000005,"15min":8.7170000000000005},"max":{"1min":54.393000000000001,"5min":54.393000000000001,"15min":54.393000000000001},"last":15.829000000000001},{"interface":"front","average":{"1min":20.928999999999998,"5min":20.928999999999998,"15min":20.928999999999998},"min":{"1min":12.134,"5min":12.134,"15min":12.134},"max":{"1min":69.263000000000005,"5min":69.263000000000005,"15min":69.263000000000005},"last":17.683}]}]},{"osd":6,"up_from":14,"seq":60129542206,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":581632,"data_stored":237147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:31:00 2023","interfaces":[{"interface":"back","average":{"1min":9.8420000000000005,"5min":9.5800000000000001,"15min":9.9199999999999999},"min":{"1min":4.1260000000000003,"5min":3.8519999999999999,"15min":3.8519999999999999},"max":{"1min":22.094999999999999,"5min":46.947000000000003,"15min":46.947000000000003},"last":4.7430000000000003},{"interface":"front","average":{"1min":11.103,"5min":10.573,"15min":10.211},"min":{"1min":4.609,"5min":4.0650000000000004,"15min":4.0650000000000004},"max":{"1min":19.335000000000001,"5min":45.953000000000003,"15min":45.953000000000003},"last":13.894}]},{"osd":1,"last update":"Wed Dec 6 18:31:00 2023","interfaces":[{"interface":"back","average":{"1min":9.1880000000000006,"5min":10.928000000000001,"15min":12.289999999999999},"min":{"1min":4.5650000000000004,"5min":4.3760000000000003,"15min":4.3760000000000003},"max":{"1min":15.869,"5min":66.313000000000002,"15min":66.313000000000002},"last":12.901},{"interface":"front","average":{"1min":11.144,"5min":13.48,"15min":14.359999999999999},"min":{"1min":6.0170000000000003,"5min":5.8540000000000001,"15min":5.8540000000000001},"max":{"1min":16.629000000000001,"5min":67.533000000000001,"15min":67.533000000000001},"last":10.747}]},{"osd":2,"last update":"Wed Dec 6 18:31:00 2023","interfaces":[{"interface":"back","average":{"1min":12.227,"5min":15.805999999999999,"15min":19.670000000000002},"min":{"1min":6.0640000000000001,"5min":6.0640000000000001,"15min":6.0640000000000001},"max":{"1min":16.919,"5min":200.88499999999999,"15min":200.88499999999999},"last":16.997},{"interface":"front","average":{"1min":16.376000000000001,"5min":16.141999999999999,"15min":16.404},"min":{"1min":7.8129999999999997,"5min":7.4119999999999999,"15min":7.4119999999999999},"max":{"1min":30.908999999999999,"5min":48.005000000000003,"15min":48.005000000000003},"last":16.260000000000002}]},{"osd":3,"last update":"Wed Dec 6 18:31:05 2023","interfaces":[{"interface":"back","average":{"1min":14.098000000000001,"5min":13.124000000000001,"15min":13.737},"min":{"1min":7.3250000000000002,"5min":7.298,"15min":7.298},"max":{"1min":36.283999999999999,"5min":36.283999999999999,"15min":36.283999999999999},"last":7.8849999999999998},{"interface":"front","average":{"1min":16.347000000000001,"5min":15.859,"15min":16.193000000000001},"min":{"1min":11.247999999999999,"5min":7.6079999999999997,"15min":7.6079999999999997},"max":{"1min":34.317,"5min":47.006999999999998,"15min":47.006999999999998},"last":14.707000000000001}]},{"osd":4,"last update":"Wed Dec 6 18:31:00 2023","interfaces":[{"interface":"back","average":{"1min":15.241,"5min":14.584,"15min":15.077999999999999},"min":{"1min":6.7999999999999998,"5min":6.1859999999999999,"15min":6.1859999999999999},"max":{"1min":49.170000000000002,"5min":49.170000000000002,"15min":49.170000000000002},"last":8.9719999999999995},{"interface":"front","average":{"1min":17.789000000000001,"5min":16.899000000000001,"15min":16.725000000000001},"min":{"1min":8.9019999999999992,"5min":7.7169999999999996,"15min":7.7169999999999996},"max":{"1min":38.085000000000001,"5min":48.988999999999997,"15min":48.988999999999997},"last":15.486000000000001}]},{"osd":5,"last update":"Wed Dec 6 18:31:00 2023","interfaces":[{"interface":"back","average":{"1min":17.669,"5min":18.295999999999999,"15min":18.401},"min":{"1min":10.053000000000001,"5min":7.8879999999999999,"15min":7.8879999999999999},"max":{"1min":25.257999999999999,"5min":57.308,"15min":57.308},"last":17.59},{"interface":"front","average":{"1min":9.8829999999999991,"5min":12.920999999999999,"15min":14.550000000000001},"min":{"1min":6.5199999999999996,"5min":6.0129999999999999,"15min":6.0129999999999999},"max":{"1min":14.032999999999999,"5min":68.463999999999999,"15min":68.463999999999999},"last":10.140000000000001}]},{"osd":7,"last update":"Wed Dec 6 18:31:10 2023","interfaces":[{"interface":"back","average":{"1min":15.401,"5min":15.401,"15min":15.401},"min":{"1min":8.5310000000000006,"5min":8.5310000000000006,"15min":8.5310000000000006},"max":{"1min":28.393999999999998,"5min":28.393999999999998,"15min":28.393999999999998},"last":11.367000000000001},{"interface":"front","average":{"1min":18.766999999999999,"5min":18.766999999999999,"15min":18.766999999999999},"min":{"1min":13.574,"5min":13.574,"15min":13.574},"max":{"1min":27.132000000000001,"5min":27.132000000000001,"15min":27.132000000000001},"last":18.164999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:31:52.985 INFO:tasks.thrashosds.thrasher:Setting 2.3 to [4, 6, 3, 1] 2023-12-06T18:31:52.985 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.3', '4', '6', '3', '1'] 2023-12-06T18:31:52.986 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.3 4 6 3 1 2023-12-06T18:31:53.035 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:31:53.035 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:31:53.049 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:53.045+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:53.095 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:53.095 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:53.101 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":40,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:31:53.105 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:31:53.122 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2023-12-06T18:31:53.122 INFO:tasks.mon_recovery:causing some monitor log activity 2023-12-06T18:31:53.122 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '1 of 30' 2023-12-06T18:31:53.149 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:53.146+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:53.250 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:53.247+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:53.253 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:53.253 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:53.263 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:31:53.351 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:53.347+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:53.403 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:53.403 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:53.412 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:31:53.452 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:53.449+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:53.554 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:53.550+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:53.592 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:53.592 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:53.603 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:31:53.654 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:53.651+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:53.731 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:53.731 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:53.740 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:31:53.785 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:53.781+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:53.855 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:53.852+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:53.867 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:53.867 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:53.876 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:31:53.960 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:53.957+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:54.043 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:54.043 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:54.053 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:31:54.057 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:54.053+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:54.076 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '2 of 30' 2023-12-06T18:31:54.141 INFO:teuthology.orchestra.run.smithi061.stderr:set 2.3 pg_upmap_items mapping to [4->6,3->1] 2023-12-06T18:31:54.157 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:54.154+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:54.226 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:54.226 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:54.236 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:31:54.277 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:54.274+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:54.359 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:54.356+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:54.404 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:54.404 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:54.414 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:31:54.460 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:54.456+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:54.561 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:54.558+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:54.584 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:54.584 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:54.593 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:31:54.662 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:54.659+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:54.762 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:54.762 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:54.763 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:54.760+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:54.771 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:31:54.862 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-12-06T18:31:54.864 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:54.860+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:54.939 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:54.939 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:54.949 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:31:54.991 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:54.987+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:55.051 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:31:55.065 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:55.061+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:55.121 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:55.121 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:55.131 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:31:55.168 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:55.165+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:55.283 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:55.280+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:55.298 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:55.298 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:55.308 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:31:55.310 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '3 of 30' 2023-12-06T18:31:55.313 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:31:55.313 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:31:55.367 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:55.363+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:55.415 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:31:55.415 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:31:55.468 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:55.464+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:55.479 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:55.479 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:55.488 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:31:55.533 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:31:55.533 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:31:55.569 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:55.565+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:55.616 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:55.616 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:55.625 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:31:55.628 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:31:55.628 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:31:55.670 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:55.666+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:55.718 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:31:55.719 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:31:55.770 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:55.767+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:55.789 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:55.789 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:55.789 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:55.789 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:55.803 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:31:55.805 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:31:55.805 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:31:55.871 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:55.868+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:55.895 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:31:55.896 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:31:55.953 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:55.953 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:55.953 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:55.954 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:55.954 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:55.962 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:31:55.980 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:31:55.980 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:31:55.980 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:55.976+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:56.073 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:56.070+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:56.103 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:56.103 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:56.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:56.103 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:31:56.103 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:56.114 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:31:56.199 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:56.195+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:56.275 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:56.272+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:56.309 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:56.309 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:56.310 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:56.310 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:56.321 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:31:56.322 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '4 of 30' 2023-12-06T18:31:56.407 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:56.403+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:56.478 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:56.474+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:56.494 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is unset 2023-12-06T18:31:56.528 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:56.528 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:56.528 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:56.528 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:56.528 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:56.538 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:31:56.580 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:56.575+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:56.680 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:56.677+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:56.773 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:56.773 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:56.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:56.773 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:56.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.773 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:31:56.773 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:56.773 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 301.36240475400001, 2023-12-06T18:31:56.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:31:56.773 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.773 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.773 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.774 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.774 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.774 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:31:56.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.774 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:56.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:31:56.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:31:56.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:31:56.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:31:56.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:31:56.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:31:56.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:31:56.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.776 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:31:56.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:31:56.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.776 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:31:56.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:31:56.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.776 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:31:56.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:31:56.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:31:56.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:31:56.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:31:56.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:31:56.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:31:56.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:31:56.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:31:56.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:31:56.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:31:56.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:31:56.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:31:56.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:31:56.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:31:56.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:31:56.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:31:56.779 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.779 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.779 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.779 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:56.779 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:56.779 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 299.253831515, 2023-12-06T18:31:56.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:31:56.779 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.779 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.779 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.780 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.780 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.780 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:31:56.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.780 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:56.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:31:56.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:31:56.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:31:56.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:31:56.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:31:56.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:31:56.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:31:56.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.782 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.782 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:31:56.782 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:31:56.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.782 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.782 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:31:56.782 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:31:56.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.782 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.782 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:31:56.782 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:31:56.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:31:56.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:31:56.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:31:56.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:31:56.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:31:56.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:31:56.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:31:56.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:31:56.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:31:56.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:31:56.784 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.784 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.784 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.784 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:56.785 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:56.785 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 299.25308628300002, 2023-12-06T18:31:56.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:31:56.785 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.785 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.785 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.785 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.785 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.785 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:31:56.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.785 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.785 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.785 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:56.785 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:31:56.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:31:56.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:31:56.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.786 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.786 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:31:56.786 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:31:56.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.787 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.787 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:31:56.787 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:31:56.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.787 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.787 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:31:56.787 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:31:56.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.787 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.787 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:31:56.787 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:31:56.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.788 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:31:56.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:31:56.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.788 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:31:56.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:31:56.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.788 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:31:56.788 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:31:56.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.788 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:31:56.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:31:56.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:31:56.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:31:56.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.789 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.789 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:31:56.789 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:31:56.789 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.789 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.789 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.790 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.790 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:56.790 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:56.790 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 299.252402528, 2023-12-06T18:31:56.790 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:31:56.790 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.790 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.790 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.790 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.790 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.790 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:31:56.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.790 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:56.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:31:56.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.791 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.791 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:31:56.791 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:31:56.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.792 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.792 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:31:56.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:31:56.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.792 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.792 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:31:56.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:31:56.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.792 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.792 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:31:56.792 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:31:56.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.792 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:31:56.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:31:56.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:31:56.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:31:56.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:31:56.793 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:31:56.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.793 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.793 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:31:56.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:31:56.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:31:56.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:31:56.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:31:56.794 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:31:56.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.794 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.794 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:31:56.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:31:56.795 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.795 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.795 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.795 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:56.795 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:56.795 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 299.246829273, 2023-12-06T18:31:56.795 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:31:56.795 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.795 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.795 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.795 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.795 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.795 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:31:56.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.796 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:56.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:31:56.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.796 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.796 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:31:56.796 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:31:56.796 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:31:56.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:31:56.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:31:56.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:31:56.797 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.797 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.797 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:31:56.797 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:31:56.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:31:56.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:31:56.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:31:56.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:31:56.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.798 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.798 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.798 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:31:56.798 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:31:56.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:31:56.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:31:56.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:31:56.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:31:56.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.799 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.799 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.799 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:31:56.799 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:31:56.799 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.800 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.800 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:31:56.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:31:56.800 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.800 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.800 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.800 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.800 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.800 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:56.800 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:56.800 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 299.246155633, 2023-12-06T18:31:56.800 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:31:56.800 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.800 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.800 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.801 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.801 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.801 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:31:56.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.801 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:56.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.801 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.801 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:31:56.801 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.801 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.801 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.802 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:31:56.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:31:56.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.802 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:31:56.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:31:56.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.802 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.802 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:31:56.802 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:31:56.802 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.802 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:31:56.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:31:56.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:31:56.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:31:56.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.803 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.803 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:31:56.803 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:31:56.803 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.803 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:31:56.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:31:56.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:31:56.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:31:56.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.804 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.804 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:31:56.804 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:31:56.804 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.804 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:31:56.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:31:56.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.805 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.805 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:31:56.805 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:31:56.805 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.805 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.805 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.805 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.805 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.805 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:56.805 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:56.806 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 299.24549343400002, 2023-12-06T18:31:56.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:31:56.806 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.806 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.806 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.806 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.806 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.806 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:31:56.806 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.806 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.806 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.806 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.806 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:56.806 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:31:56.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:31:56.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:31:56.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.807 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.807 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.807 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:31:56.807 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:31:56.807 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.808 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.808 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:31:56.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:31:56.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.808 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.808 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:31:56.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:31:56.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.808 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.808 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:31:56.808 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:31:56.808 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.808 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:31:56.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:31:56.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:31:56.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:31:56.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.809 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.809 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:31:56.809 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:31:56.809 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.809 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:31:56.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:31:56.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:31:56.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:31:56.810 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.810 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.810 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.810 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:31:56.810 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:31:56.810 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.811 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.811 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.811 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.811 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.811 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:56.811 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:56.811 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 299.17278399899999, 2023-12-06T18:31:56.811 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:31:56.811 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.811 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.811 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.811 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.811 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.811 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:31:56.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.812 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:56.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.812 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:31:56.812 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.812 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.812 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.812 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:31:56.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:31:56.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:31:56.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:31:56.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:31:56.813 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:31:56.813 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.813 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.813 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.813 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:31:56.814 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:31:56.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.814 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.814 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:31:56.814 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:31:56.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.814 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.814 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:31:56.814 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:31:56.814 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.814 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.815 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.815 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:31:56.815 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:31:56.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.815 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.815 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.815 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:31:56.815 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:31:56.815 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.815 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.815 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.815 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:31:56.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:31:56.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:31:56.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:31:56.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.816 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.816 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.816 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:31:56.816 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:31:56.816 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.816 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.816 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.816 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.817 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:31:56.817 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:56.817 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 299.17210229, 2023-12-06T18:31:56.817 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:31:56.817 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.817 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.817 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.817 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.817 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.817 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:31:56.817 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.817 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.817 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.817 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:56.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:31:56.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:31:56.818 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:31:56.818 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.818 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.818 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.818 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:31:56.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:31:56.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.819 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.819 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.819 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:31:56.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:31:56.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.819 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.819 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.819 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:31:56.819 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:31:56.819 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.819 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.819 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.819 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:31:56.820 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:31:56.820 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.820 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.820 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.820 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:31:56.820 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:31:56.820 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.820 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.820 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.820 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:31:56.820 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:31:56.820 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.820 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.820 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.820 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:31:56.820 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:31:56.821 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.821 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.821 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.821 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:31:56.821 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:31:56.821 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.821 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.821 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.821 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:31:56.821 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:31:56.821 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.821 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.821 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.821 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:31:56.821 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:31:56.821 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.822 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.822 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.822 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.822 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.822 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:56.822 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:56.822 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 298.09329820200003, 2023-12-06T18:31:56.822 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:31:56.822 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.822 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.822 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.822 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.822 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.822 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:31:56.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.823 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:56.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:31:56.823 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.823 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.823 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.823 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.823 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:31:56.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:31:56.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.824 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:31:56.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:31:56.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.824 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.824 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:31:56.824 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:31:56.824 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.824 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.825 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.825 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:31:56.825 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:31:56.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.825 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.825 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.825 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:31:56.825 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:31:56.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.825 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.825 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.825 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:31:56.825 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:31:56.825 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.825 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.826 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.826 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:31:56.826 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:31:56.826 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.826 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.826 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.826 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:31:56.826 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:31:56.826 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.826 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.826 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.826 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:31:56.826 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:31:56.826 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.826 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.827 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.827 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:31:56.827 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:31:56.827 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.827 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.827 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.827 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:31:56.827 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:31:56.827 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.827 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.827 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.827 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.827 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.827 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:56.827 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:56.827 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 298.09188162800001, 2023-12-06T18:31:56.828 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:31:56.828 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.828 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.828 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.828 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.828 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.828 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:31:56.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.828 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.828 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.828 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:56.828 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.828 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.828 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.828 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:31:56.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.829 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.829 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:31:56.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:31:56.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.829 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.829 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.829 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:31:56.829 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:31:56.829 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.829 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.830 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.830 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:31:56.830 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:31:56.830 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.830 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.830 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.830 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:31:56.830 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:31:56.830 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.830 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.830 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.830 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:31:56.831 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:31:56.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.831 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.831 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.831 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:31:56.831 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:31:56.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.831 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.831 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.831 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:31:56.831 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:31:56.831 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.831 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.831 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.831 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:31:56.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:31:56.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.832 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:31:56.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:31:56.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.832 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:31:56.832 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:31:56.832 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.832 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.832 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.832 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:31:56.833 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:31:56.833 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.833 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.833 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.833 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.833 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.833 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:56.833 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:56.833 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 298.09079650299998, 2023-12-06T18:31:56.833 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:31:56.833 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.833 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.833 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.833 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.833 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.834 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:31:56.834 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.834 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.834 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.834 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.834 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:56.834 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.834 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.834 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.834 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.834 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:31:56.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.835 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.835 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.835 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:31:56.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:31:56.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.835 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.835 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.835 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:31:56.835 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:31:56.835 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.835 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.835 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.836 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:31:56.836 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:31:56.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.836 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.836 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.836 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:31:56.836 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:31:56.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.836 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.836 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.836 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:31:56.836 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:31:56.836 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.836 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.836 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.836 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:31:56.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:31:56.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.837 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.837 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:31:56.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:31:56.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.837 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.837 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:31:56.837 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:31:56.837 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.837 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.837 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:31:56.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:31:56.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:31:56.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:31:56.838 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.838 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.838 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.838 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:31:56.838 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:31:56.838 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.838 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.839 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.839 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.839 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:31:56.839 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:56.839 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 298.05564740099999, 2023-12-06T18:31:56.839 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:31:56.839 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.839 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.839 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.839 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.839 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.839 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:31:56.839 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.839 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.840 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:56.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.840 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:31:56.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.840 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.840 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.840 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:31:56.840 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:31:56.840 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.841 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.841 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.841 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:31:56.841 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:31:56.841 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.841 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.841 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.841 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:31:56.841 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:31:56.841 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.841 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.841 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.841 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:31:56.841 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:31:56.841 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.841 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.842 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.842 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:31:56.842 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:31:56.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.842 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.842 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.842 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:31:56.842 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:31:56.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.842 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.842 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.842 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:31:56.842 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:31:56.842 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:31:56.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:31:56.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:31:56.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:31:56.843 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.843 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.843 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.843 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:31:56.843 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:31:56.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.844 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.844 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:31:56.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:31:56.844 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.844 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.844 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.844 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.844 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.844 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:56.844 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:56.844 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 291.50591687600001, 2023-12-06T18:31:56.844 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:31:56.844 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.844 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.844 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.845 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.845 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.845 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:31:56.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.845 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.845 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:56.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.845 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.845 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:31:56.845 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.845 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.845 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.846 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.846 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:31:56.846 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:31:56.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.846 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.846 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.846 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:31:56.846 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:31:56.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.846 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.846 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.846 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:31:56.846 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:31:56.846 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.847 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.847 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.847 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:31:56.847 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:31:56.847 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.847 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.847 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.847 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:31:56.847 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:31:56.847 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.847 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.847 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.847 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:31:56.847 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:31:56.847 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:31:56.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:31:56.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:31:56.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:31:56.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:31:56.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:31:56.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:31:56.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:31:56.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:31:56.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:31:56.849 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.849 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.849 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.849 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:56.849 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:56.849 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 291.50511126800001, 2023-12-06T18:31:56.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:31:56.850 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.850 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.850 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.850 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.850 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.850 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:31:56.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.850 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:56.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.851 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:31:56.851 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.851 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:31:56.851 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:31:56.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.851 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.851 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:31:56.851 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:31:56.851 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.851 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.852 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.852 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:31:56.852 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:31:56.852 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.852 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.852 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.852 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:31:56.852 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:31:56.852 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.852 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.852 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.852 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:31:56.852 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:31:56.852 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.852 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.852 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.853 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:31:56.853 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:31:56.853 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.853 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.853 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.853 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:31:56.853 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:31:56.853 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.853 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.853 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.853 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:31:56.853 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:31:56.853 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.853 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.853 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.854 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:31:56.854 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:31:56.854 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.854 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.854 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.854 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:31:56.854 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:31:56.854 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.854 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.854 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.854 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:31:56.854 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:31:56.854 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.854 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.854 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.855 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.855 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.855 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:56.855 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:56.855 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 291.50442045900002, 2023-12-06T18:31:56.855 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:31:56.855 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.855 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.855 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.855 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.855 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.855 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:31:56.855 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.855 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.855 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.855 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.856 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:56.856 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.856 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.856 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.856 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.856 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:31:56.856 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.856 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.856 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.856 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.856 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:31:56.856 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:31:56.856 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.856 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.856 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.857 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:31:56.857 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:31:56.857 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.857 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.857 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.857 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:31:56.857 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:31:56.857 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.857 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.857 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.857 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:31:56.857 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:31:56.857 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.857 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.857 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.858 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:31:56.858 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:31:56.858 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.858 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.858 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.858 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:31:56.858 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:31:56.858 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.858 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.858 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.858 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:31:56.858 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:31:56.858 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.858 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.858 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:31:56.859 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:31:56.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.859 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:31:56.859 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:31:56.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.859 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.859 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:31:56.859 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:31:56.859 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.859 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.859 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.860 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:31:56.860 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:31:56.860 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.860 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.860 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.860 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.860 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.860 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:56.860 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:56.860 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 291.50012295099998, 2023-12-06T18:31:56.860 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:31:56.860 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.860 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.860 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.860 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.860 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.861 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:31:56.861 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.861 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.861 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.861 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.861 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:56.861 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.861 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.861 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.861 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.861 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:31:56.861 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.861 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.861 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.861 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.861 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:31:56.862 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:31:56.862 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.862 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.862 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.862 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:31:56.862 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:31:56.862 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.862 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.862 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.862 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:31:56.862 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:31:56.862 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.862 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.862 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.862 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:31:56.862 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:31:56.863 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.863 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.863 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.863 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:31:56.863 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:31:56.863 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.863 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.863 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.863 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:31:56.863 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:31:56.863 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.863 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.863 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.863 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:31:56.863 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:31:56.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.864 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.864 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.864 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:31:56.864 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:31:56.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.864 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.864 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.864 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:31:56.864 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:31:56.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.864 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.864 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.864 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:31:56.864 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:31:56.864 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.865 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.865 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.865 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:31:56.865 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:31:56.865 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.865 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.865 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.865 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.865 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.865 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:56.865 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:56.865 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 291.49945527599999, 2023-12-06T18:31:56.865 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:31:56.865 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.865 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.865 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.865 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.866 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.866 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:31:56.866 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.866 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.866 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.866 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.866 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:56.866 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.866 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.866 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.866 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.866 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:31:56.866 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.866 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.866 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.867 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.867 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:31:56.867 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:31:56.867 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.867 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.867 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.867 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:31:56.867 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:31:56.867 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.867 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.867 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.867 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:31:56.867 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:31:56.867 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.867 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.867 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.868 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:31:56.868 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:31:56.868 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.868 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.868 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.868 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:31:56.868 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:31:56.868 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.868 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.868 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.868 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:31:56.868 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:31:56.868 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.868 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.868 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.868 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:31:56.869 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:31:56.869 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.869 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.869 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.869 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:31:56.869 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:31:56.869 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.869 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.869 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.869 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:31:56.869 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:31:56.869 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.869 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.869 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.869 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:31:56.869 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:31:56.869 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.870 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.870 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.870 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:31:56.870 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:31:56.870 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.870 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.870 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.870 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.870 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.870 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:56.870 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:56.870 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 291.49879984799998, 2023-12-06T18:31:56.870 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:31:56.870 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.871 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.871 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.871 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.871 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.871 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:31:56.871 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.871 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.871 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.871 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.871 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:56.871 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.871 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.871 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.871 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.871 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:31:56.871 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.871 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.872 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.872 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.872 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:31:56.872 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:31:56.872 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.872 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.872 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.872 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:31:56.872 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:31:56.872 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.872 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.872 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.872 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:31:56.872 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:31:56.872 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.872 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.873 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.873 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:31:56.873 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:31:56.873 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.873 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.873 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.873 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:31:56.873 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:31:56.873 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.873 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.873 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.873 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:31:56.873 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:31:56.873 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.873 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.874 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.874 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:31:56.874 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:31:56.874 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.874 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.874 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.874 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:31:56.874 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:31:56.874 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.874 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.874 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.874 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:31:56.874 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:31:56.874 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.874 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.874 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.875 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:31:56.875 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:31:56.875 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.875 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.875 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.875 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:31:56.875 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:31:56.875 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.875 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.875 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.875 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.875 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.875 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:31:56.875 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:56.875 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 291.458145301, 2023-12-06T18:31:56.876 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:31:56.876 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:56.876 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:31:56.876 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:56.876 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:31:56.876 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:31:56.876 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:31:56.876 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.876 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:56.876 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.876 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:56.876 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:56.876 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.876 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.876 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.876 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:56.877 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:31:56.877 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:56.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.877 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.877 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:56.877 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:31:56.877 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:31:56.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.877 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.877 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:56.877 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:31:56.877 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:31:56.877 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.877 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.877 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:56.878 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:31:56.878 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:31:56.878 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.878 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.878 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:56.878 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:31:56.878 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:31:56.878 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.878 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.878 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:56.878 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:31:56.878 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:31:56.878 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.878 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.878 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:56.879 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:31:56.879 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:31:56.879 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.879 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.879 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:31:56.879 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:31:56.879 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:31:56.879 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.879 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.879 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:31:56.879 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:31:56.879 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:31:56.879 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.879 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.879 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:31:56.880 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:31:56.880 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:31:56.880 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.880 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.880 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:31:56.880 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:31:56.880 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:31:56.880 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:56.880 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:56.880 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:56.880 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:31:56.880 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:31:56.880 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.880 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.880 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.880 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:56.881 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:56.881 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:56.882 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:31:56.882 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:56.878+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:56.983 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:56.980+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:57.081 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:57.082 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:57.082 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:57.082 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:57.084 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:57.081+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:57.091 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:31:57.185 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:57.182+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:57.287 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:57.283+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:57.303 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:57.303 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:57.303 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:57.303 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:57.303 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:57.316 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:31:57.327 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '5 of 30' 2023-12-06T18:31:57.387 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:57.384+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:57.488 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:57.485+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:57.511 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:57.511 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:57.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:57.511 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:57.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:57.511 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:31:57.511 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:57.511 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 275.69500036599999, 2023-12-06T18:31:57.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:31:57.511 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:57.512 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:31:57.512 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:57.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:57.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:57.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:57.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:57.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:57.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:57.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:57.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:31:57.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:57.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:57.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:57.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:57.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:31:57.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:31:57.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:57.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:57.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:57.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:31:57.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:31:57.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:57.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:57.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:57.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:31:57.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:31:57.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:57.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:57.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:57.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:31:57.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:31:57.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:57.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:57.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:57.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:31:57.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:31:57.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:57.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:57.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:31:57.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:31:57.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:31:57.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:57.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:57.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:57.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:31:57.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:31:57.515 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:57.515 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:57.515 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:57.515 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:57.515 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:57.515 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:57.520 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:31:57.605 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:57.601+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:57.691 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:57.687+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:57.698 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:57.699 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:57.699 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:31:57.699 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:57.708 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:31:57.793 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:57.789+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:57.888 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:57.888 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:31:57.889 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:31:57.889 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:31:57.889 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:57.892 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:57.889+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:57.898 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:31:57.993 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:57.990+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:57.997 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:31:58.084 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:31:58.085 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:31:58.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:31:58.085 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:31:58.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:58.085 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:31:58.085 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:58.085 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 156.98502536999999, 2023-12-06T18:31:58.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:31:58.085 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:58.085 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:58.085 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:58.085 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:58.085 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:58.085 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:31:58.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:58.086 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:58.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:58.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:58.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:58.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:58.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:58.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:58.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:58.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:31:58.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:58.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:58.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:58.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:58.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:31:58.087 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:31:58.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:58.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:58.087 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:58.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:31:58.087 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:31:58.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:58.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:58.087 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:58.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:31:58.087 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:31:58.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:58.088 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:58.088 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:58.088 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:31:58.088 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:31:58.088 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:58.088 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:58.088 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:58.088 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:31:58.088 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:31:58.088 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:58.088 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:58.088 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:58.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:31:58.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:31:58.089 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:58.089 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:58.089 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:58.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:58.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:58.089 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:31:58.089 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:58.089 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 96.966768067999993, 2023-12-06T18:31:58.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:31:58.089 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:31:58.089 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:31:58.089 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:31:58.089 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:31:58.089 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:31:58.089 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:31:58.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:58.090 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:31:58.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:58.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:31:58.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:58.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:58.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:58.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:58.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:31:58.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:31:58.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:31:58.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:58.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:58.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:31:58.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:31:58.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:31:58.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:58.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:58.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:31:58.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:31:58.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:31:58.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:58.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:58.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:31:58.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:31:58.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:31:58.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:58.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:58.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:31:58.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:31:58.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:31:58.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:58.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:58.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:31:58.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:31:58.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:31:58.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:31:58.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:31:58.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:31:58.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:31:58.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:31:58.092 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:58.092 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:58.092 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:58.092 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:31:58.092 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:31:58.092 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:31:58.094 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:31:58.095 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:58.090+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:58.195 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:58.191+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:58.235 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:58.235 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:58.235 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:31:58.235 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:58.246 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:31:58.251 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:31:58.251 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:31:58.330 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:58.326+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:58.346 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:31:58.346 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:31:58.396 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:58.393+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:58.431 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:31:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:31:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:31:58.431 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:31:58.431 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:31:58.441 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:31:58.448 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '6 of 30' 2023-12-06T18:31:58.448 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:31:58.448 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:31:58.511 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-12-06T18:31:58.526 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:58.522+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:58.545 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:31:58.546 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:31:58.644 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:31:58.644 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:31:58.644 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:31:58.640+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:31:58.645 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:58.645 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:58.657 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:31:58.701 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:58.698+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:58.748 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:31:58.748 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:31:58.801 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:31:58.796+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:31:58.801 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:58.801 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:58.811 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:31:58.840 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:31:58.840 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:31:58.947 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:31:58.948 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:31:58.948 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:58.944+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:58.999 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:58.999 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:59.002 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:58.998+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:59.008 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:31:59.103 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:31:59.100+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:31:59.137 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:59.138 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:59.146 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:31:59.159 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:31:59.159 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:31:59.159 INFO:tasks.thrashosds.thrasher:primary_affinity 2023-12-06T18:31:59.159 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.720809 2023-12-06T18:31:59.159 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 0.7208093371139873 2023-12-06T18:31:59.205 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:31:59.201+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:31:59.306 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:59.302+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:59.315 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:59.315 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:59.324 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:31:59.407 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:31:59.403+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:31:59.454 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:59.454 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:59.464 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:31:59.508 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:59.504+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:59.609 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:31:59.605+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:31:59.634 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:59.634 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:59.643 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:31:59.651 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is unset 2023-12-06T18:31:59.692 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '7 of 30' 2023-12-06T18:31:59.734 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:31:59.730+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:31:59.811 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:59.807+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:59.811 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:59.811 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:59.821 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:31:59.911 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:31:59.908+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:31:59.959 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:31:59.959 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:31:59.969 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:32:00.014 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:00.011+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:00.098 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:00.098 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:00.108 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:32:00.150 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:00.146+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:00.214 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:00.210+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:00.247 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:00.247 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:00.257 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:32:00.315 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:00.311+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:00.395 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:00.395 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:00.404 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:32:00.416 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:00.413+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:00.517 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:00.513+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:00.581 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:00.581 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:00.590 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:32:00.632 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:00.628+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:00.709 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '8 of 30' 2023-12-06T18:32:00.718 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:00.715+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:00.808 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:00.808 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:00.818 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:32:00.820 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:00.816+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:00.847 INFO:teuthology.orchestra.run.smithi061.stderr:set osd.4 primary-affinity to 0.720809 (8472382) 2023-12-06T18:32:00.920 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:00.917+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:00.961 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:32:00.964 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:00.964 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:00.974 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:32:01.058 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:01.054+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:01.122 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:01.118+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:01.165 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:01.166 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:01.175 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:32:01.244 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:00.975 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:01.245 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:01.242+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:01.248 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:00.979 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:01.275 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:01.006 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:01.308 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:40.753 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:01.317 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:40.762 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:01.355 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:40.801 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:01.359 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:01.355+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:01.375 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:01.376 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:01.388 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:32:01.431 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:01.426+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:01.453 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:32:01.453 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:32:01.532 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:01.528+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:01.600 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:32:01.600 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:32:01.627 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:01.623+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:01.629 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:01.629 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:01.641 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:32:01.667 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-12-06T18:32:01.692 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:32:01.693 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:32:01.730 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '9 of 30' 2023-12-06T18:32:01.731 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:01.727+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:01.813 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:32:01.813 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:32:01.828 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:01.824+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:01.846 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:01.847 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:01.847 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:01.847 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:32:01.847 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:01.847 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:32:01.847 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:01.847 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 280.03084713599998, 2023-12-06T18:32:01.847 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:32:01.847 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:01.847 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:32:01.847 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:01.847 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:01.847 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:01.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:01.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:01.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:01.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:01.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:01.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:01.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:01.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:01.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:01.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:01.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:32:01.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:32:01.848 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:01.848 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:01.848 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:01.848 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:32:01.848 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:32:01.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:01.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:01.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:01.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:32:01.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:32:01.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:01.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:01.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:01.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:32:01.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:32:01.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:01.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:01.849 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:01.849 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:32:01.849 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:32:01.849 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:01.849 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:01.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:01.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:32:01.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:32:01.850 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:01.850 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:01.850 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:01.850 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:32:01.850 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:32:01.850 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:01.850 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:01.850 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:01.850 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:01.850 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:01.850 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:01.857 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:32:01.907 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:32:01.908 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:32:01.929 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:01.926+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:02.015 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:32:02.015 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:32:02.029 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:02.026+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:02.037 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:02.037 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:02.037 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:02.037 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:02.046 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:32:02.100 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:32:02.101 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:32:02.130 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:02.127+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:02.188 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:32:02.188 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:32:02.247 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:02.243+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:02.247 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:02.247 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:02.247 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:02.247 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:02.248 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:02.257 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:32:02.341 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:02.337+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:02.434 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:02.430+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:02.439 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:02.439 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:02.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:02.439 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:32:02.440 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:02.440 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:32:02.440 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:32:02.440 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 161.339994265, 2023-12-06T18:32:02.440 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:32:02.440 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:02.440 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:32:02.440 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:02.440 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:32:02.440 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:02.440 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:32:02.440 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:02.440 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:02.440 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:32:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:32:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:32:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:32:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:02.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:32:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:32:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:32:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:32:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:32:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:32:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:02.442 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:02.443 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:02.443 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:32:02.443 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:32:02.443 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:02.443 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:02.443 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:02.443 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:32:02.443 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:32:02.443 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:02.443 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:02.443 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:02.443 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:02.443 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:02.443 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:32:02.444 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:32:02.444 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 101.32173696300001, 2023-12-06T18:32:02.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:32:02.444 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:02.444 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:32:02.444 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:02.444 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:32:02.444 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:02.444 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:32:02.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:02.444 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:02.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:02.444 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:02.444 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:32:02.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:02.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:02.445 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:02.445 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:02.445 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:32:02.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:02.445 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:02.445 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:02.445 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:02.445 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:32:02.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:32:02.445 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:02.445 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:02.445 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:02.445 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:32:02.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:32:02.445 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:02.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:02.446 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:02.446 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:32:02.446 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:32:02.446 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:02.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:02.446 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:02.446 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:32:02.446 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:32:02.446 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:02.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:02.446 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:02.446 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:32:02.446 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:32:02.447 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:02.447 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:02.447 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:02.447 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:32:02.447 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:32:02.447 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:02.447 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:02.447 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:02.447 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:02.447 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:02.447 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:02.449 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:32:02.534 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:02.531+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:02.580 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:02.580 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:02.580 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:02.580 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:02.590 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:32:02.636 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:02.632+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:02.737 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:02.733+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:02.758 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:02.758 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:02.758 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:02.759 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:02.759 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:02.768 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:32:02.837 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:02.833+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:02.839 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is set 2023-12-06T18:32:02.939 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:02.935+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:02.940 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:02.940 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:02.940 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:02.941 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:02.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:02.941 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:32:02.941 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:02.941 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 221.44722905399999, 2023-12-06T18:32:02.941 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:32:02.941 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:02.941 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:32:02.941 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:32:02.941 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:32:02.941 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:02.941 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:32:02.941 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:02.941 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:02.941 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:02.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:02.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:02.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:02.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:02.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:02.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:02.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:02.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:02.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:02.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:02.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:02.942 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:32:02.942 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:32:02.942 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:02.942 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:02.942 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:02.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:32:02.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:32:02.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:02.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:02.943 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:02.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:32:02.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:32:02.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:02.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:02.943 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:02.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:32:02.943 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:32:02.943 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:02.943 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:02.943 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:02.943 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:32:02.944 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:32:02.944 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:02.944 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:02.944 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:02.944 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:32:02.944 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:32:02.944 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:02.944 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:02.944 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:02.944 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:02.944 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:02.944 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:02.951 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:32:03.039 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:03.036+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:03.079 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:03.079 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:03.079 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:03.079 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:03.089 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:32:03.141 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:03.137+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:03.220 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:03.220 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:03.220 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:03.220 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:03.220 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:03.230 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:32:03.242 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:03.238+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:03.342 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:03.339+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:03.437 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:03.437 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:03.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:03.437 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:03.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.437 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:32:03.437 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:03.437 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 306.82158378700001, 2023-12-06T18:32:03.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:32:03.437 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.437 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.437 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:03.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:03.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:32:03.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:32:03.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:32:03.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:32:03.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:32:03.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:32:03.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:32:03.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:32:03.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:32:03.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:32:03.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:32:03.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:32:03.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:32:03.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:32:03.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:32:03.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:32:03.441 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.441 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.441 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.441 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:32:03.441 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:03.441 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 305.79875677799998, 2023-12-06T18:32:03.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:32:03.442 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.442 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.442 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:03.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:03.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:32:03.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:32:03.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:32:03.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:32:03.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:32:03.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:32:03.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:32:03.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:32:03.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:32:03.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:32:03.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:32:03.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:32:03.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:32:03.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:32:03.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:32:03.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:32:03.445 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.445 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.445 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.445 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:32:03.445 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:03.445 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 305.78671014000003, 2023-12-06T18:32:03.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:32:03.446 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.446 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.446 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:03.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:03.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:32:03.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:32:03.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:32:03.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:32:03.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:32:03.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:32:03.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.448 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:32:03.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:32:03.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.448 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:32:03.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:32:03.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:32:03.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:32:03.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:32:03.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:32:03.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:32:03.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:32:03.449 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.449 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.450 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.450 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:32:03.450 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:03.450 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 305.76530953299999, 2023-12-06T18:32:03.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:32:03.450 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.450 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.450 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:03.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:03.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:32:03.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:32:03.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:32:03.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:32:03.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:32:03.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:32:03.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:32:03.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:32:03.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:32:03.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:32:03.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:32:03.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:32:03.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:32:03.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:32:03.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:32:03.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:32:03.454 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.454 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.454 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.454 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:32:03.454 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:03.454 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 305.76413021000002, 2023-12-06T18:32:03.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:32:03.454 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.454 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.454 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:03.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:03.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:32:03.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:32:03.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:32:03.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:32:03.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:32:03.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:32:03.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:32:03.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:32:03.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:32:03.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:32:03.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:32:03.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:32:03.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:32:03.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:32:03.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:32:03.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:32:03.457 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.457 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.457 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.458 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:32:03.458 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:03.458 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 305.76355822900001, 2023-12-06T18:32:03.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:32:03.458 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.458 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.458 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:03.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:03.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:32:03.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:32:03.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:32:03.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:32:03.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:32:03.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:32:03.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:32:03.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:32:03.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:32:03.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:32:03.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:32:03.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:32:03.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:32:03.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:32:03.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:32:03.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:32:03.462 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.462 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.462 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.462 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:32:03.462 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:03.462 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 305.31772275899999, 2023-12-06T18:32:03.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:32:03.462 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.463 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.463 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:03.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:03.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:32:03.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:32:03.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:32:03.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:32:03.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:32:03.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:32:03.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:32:03.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:32:03.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:32:03.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:32:03.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:32:03.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:32:03.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:32:03.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:32:03.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:32:03.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:32:03.466 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.466 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.466 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.466 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:32:03.466 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:03.466 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 305.31612290300001, 2023-12-06T18:32:03.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:32:03.466 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.467 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.467 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:03.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:03.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:32:03.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:32:03.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:32:03.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:32:03.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:32:03.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:32:03.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:32:03.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:32:03.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:32:03.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:32:03.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:32:03.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:32:03.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:32:03.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:32:03.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.470 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:32:03.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:32:03.470 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.470 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.470 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.470 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:32:03.470 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:03.470 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 304.93691728700003, 2023-12-06T18:32:03.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:32:03.470 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.470 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.470 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:03.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:03.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:32:03.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:32:03.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:32:03.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:32:03.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:32:03.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:32:03.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:32:03.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:32:03.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:32:03.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:32:03.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:32:03.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:32:03.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:32:03.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:32:03.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:32:03.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:32:03.474 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.474 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.474 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.474 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:32:03.475 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:03.475 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 304.69941510500001, 2023-12-06T18:32:03.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:32:03.475 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.475 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.475 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:03.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:03.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:32:03.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:32:03.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:32:03.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:32:03.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:32:03.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:32:03.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:32:03.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:32:03.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:32:03.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:32:03.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:32:03.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:32:03.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:32:03.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:32:03.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:32:03.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:32:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:32:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 304.69868710600002, 2023-12-06T18:32:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:32:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:32:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:32:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:32:03.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:32:03.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:32:03.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:32:03.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:32:03.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:32:03.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:32:03.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:32:03.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:32:03.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:32:03.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:32:03.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:32:03.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:32:03.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:32:03.483 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.483 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.483 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.483 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:32:03.483 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:03.483 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 304.69367469600002, 2023-12-06T18:32:03.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:32:03.483 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.483 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.483 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:03.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:03.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:32:03.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:32:03.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:32:03.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:32:03.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:32:03.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:32:03.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:32:03.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:32:03.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:32:03.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:32:03.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:32:03.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:32:03.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.487 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.487 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:32:03.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:32:03.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.487 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.487 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:32:03.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:32:03.487 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.487 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.487 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.487 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:32:03.488 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:03.488 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 304.68814761700003, 2023-12-06T18:32:03.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:32:03.488 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.488 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.488 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:03.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:03.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:32:03.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:32:03.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:32:03.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:32:03.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:32:03.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:32:03.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:32:03.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:32:03.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:32:03.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:32:03.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.491 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:32:03.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:32:03.491 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.491 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.491 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:32:03.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:32:03.491 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.491 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.491 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:32:03.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:32:03.491 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.491 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.492 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.492 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:32:03.492 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:03.492 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 304.68763677999999, 2023-12-06T18:32:03.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:32:03.492 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.492 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.492 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:03.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:03.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:32:03.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:32:03.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:32:03.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:32:03.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:32:03.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:32:03.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:32:03.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:32:03.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:32:03.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:32:03.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.495 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:32:03.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:32:03.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.495 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:32:03.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:32:03.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.495 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:32:03.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:32:03.496 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.496 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.496 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.496 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:32:03.496 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:03.496 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 304.68717751899999, 2023-12-06T18:32:03.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:32:03.496 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.496 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.496 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:03.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:03.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:32:03.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:32:03.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:32:03.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:32:03.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:32:03.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:32:03.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:32:03.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:32:03.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.499 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.499 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:32:03.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:32:03.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.499 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.499 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:32:03.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:32:03.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.499 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.499 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:32:03.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:32:03.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:32:03.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:32:03.500 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.500 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.500 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.500 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:32:03.500 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:03.500 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 298.10726171200002, 2023-12-06T18:32:03.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:32:03.500 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.500 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.501 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:03.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:03.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:32:03.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:32:03.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:32:03.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:32:03.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:32:03.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:32:03.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:32:03.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:32:03.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:32:03.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:32:03.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.503 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:32:03.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:32:03.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:32:03.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:32:03.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:32:03.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:32:03.504 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.504 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.504 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.505 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:32:03.505 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:03.505 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 298.10102390700001, 2023-12-06T18:32:03.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:32:03.505 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.505 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.505 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:03.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:03.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:32:03.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:32:03.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:32:03.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:32:03.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:32:03.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:32:03.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:32:03.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:32:03.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:32:03.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:32:03.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:32:03.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:32:03.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:32:03.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:32:03.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:32:03.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:32:03.509 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.509 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.509 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.509 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:32:03.509 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:03.509 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 298.09496082800001, 2023-12-06T18:32:03.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:32:03.509 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.509 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.509 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:03.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:03.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:32:03.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:32:03.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:32:03.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:32:03.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:32:03.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:32:03.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:32:03.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:32:03.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:32:03.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:32:03.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:32:03.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:32:03.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:32:03.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:32:03.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:32:03.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:32:03.513 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.513 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.513 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.513 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:32:03.513 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:03.513 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 297.95757374300001, 2023-12-06T18:32:03.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:32:03.514 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.514 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.514 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:03.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:03.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:32:03.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:32:03.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:32:03.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:32:03.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:32:03.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:32:03.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:32:03.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:32:03.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:32:03.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:32:03.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:32:03.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:32:03.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:32:03.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:32:03.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:32:03.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:32:03.517 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.517 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.517 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.518 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:32:03.518 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:03.518 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 297.95090301599998, 2023-12-06T18:32:03.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:32:03.518 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:03.518 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:03.518 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:03.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:03.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:03.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:03.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:03.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:03.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:03.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:32:03.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:32:03.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:03.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:32:03.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:32:03.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:03.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:32:03.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:32:03.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:03.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:32:03.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:32:03.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:03.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:32:03.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:32:03.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:03.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:32:03.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:32:03.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:03.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:32:03.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:32:03.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:03.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:03.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:03.522 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:32:03.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:32:03.522 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.522 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.522 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.522 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:03.522 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:03.522 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:03.523 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:32:03.524 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:03.520+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:03.625 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:03.621+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:03.654 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:03.654 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:03.654 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:03.654 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:03.664 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:32:03.726 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:03.723+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:03.794 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:03.794 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:03.794 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:03.794 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:03.794 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:03.804 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:32:03.847 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:03.844+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:03.927 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:03.924+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:03.928 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '10 of 30' 2023-12-06T18:32:03.931 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:03.931 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:03.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:03.931 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:32:03.931 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:03.940 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:32:04.028 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:04.024+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:04.107 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:04.108 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:04.108 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:04.108 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:04.118 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:32:04.128 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:04.125+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:04.205 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:32:04.249 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:04.246+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:04.252 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:04.252 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:04.252 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:04.252 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:04.252 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:04.262 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:32:04.330 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:04.327+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:04.393 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:04.393 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:04.393 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:04.393 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:04.394 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:04.394 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:32:04.394 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:32:04.394 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 102.731412574, 2023-12-06T18:32:04.394 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:32:04.394 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:04.394 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:32:04.394 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:32:04.394 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:32:04.394 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:04.394 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:32:04.394 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:04.395 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:04.395 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:04.395 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:04.395 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:32:04.395 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:04.395 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:04.395 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:04.395 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:04.395 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:32:04.395 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:04.395 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:04.395 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:04.395 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:04.395 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:32:04.396 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:32:04.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:04.396 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:04.396 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:04.396 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:32:04.396 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:32:04.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:04.396 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:04.396 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:04.396 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:32:04.396 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:32:04.396 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:04.396 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:04.396 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:04.396 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:32:04.396 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:32:04.397 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:04.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:04.397 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:04.397 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:32:04.397 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:32:04.397 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:04.397 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:04.397 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:04.397 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:32:04.397 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:32:04.397 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:04.397 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:04.397 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:04.397 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:04.397 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:04.397 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:04.402 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:32:04.448 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:04.444+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:04.537 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:04.533+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:04.551 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:32:04.551 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:32:04.604 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:04.604 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:04.616 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:32:04.657 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:04.653+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:04.666 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:32:04.666 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:32:04.733 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:04.730+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:04.762 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:32:04.762 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:32:04.798 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:04.798 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:04.808 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:32:04.834 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:04.830+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:04.854 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-12-06T18:32:04.855 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:32:04.856 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:32:04.935 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:04.932+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:04.956 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:32:04.957 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:32:04.999 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:04.999 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:05.009 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:32:05.035 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:05.032+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:05.047 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:32:05.048 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:32:05.076 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '11 of 30' 2023-12-06T18:32:05.132 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:32:05.133 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:32:05.136 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:05.133+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:05.190 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:05.190 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:05.200 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:32:05.235 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:32:05.236 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:32:05.238 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:05.235+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:05.339 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:05.335+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:05.377 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:05.377 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:05.386 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:32:05.471 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:05.467+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:05.540 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:05.537+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:05.564 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:05.564 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:05.575 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:32:05.641 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:05.638+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:05.743 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:05.738+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:05.752 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:05.752 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:05.762 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:32:05.843 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:05.839+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:05.863 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:32:05.863 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:32:05.943 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:05.940+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:05.945 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:05.946 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:05.955 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:32:06.044 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:06.041+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:06.062 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '12 of 30' 2023-12-06T18:32:06.145 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:06.142+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:06.200 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:06.200 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:06.213 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:32:06.229 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is set 2023-12-06T18:32:06.246 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:06.242+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:06.347 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:06.344+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:06.370 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:06.371 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:06.380 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:32:06.450 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:06.447+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:06.526 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:06.526 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:06.538 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:32:06.549 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:06.545+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:06.650 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:06.647+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:06.680 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:06.680 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:06.689 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:32:06.750 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:06.747+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:06.827 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:06.827 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:06.836 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:32:06.879 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:06.876+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:06.952 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:06.949+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:06.967 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:06.967 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:06.976 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:32:07.061 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:07.057+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:07.072 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '13 of 30' 2023-12-06T18:32:07.144 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:07.144 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:07.154 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:32:07.156 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:07.153+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:07.249 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:32:07.294 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:07.290+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:07.323 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:07.323 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:07.332 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:32:07.356 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:07.352+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:07.457 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:07.453+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:07.460 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:07.460 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:07.470 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:32:07.504 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:32:07.505 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:32:07.558 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:07.555+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:07.597 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:32:07.598 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:32:07.598 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:07.598 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:07.607 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:32:07.690 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:32:07.691 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:32:07.691 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:07.688+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:07.760 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:07.756+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:07.776 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:07.776 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:07.786 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:32:07.797 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:32:07.797 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:32:07.861 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:07.857+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:07.887 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:32:07.887 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:32:07.959 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:07.959 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:07.962 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:07.958+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:07.968 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:32:07.979 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:32:07.979 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:32:08.063 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:08.059+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:08.068 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:32:08.068 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:32:08.107 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:08.107 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:08.107 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:08.107 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:32:08.107 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:08.117 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:32:08.160 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:32:08.160 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:32:08.164 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '14 of 30' 2023-12-06T18:32:08.164 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:08.160+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:08.246 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-12-06T18:32:08.246 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:08.247 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:08.247 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:08.247 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:08.256 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:32:08.301 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:08.297+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:08.365 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:08.362+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:08.384 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:08.385 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:08.385 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:08.385 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:08.385 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:08.394 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:32:08.467 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:08.463+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:08.564 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:08.564 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:08.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:08.564 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:08.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:08.564 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:32:08.564 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:32:08.564 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 106.90212886099999, 2023-12-06T18:32:08.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:32:08.564 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:08.564 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:32:08.564 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:32:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:32:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:32:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:32:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:32:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:08.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:08.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:08.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:32:08.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:32:08.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:08.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:08.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:08.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:32:08.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:32:08.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:08.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:08.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:08.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:32:08.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:32:08.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:08.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:08.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:08.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:32:08.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:32:08.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:08.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:08.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:08.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:32:08.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:32:08.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:08.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:08.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:08.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:32:08.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:32:08.567 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:08.568 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:08.568 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:08.568 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:08.568 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:08.568 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:08.569 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:08.566+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:08.572 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:32:08.670 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:08.667+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:08.717 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:08.717 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:08.717 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:08.717 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:08.726 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:32:08.771 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:08.768+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:08.872 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:08.868+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:08.918 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:08.918 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:08.918 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:08.918 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:08.918 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:08.927 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:32:08.973 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:08.969+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:09.065 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:09.065 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:09.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:09.065 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:32:09.065 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:09.074 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:09.070+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:09.075 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:32:09.159 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '15 of 30' 2023-12-06T18:32:09.202 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:09.197+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:09.276 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:09.273+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:09.276 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:09.277 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:09.277 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:09.277 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:09.289 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:32:09.376 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:09.372+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:09.430 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:09.430 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:09.430 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:09.430 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:09.430 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:09.439 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:32:09.482 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:09.477+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:09.578 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:09.574+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:09.677 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:32:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:32:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:32:09.677 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 314.265560935, 2023-12-06T18:32:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:32:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:32:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:32:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:32:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:32:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:32:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:32:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:32:09.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:32:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:32:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:32:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:32:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:32:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:32:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:32:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:32:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:32:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:32:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:32:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:32:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:32:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:32:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:32:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:32:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:32:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:32:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:32:09.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:32:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:32:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:32:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:32:09.683 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:32:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 312.15698769599999, 2023-12-06T18:32:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:32:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:32:09.684 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:32:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:32:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:32:09.685 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:32:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:32:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:32:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:32:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:32:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.686 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:32:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:32:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:32:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:32:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:32:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:32:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:32:09.687 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:32:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:32:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:32:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:32:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:32:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:32:09.688 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:32:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:32:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:32:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:32:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 312.156242464, 2023-12-06T18:32:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:32:09.689 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:32:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:32:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.690 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:32:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:32:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:32:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:32:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:32:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.691 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:32:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:32:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:32:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:32:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:32:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:32:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.692 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:32:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:32:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:32:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:32:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:32:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:32:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.693 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:32:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:32:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:32:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:32:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:32:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:32:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.694 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:32:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 312.15555870899999, 2023-12-06T18:32:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:32:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:32:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.695 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:32:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:32:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:32:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:32:09.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:32:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:32:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:32:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:32:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:32:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:32:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:32:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:32:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:32:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:32:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:32:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:32:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:32:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:32:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:32:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:32:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:32:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:32:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:32:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:32:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:32:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 312.14998545399999, 2023-12-06T18:32:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:32:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.700 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:32:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:32:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:32:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:32:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:32:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:32:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:32:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:32:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:32:09.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:32:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:32:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:32:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:32:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:32:09.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:32:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:32:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:32:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:32:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:32:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:32:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:32:09.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:32:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:32:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:32:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:32:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:09.705 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:32:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 312.14931181399999, 2023-12-06T18:32:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:32:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:32:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:32:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:32:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:32:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:32:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:32:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:32:09.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:32:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:32:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:32:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:32:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:32:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:32:09.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:32:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:32:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:32:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:32:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:32:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:32:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:32:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:32:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:32:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:32:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:32:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:32:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.710 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:32:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 312.14864961500001, 2023-12-06T18:32:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:32:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:32:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:32:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:32:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:32:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:32:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.712 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:32:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:32:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:32:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:32:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:32:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:32:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:32:09.713 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:32:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:32:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:32:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:32:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:32:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:32:09.714 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:32:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:32:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:32:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:32:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:32:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:32:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:32:09.715 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:32:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 312.07594017999998, 2023-12-06T18:32:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:32:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:32:09.716 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:32:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:32:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:32:09.717 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:32:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:32:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:32:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:32:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:32:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:32:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:32:09.718 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:32:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:32:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:32:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:32:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:32:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:32:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.719 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:32:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:32:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:32:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:32:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:32:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:32:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.720 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:32:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:32:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:32:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 312.07525847099998, 2023-12-06T18:32:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:32:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.721 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:32:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:32:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:32:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.722 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:32:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:32:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:32:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:32:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:32:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:32:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.723 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:32:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:32:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:32:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:32:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:32:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:32:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:32:09.724 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:32:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:32:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:32:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:32:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:32:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:32:09.725 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:32:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:32:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:32:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:32:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 310.99645438300001, 2023-12-06T18:32:09.726 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:32:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:32:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:32:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.727 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:32:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:32:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:32:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:32:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:32:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.728 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:32:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:32:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:32:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:32:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:32:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:32:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.729 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:32:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:32:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:32:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:32:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:32:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:32:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.730 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:32:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:32:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:32:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:32:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:32:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:32:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.731 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:32:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 310.995037809, 2023-12-06T18:32:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:32:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:32:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.732 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:32:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:32:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:32:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:32:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.733 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:32:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:32:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:32:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:32:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:32:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:32:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.734 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:32:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:32:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:32:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:32:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:32:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:32:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:32:09.735 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:32:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:32:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:32:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:32:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:32:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:32:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:32:09.736 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:32:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 310.99395268400002, 2023-12-06T18:32:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:32:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:32:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.737 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:32:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:32:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:32:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:32:09.738 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:32:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:32:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:32:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:32:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:32:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:32:09.739 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:32:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:32:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:32:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:32:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:32:09.740 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:32:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:32:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:32:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:32:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:32:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:32:09.741 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:32:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:32:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:32:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:32:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 310.95880358199997, 2023-12-06T18:32:09.742 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:32:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:32:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:32:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.743 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:32:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:32:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:32:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:32:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:32:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.744 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:32:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:32:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:32:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:32:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:32:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:32:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.745 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:32:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:32:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:32:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:32:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:32:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:32:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.746 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:32:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:32:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:32:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:32:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:32:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:32:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.747 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:32:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 304.409073057, 2023-12-06T18:32:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:32:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:32:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:32:09.748 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.749 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.749 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.749 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:32:09.749 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.749 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.749 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.749 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:32:09.749 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:32:09.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.749 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.749 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.749 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:32:09.749 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:32:09.749 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.750 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.750 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.750 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:32:09.750 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:32:09.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.750 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.750 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.750 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:32:09.750 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:32:09.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.750 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.750 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.750 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:32:09.750 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:32:09.750 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.751 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.751 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:32:09.751 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:32:09.751 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.751 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.751 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:32:09.751 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:32:09.751 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.751 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.751 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:32:09.751 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:32:09.751 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:32:09.752 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:32:09.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:32:09.752 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:32:09.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.752 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.752 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.752 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:32:09.752 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:32:09.752 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.753 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.753 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.753 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.753 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:09.753 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:32:09.753 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 304.40826744899999, 2023-12-06T18:32:09.753 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:32:09.753 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.753 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.753 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.753 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.753 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.753 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:32:09.753 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.753 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.754 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.754 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.754 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:32:09.754 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.754 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.754 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.754 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:32:09.754 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.754 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.754 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.754 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:32:09.754 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:32:09.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.755 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.755 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.755 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:32:09.755 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:32:09.755 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.755 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.755 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.755 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:32:09.755 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:32:09.755 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.755 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.755 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.755 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:32:09.755 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:32:09.755 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.756 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.756 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.756 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:32:09.756 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:32:09.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.756 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.756 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.756 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:32:09.756 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:32:09.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.756 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.756 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.756 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:32:09.756 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:32:09.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.756 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:32:09.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:32:09.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:32:09.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:32:09.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:32:09.757 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:32:09.757 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.757 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.757 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.757 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:32:09.758 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:32:09.758 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.758 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.758 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.758 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.758 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.758 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:09.758 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:32:09.758 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 304.40757664, 2023-12-06T18:32:09.758 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:32:09.758 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.758 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.758 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.758 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.759 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.759 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:32:09.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.759 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:32:09.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.759 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.759 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.759 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:32:09.759 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.759 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.760 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.760 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.760 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:32:09.760 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:32:09.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.760 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.760 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.760 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:32:09.760 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:32:09.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.760 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.760 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.760 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:32:09.760 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:32:09.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.761 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.761 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.761 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:32:09.761 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:32:09.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.761 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.761 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.761 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:32:09.761 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:32:09.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.761 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.761 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.761 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:32:09.761 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:32:09.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.762 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.762 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:32:09.762 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:32:09.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.762 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.762 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:32:09.762 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:32:09.762 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.762 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.762 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:32:09.762 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:32:09.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.763 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:32:09.763 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:32:09.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.763 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.763 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:32:09.763 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:32:09.763 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.763 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.763 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.763 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:09.764 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:32:09.764 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 304.40327913200002, 2023-12-06T18:32:09.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:32:09.764 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.764 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.764 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.764 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.764 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.764 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:32:09.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.764 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.764 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.764 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:32:09.764 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.765 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:32:09.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.765 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:32:09.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:32:09.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.765 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.765 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:32:09.765 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:32:09.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.766 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.766 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.766 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:32:09.766 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:32:09.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.766 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.766 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.766 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:32:09.766 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:32:09.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.766 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.766 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.766 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:32:09.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:32:09.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.767 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:32:09.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:32:09.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.767 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:32:09.767 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:32:09.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.767 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.767 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:32:09.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:32:09.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:32:09.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:32:09.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:32:09.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:32:09.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.768 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.768 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:32:09.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:32:09.769 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.769 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.769 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.769 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:09.769 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:32:09.769 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 304.40261145699998, 2023-12-06T18:32:09.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:32:09.769 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.769 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.769 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.769 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.769 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.769 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:32:09.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.770 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:32:09.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:32:09.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:32:09.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:32:09.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:32:09.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:32:09.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:32:09.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:32:09.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:32:09.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:32:09.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.772 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:32:09.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:32:09.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.772 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:32:09.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:32:09.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.772 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:32:09.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:32:09.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:32:09.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:32:09.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:32:09.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:32:09.773 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:32:09.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:32:09.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:32:09.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:32:09.774 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.774 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.774 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.774 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:09.774 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:32:09.774 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 304.40195602900002, 2023-12-06T18:32:09.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:32:09.774 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.775 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.775 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.775 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.775 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.775 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:32:09.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.775 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:32:09.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:32:09.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.776 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:32:09.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:32:09.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.776 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:32:09.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:32:09.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.776 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:32:09.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:32:09.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:32:09.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:32:09.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:32:09.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:32:09.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.777 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.777 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:32:09.777 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:32:09.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:32:09.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:32:09.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:32:09.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:32:09.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.778 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.778 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:32:09.778 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:32:09.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:32:09.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:32:09.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.779 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.779 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:32:09.779 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:32:09.779 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.779 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.779 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.779 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:09.780 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:32:09.780 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 304.36130148199999, 2023-12-06T18:32:09.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:32:09.780 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:09.780 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:09.780 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:09.780 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:09.780 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:09.780 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:32:09.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.780 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:09.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.780 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:09.780 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:32:09.780 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.780 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:09.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:32:09.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:09.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:09.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:32:09.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:32:09.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:09.781 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:32:09.781 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:32:09.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.781 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:09.782 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:32:09.782 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:32:09.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.782 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:09.782 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:32:09.782 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:32:09.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.782 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:09.782 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:32:09.782 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:32:09.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:09.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:32:09.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:32:09.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:09.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:32:09.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:32:09.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:09.783 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:32:09.783 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:32:09.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.783 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:09.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:32:09.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:32:09.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:09.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:32:09.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:32:09.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:09.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:09.784 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:09.784 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:32:09.784 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:32:09.784 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.784 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.784 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.784 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:09.785 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:09.785 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:09.786 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:32:09.787 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:09.782+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:09.887 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:09.884+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:09.965 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:09.966 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:09.966 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:09.966 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:09.976 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:32:09.988 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:09.985+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:10.090 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:10.087+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:10.099 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is unset 2023-12-06T18:32:10.172 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:32:10.173 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:10.173 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:10.173 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:10.173 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:10.173 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:10.183 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:32:10.190 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:10.187+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:10.270 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '16 of 30' 2023-12-06T18:32:10.315 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:10.311+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:10.384 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:10.384 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:10.384 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:10.384 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:32:10.384 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:10.384 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:32:10.384 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:10.384 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 288.56786570499997, 2023-12-06T18:32:10.384 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:32:10.384 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:10.384 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:32:10.385 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:10.385 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:10.385 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:10.385 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:10.385 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:10.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:10.385 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:10.385 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:10.385 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:10.385 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:10.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:10.385 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:10.385 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:10.385 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:32:10.385 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:32:10.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:10.385 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:10.386 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:10.386 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:32:10.386 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:32:10.386 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:10.386 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:10.386 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:10.386 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:32:10.386 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:32:10.386 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:10.386 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:10.386 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:10.386 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:32:10.386 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:32:10.386 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:10.386 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:10.387 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:10.387 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:32:10.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:32:10.387 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:10.387 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:10.387 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:10.387 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:32:10.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:32:10.387 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:10.387 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:10.387 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:10.387 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:32:10.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:32:10.387 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:10.387 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:10.387 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:10.388 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:10.388 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:10.388 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:10.392 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:10.389+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:10.393 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:32:10.470 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:32:10.471 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:32:10.493 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:10.490+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:10.570 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:32:10.570 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:32:10.605 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:10.601+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:10.613 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:10.614 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:10.614 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:10.614 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:10.623 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:32:10.673 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:32:10.674 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:32:10.694 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:10.691+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:10.783 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:32:10.784 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:32:10.811 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:10.807+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:10.827 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:10.828 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:10.838 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:32:10.864 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:32:10.864 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:32:10.864 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2023-12-06T18:32:10.865 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2023-12-06T18:32:10.880 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:32:10.880 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:32:10.923 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:10.919+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:10.967 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:32:10.967 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:32:10.997 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:10.993+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:11.009 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:10.739 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:11.015 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:10.746 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:11.033 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:11.033 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:11.043 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:32:11.056 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:32:11.056 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:32:11.060 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:50.506 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:11.065 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:50.510 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:11.081 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:00:50.526 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:11.098 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:11.094+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:11.145 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:32:11.145 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:32:11.175 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:11.176 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:11.186 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:32:11.203 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:11.199+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:11.299 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:11.296+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:11.356 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:11.356 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:11.366 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:32:11.371 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:11.101 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:11.400 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:11.397+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:11.502 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:11.498+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:11.530 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:32:11.526+0000 7f46b2672640 -1 mgr.server reply reply (1) Operation not permitted no change 2023-12-06T18:32:11.530 INFO:teuthology.orchestra.run.smithi061.stdout:moved 0 / 18 (0%) 2023-12-06T18:32:11.530 INFO:teuthology.orchestra.run.smithi061.stdout:avg 2.25 2023-12-06T18:32:11.530 INFO:teuthology.orchestra.run.smithi061.stdout:stddev 1.39194 -> 1.39194 (expected baseline 1.40312) 2023-12-06T18:32:11.530 INFO:teuthology.orchestra.run.smithi061.stdout:min osd.0 with 1 -> 1 pgs (0.444444 -> 0.444444 * mean) 2023-12-06T18:32:11.530 INFO:teuthology.orchestra.run.smithi061.stdout:max osd.5 with 5 -> 5 pgs (2.22222 -> 2.22222 * mean) 2023-12-06T18:32:11.530 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:32:11.531 INFO:teuthology.orchestra.run.smithi061.stdout:oload 110 2023-12-06T18:32:11.531 INFO:teuthology.orchestra.run.smithi061.stdout:max_change 3 2023-12-06T18:32:11.531 INFO:teuthology.orchestra.run.smithi061.stdout:max_change_osds 4 2023-12-06T18:32:11.531 INFO:teuthology.orchestra.run.smithi061.stdout:average_utilization 0.0003 2023-12-06T18:32:11.531 INFO:teuthology.orchestra.run.smithi061.stdout:overload_utilization 0.0003 2023-12-06T18:32:11.531 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7 weight 1.0000 -> 0.8647 2023-12-06T18:32:11.531 INFO:teuthology.orchestra.run.smithi061.stderr:no change 2023-12-06T18:32:11.535 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:11.535 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:11.544 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2023-12-06T18:32:11.544 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2023-12-06T18:32:11.545 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:32:11.559 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '17 of 30' 2023-12-06T18:32:11.603 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:11.600+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:11.704 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:11.700+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:11.714 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:11.714 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:11.723 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:32:11.804 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:11.801+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:11.890 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:11.890 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:11.899 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:32:11.943 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:11.939+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:12.006 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:12.003+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:12.026 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:12.026 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:12.035 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:32:12.115 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-12-06T18:32:12.119 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:12.116+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:12.204 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:12.204 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:12.208 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:12.205+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:12.213 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:32:12.309 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:12.306+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:12.382 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:12.382 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:12.391 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:32:12.438 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:12.431+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:12.512 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:12.508+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:12.520 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:12.520 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:12.530 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:32:12.612 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:12.609+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:12.699 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:12.699 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:12.709 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:32:12.753 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:12.749+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:12.822 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:12.819+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:12.836 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:12.836 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:12.842 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '18 of 30' 2023-12-06T18:32:12.846 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:32:12.902 INFO:teuthology.orchestra.run.smithi061.stdout:moved 0 / 18 (0%) 2023-12-06T18:32:12.902 INFO:teuthology.orchestra.run.smithi061.stdout:avg 2.25 2023-12-06T18:32:12.902 INFO:teuthology.orchestra.run.smithi061.stdout:stddev 1.39194 -> 1.39194 (expected baseline 1.40312) 2023-12-06T18:32:12.902 INFO:teuthology.orchestra.run.smithi061.stdout:min osd.0 with 1 -> 1 pgs (0.444444 -> 0.444444 * mean) 2023-12-06T18:32:12.902 INFO:teuthology.orchestra.run.smithi061.stdout:max osd.5 with 5 -> 5 pgs (2.22222 -> 2.22222 * mean) 2023-12-06T18:32:12.902 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:32:12.903 INFO:teuthology.orchestra.run.smithi061.stdout:oload 110 2023-12-06T18:32:12.903 INFO:teuthology.orchestra.run.smithi061.stdout:max_change 3 2023-12-06T18:32:12.903 INFO:teuthology.orchestra.run.smithi061.stdout:max_change_osds 4 2023-12-06T18:32:12.903 INFO:teuthology.orchestra.run.smithi061.stdout:average_utilization 0.0003 2023-12-06T18:32:12.903 INFO:teuthology.orchestra.run.smithi061.stdout:overload_utilization 0.0003 2023-12-06T18:32:12.903 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7 weight 1.0000 -> 0.8649 2023-12-06T18:32:12.918 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2023-12-06T18:32:12.918 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2023-12-06T18:32:12.931 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:12.927+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:13.016 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:13.013+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:13.047 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:13.047 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:13.056 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:32:13.117 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:13.114+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:13.162 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:32:13.198 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:13.198 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:13.207 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:32:13.249 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:13.245+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:13.320 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:13.316+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:13.449 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:32:13.450 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:32:13.450 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:13.446+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:13.451 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:13.452 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:13.452 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:13.452 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:13.452 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:13.464 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:32:13.549 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:13.545+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:13.623 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:13.619+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:13.630 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:32:13.630 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:32:13.676 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:13.676 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:13.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:13.676 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:32:13.676 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:13.689 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:32:13.741 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:32:13.741 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:32:13.742 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:13.737+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:13.825 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:13.821+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:13.871 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:32:13.871 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:32:13.903 INFO:teuthology.orchestra.run.smithi061.stdout:moved 0 / 18 (0%) 2023-12-06T18:32:13.903 INFO:teuthology.orchestra.run.smithi061.stdout:avg 2.25 2023-12-06T18:32:13.904 INFO:teuthology.orchestra.run.smithi061.stdout:stddev 1.39194 -> 1.39194 (expected baseline 1.40312) 2023-12-06T18:32:13.904 INFO:teuthology.orchestra.run.smithi061.stdout:min osd.0 with 1 -> 1 pgs (0.444444 -> 0.444444 * mean) 2023-12-06T18:32:13.904 INFO:teuthology.orchestra.run.smithi061.stdout:max osd.5 with 5 -> 5 pgs (2.22222 -> 2.22222 * mean) 2023-12-06T18:32:13.904 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:32:13.904 INFO:teuthology.orchestra.run.smithi061.stdout:oload 1000 2023-12-06T18:32:13.904 INFO:teuthology.orchestra.run.smithi061.stdout:max_change 3 2023-12-06T18:32:13.904 INFO:teuthology.orchestra.run.smithi061.stdout:max_change_osds 4 2023-12-06T18:32:13.904 INFO:teuthology.orchestra.run.smithi061.stdout:average_utilization 0.0003 2023-12-06T18:32:13.904 INFO:teuthology.orchestra.run.smithi061.stdout:overload_utilization 0.0030 2023-12-06T18:32:13.904 INFO:teuthology.orchestra.run.smithi061.stderr:no change 2023-12-06T18:32:13.905 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:13.905 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:13.905 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:13.905 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:13.915 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:32:13.917 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '19 of 30' 2023-12-06T18:32:13.921 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2023-12-06T18:32:13.922 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2023-12-06T18:32:13.959 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:32:13.960 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:32:13.962 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:13.957+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:13.999 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is unset 2023-12-06T18:32:14.058 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:32:14.058 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:32:14.059 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:14.055+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:14.094 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:14.094 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:14.094 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:14.095 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:14.095 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:14.106 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:32:14.126 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:14.123+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:14.150 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:32:14.150 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:32:14.228 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:14.224+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:14.236 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:32:14.236 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:32:14.329 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:14.325+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:14.348 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:14.348 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:14.348 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:14.348 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:32:14.348 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.348 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:32:14.349 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:32:14.349 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 318.93734250400001, 2023-12-06T18:32:14.349 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:32:14.349 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.349 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.349 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.349 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.349 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.349 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:32:14.349 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.349 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.349 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.350 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.350 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:32:14.350 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.350 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.350 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.350 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.350 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:32:14.350 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.350 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.350 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.350 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.350 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:32:14.350 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:32:14.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.351 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:32:14.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:32:14.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.351 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:32:14.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:32:14.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:32:14.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:32:14.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:32:14.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:32:14.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:32:14.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:32:14.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:32:14.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:32:14.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:32:14.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:32:14.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:32:14.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:32:14.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:32:14.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.354 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.354 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.354 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:32:14.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:32:14.354 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.354 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.354 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.354 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:32:14.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:32:14.354 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.354 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.354 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.354 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:32:14.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:32:14.355 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.355 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.355 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.355 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:32:14.355 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:32:14.355 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.355 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.355 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.355 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.355 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.355 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:14.355 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:32:14.355 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 316.82876926500001, 2023-12-06T18:32:14.355 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:32:14.355 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.356 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.356 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.356 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.356 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.356 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:32:14.356 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.356 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.356 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.356 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.356 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:32:14.356 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.356 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.356 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.356 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.356 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:32:14.356 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.357 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:32:14.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:32:14.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.357 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:32:14.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:32:14.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.357 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:32:14.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:32:14.358 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.358 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.358 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.358 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:32:14.358 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:32:14.358 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.358 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.358 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.358 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:32:14.358 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:32:14.358 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.358 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.358 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.358 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:32:14.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:32:14.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.359 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.359 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.359 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:32:14.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:32:14.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.359 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.359 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.359 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:32:14.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:32:14.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.359 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.360 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.360 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:32:14.360 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:32:14.360 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.360 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.360 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.360 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:32:14.360 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:32:14.360 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.360 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.360 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.360 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:32:14.360 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:32:14.360 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.360 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.361 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.361 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.361 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.361 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:14.361 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:32:14.361 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 316.82802403300002, 2023-12-06T18:32:14.361 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:32:14.361 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.361 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.361 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.361 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.361 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.361 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:32:14.361 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.361 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.362 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.362 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:32:14.362 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.362 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.362 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.362 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:32:14.362 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.362 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.362 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.362 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:32:14.362 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:32:14.362 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:32:14.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:32:14.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:32:14.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:32:14.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:32:14.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:32:14.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:32:14.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:32:14.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:32:14.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:32:14.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:32:14.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:32:14.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.365 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:32:14.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:32:14.365 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.365 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:32:14.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:32:14.365 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.365 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:32:14.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:32:14.365 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.366 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.366 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.366 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:32:14.366 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:32:14.366 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.366 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.366 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.366 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.366 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.366 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:14.366 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:32:14.366 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 316.82734027800001, 2023-12-06T18:32:14.366 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:32:14.366 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.366 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.367 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.367 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.367 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.367 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:32:14.367 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.367 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.367 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.367 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.367 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:32:14.367 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.367 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.367 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.367 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.367 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:32:14.367 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.368 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.368 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.368 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:32:14.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:32:14.368 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.368 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.368 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:32:14.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:32:14.368 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.368 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.368 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:32:14.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:32:14.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.369 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.369 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.369 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:32:14.369 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:32:14.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.369 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.369 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.369 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:32:14.369 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:32:14.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.369 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.369 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.369 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:32:14.370 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:32:14.370 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.370 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.370 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.370 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:32:14.370 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:32:14.370 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.370 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.370 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.370 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:32:14.370 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:32:14.370 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.370 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.370 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.371 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:32:14.371 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:32:14.371 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.371 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.371 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.371 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:32:14.371 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:32:14.371 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.371 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.371 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.371 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:32:14.371 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:32:14.371 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.371 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.371 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.372 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.372 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.372 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:14.372 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:32:14.372 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 316.82176702300001, 2023-12-06T18:32:14.372 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:32:14.372 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.372 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.372 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.372 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.372 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.372 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:32:14.372 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.372 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.372 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.373 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.373 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:32:14.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.373 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.373 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.373 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:32:14.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.373 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.373 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.373 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:32:14.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:32:14.373 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:32:14.374 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:32:14.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:32:14.374 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:32:14.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:32:14.374 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:32:14.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:32:14.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:32:14.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:32:14.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:32:14.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:32:14.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:32:14.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:32:14.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:32:14.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:32:14.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:32:14.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:32:14.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:32:14.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.377 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.377 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:32:14.377 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:32:14.377 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.377 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.377 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.377 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.377 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:14.377 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:32:14.377 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 316.821093383, 2023-12-06T18:32:14.378 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:32:14.378 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.378 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.378 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.378 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.378 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.378 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:32:14.378 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.378 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.378 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.378 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.378 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:32:14.378 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.378 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.379 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.379 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.379 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:32:14.379 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.379 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.379 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.379 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.379 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:32:14.379 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:32:14.379 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.379 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.379 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.379 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:32:14.379 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:32:14.379 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.380 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.380 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.380 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:32:14.380 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:32:14.380 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.380 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.380 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.380 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:32:14.380 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:32:14.380 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.380 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.380 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.380 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:32:14.380 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:32:14.380 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.381 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.381 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.381 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:32:14.381 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:32:14.381 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.381 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.381 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.381 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:32:14.381 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:32:14.381 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.381 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.381 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.381 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:32:14.381 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:32:14.382 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.382 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.382 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.382 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:32:14.382 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:32:14.382 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.382 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.382 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.382 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:32:14.382 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:32:14.382 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.382 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.382 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.382 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:32:14.383 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:32:14.383 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.383 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.383 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.383 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.383 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.383 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:14.383 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:32:14.383 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 316.82043118399997, 2023-12-06T18:32:14.383 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:32:14.383 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.383 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.383 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.383 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.383 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.383 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:32:14.384 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.384 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.384 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.384 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.384 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:32:14.384 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.384 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.384 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.384 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.384 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:32:14.384 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.384 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.384 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.384 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.384 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:32:14.384 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:32:14.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.385 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.385 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.385 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:32:14.385 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:32:14.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.385 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.385 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.385 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:32:14.385 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:32:14.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.385 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.385 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.385 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:32:14.385 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:32:14.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.386 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.386 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.386 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:32:14.386 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:32:14.386 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.386 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.386 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.386 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:32:14.386 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:32:14.386 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.386 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.386 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.386 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:32:14.386 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:32:14.387 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.387 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.387 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.387 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:32:14.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:32:14.387 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.387 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.387 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.387 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:32:14.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:32:14.387 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.387 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.387 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.387 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:32:14.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:32:14.388 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.388 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.388 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.388 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:32:14.388 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:32:14.388 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.388 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.388 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.388 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.388 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.388 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:14.388 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:32:14.388 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 316.74772174899999, 2023-12-06T18:32:14.388 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:32:14.388 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.389 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.389 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.389 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.389 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.389 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:32:14.389 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.389 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.389 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.389 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.389 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:32:14.389 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.389 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.389 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.389 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.389 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:32:14.390 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.390 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.390 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.390 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.390 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:32:14.390 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:32:14.390 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.390 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.390 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.390 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:32:14.390 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:32:14.390 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.390 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.390 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.390 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:32:14.391 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:32:14.391 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.391 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.391 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.391 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:32:14.391 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:32:14.391 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.391 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.391 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.391 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:32:14.391 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:32:14.391 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.391 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.391 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.392 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:32:14.392 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:32:14.392 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.392 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.392 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.392 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:32:14.392 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:32:14.392 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.392 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.392 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.392 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:32:14.392 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:32:14.392 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.392 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.392 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.392 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:32:14.393 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:32:14.393 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.393 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.393 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.393 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:32:14.393 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:32:14.393 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.393 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.393 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.393 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:32:14.393 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:32:14.393 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.393 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.393 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.393 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.393 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.394 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:14.394 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:32:14.394 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 316.74704004, 2023-12-06T18:32:14.394 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:32:14.394 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.394 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.394 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.394 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.394 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.394 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:32:14.394 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.394 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.394 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.394 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.394 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:32:14.395 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.395 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.395 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.395 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.395 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:32:14.395 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.395 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.395 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.395 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.395 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:32:14.395 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:32:14.395 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.395 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.395 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.395 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:32:14.396 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:32:14.396 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.396 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.396 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.396 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:32:14.396 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:32:14.396 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.396 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.396 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.396 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:32:14.396 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:32:14.396 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.397 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.397 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.397 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:32:14.397 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:32:14.397 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.397 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.397 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.397 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:32:14.397 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:32:14.397 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.397 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.397 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.397 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:32:14.397 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:32:14.398 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.398 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.398 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.398 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:32:14.398 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:32:14.398 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.398 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.398 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.398 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:32:14.398 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:32:14.398 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.398 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.398 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.398 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:32:14.399 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:32:14.399 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.399 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.399 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.399 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:32:14.399 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:32:14.399 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.399 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.399 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.399 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.399 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.399 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:14.399 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:32:14.399 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 315.66823595199997, 2023-12-06T18:32:14.399 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:32:14.399 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.399 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.399 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.400 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.400 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.400 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:32:14.400 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.400 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.400 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.400 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.400 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:32:14.400 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.400 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.400 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.400 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.400 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:32:14.400 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.400 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.400 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.400 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.401 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:32:14.401 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:32:14.401 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.401 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.401 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.401 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:32:14.401 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:32:14.401 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.401 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.401 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.401 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:32:14.401 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:32:14.401 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.401 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.401 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.401 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:32:14.402 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:32:14.402 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.402 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.402 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.402 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:32:14.402 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:32:14.402 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.402 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.402 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.402 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:32:14.402 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:32:14.402 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.402 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.402 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.402 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:32:14.402 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:32:14.403 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.403 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.403 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:32:14.403 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:32:14.403 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.403 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.403 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:32:14.403 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:32:14.403 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.403 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.403 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:32:14.403 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:32:14.403 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.404 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:32:14.404 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:32:14.404 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.404 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.404 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.404 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.404 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:14.404 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:32:14.404 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 315.66681937800001, 2023-12-06T18:32:14.404 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:32:14.404 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.404 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.405 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.405 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.405 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.405 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:32:14.405 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.405 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.405 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.405 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.405 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:32:14.405 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.405 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.405 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.405 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.405 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:32:14.406 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.406 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.406 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.406 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.406 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:32:14.406 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:32:14.406 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.406 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.406 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.406 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:32:14.406 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:32:14.406 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.406 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.406 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.406 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:32:14.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:32:14.407 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.407 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:32:14.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:32:14.407 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.407 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:32:14.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:32:14.407 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.407 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:32:14.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:32:14.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.408 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.408 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:32:14.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:32:14.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.408 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.408 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:32:14.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:32:14.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.408 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.408 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:32:14.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:32:14.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.409 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.409 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.409 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:32:14.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:32:14.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.409 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.409 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.409 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:32:14.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:32:14.409 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.409 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.409 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.409 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.410 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:14.410 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:32:14.410 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 315.66573425299998, 2023-12-06T18:32:14.410 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:32:14.410 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.410 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.410 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.410 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.410 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.410 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:32:14.410 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.410 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.410 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.410 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.411 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:32:14.411 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.411 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.411 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.411 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.411 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:32:14.411 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.411 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.411 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.411 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.411 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:32:14.411 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:32:14.411 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.411 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.411 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:32:14.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:32:14.412 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:32:14.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:32:14.412 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:32:14.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:32:14.412 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:32:14.413 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:32:14.413 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.413 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.413 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.413 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:32:14.413 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:32:14.413 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.413 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.413 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.413 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:32:14.413 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:32:14.413 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.413 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.413 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.413 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:32:14.414 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:32:14.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.414 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.414 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.414 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:32:14.414 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:32:14.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.414 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.414 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.414 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:32:14.414 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:32:14.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.414 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.414 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.414 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:32:14.415 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:32:14.415 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.415 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.415 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.415 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.415 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.415 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:14.415 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:32:14.415 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 315.63058515099999, 2023-12-06T18:32:14.415 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:32:14.415 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.415 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.415 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.415 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.415 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.415 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:32:14.416 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.416 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.416 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.416 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.416 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:32:14.416 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.416 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.416 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.416 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.416 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:32:14.416 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.416 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.416 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.416 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.416 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:32:14.416 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:32:14.417 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.417 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.417 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.417 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:32:14.417 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:32:14.417 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.417 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.417 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.417 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:32:14.417 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:32:14.417 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.417 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.417 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.417 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:32:14.417 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:32:14.418 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.418 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.418 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.418 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:32:14.418 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:32:14.418 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.418 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.418 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.418 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:32:14.418 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:32:14.418 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.418 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.418 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.418 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:32:14.418 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:32:14.419 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.419 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.419 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.419 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:32:14.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:32:14.419 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.419 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.419 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.419 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:32:14.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:32:14.419 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.419 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.419 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.419 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:32:14.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:32:14.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:32:14.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:32:14.420 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.420 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.420 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.420 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:14.420 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:32:14.421 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 309.08085462600002, 2023-12-06T18:32:14.421 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:32:14.421 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.421 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.421 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.421 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.421 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.421 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:32:14.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.422 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.422 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.422 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.422 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:32:14.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.422 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.422 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.422 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.422 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:32:14.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.422 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.422 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.422 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.423 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:32:14.423 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:32:14.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.423 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.423 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:32:14.423 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:32:14.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.423 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.423 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:32:14.423 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:32:14.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.423 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.423 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:32:14.424 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:32:14.424 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.424 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.424 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:32:14.424 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:32:14.424 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.424 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.424 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:32:14.424 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:32:14.424 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.424 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.424 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:32:14.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:32:14.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:32:14.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:32:14.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:32:14.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:32:14.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:32:14.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:32:14.426 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.426 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.426 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.426 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:32:14.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:32:14.426 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.426 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.426 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.426 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.426 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.426 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:14.426 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:32:14.426 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 309.08004901800001, 2023-12-06T18:32:14.427 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:32:14.427 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.427 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.427 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.427 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.427 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.427 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:32:14.427 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.427 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.427 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.427 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.427 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:32:14.427 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.427 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.427 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.428 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.428 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:32:14.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.428 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.428 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.428 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:32:14.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:32:14.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.428 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.428 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.428 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:32:14.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:32:14.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.428 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.429 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.429 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:32:14.429 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:32:14.429 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.429 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.429 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.429 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:32:14.429 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:32:14.429 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.429 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.429 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.429 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:32:14.429 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:32:14.429 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.429 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.430 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.430 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:32:14.430 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:32:14.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.430 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.430 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.430 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:32:14.430 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:32:14.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.430 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.430 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.430 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:32:14.430 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:32:14.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.430 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.431 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.431 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:32:14.431 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:32:14.431 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.431 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.431 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.431 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:32:14.431 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:32:14.431 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.431 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.431 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.431 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:32:14.431 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:32:14.431 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.431 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.431 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.432 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.432 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.432 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:14.432 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:32:14.432 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 309.07935820900002, 2023-12-06T18:32:14.432 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:32:14.432 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.432 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.432 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.432 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.432 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.432 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:32:14.432 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.432 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.432 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.433 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.433 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:32:14.433 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.433 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.433 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.433 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.433 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:32:14.433 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.433 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.433 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.433 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.433 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:32:14.433 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:32:14.433 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.433 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.433 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.434 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:32:14.434 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:32:14.434 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.434 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.434 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.434 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:32:14.434 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:32:14.434 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.434 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.434 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.434 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:32:14.434 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:32:14.434 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.434 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.435 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.435 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:32:14.435 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:32:14.435 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.435 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.435 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.435 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:32:14.435 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:32:14.435 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.435 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.435 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.435 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:32:14.435 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:32:14.435 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.435 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.435 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.436 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:32:14.436 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:32:14.436 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.436 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.436 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:32:14.436 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:32:14.436 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.436 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.436 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:32:14.436 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:32:14.436 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.436 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.436 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:32:14.437 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:32:14.437 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.437 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.437 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.437 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.437 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.437 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:14.437 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:32:14.437 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 309.07506070099998, 2023-12-06T18:32:14.437 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:32:14.437 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.437 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.437 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.437 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.437 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.437 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:32:14.438 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.438 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.438 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.438 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.438 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:32:14.438 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.438 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.438 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.438 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.438 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:32:14.438 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.438 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.438 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.438 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.439 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:32:14.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:32:14.439 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.439 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.439 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.439 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:32:14.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:32:14.439 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.439 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.439 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.439 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:32:14.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:32:14.439 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.439 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.439 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.440 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:32:14.440 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:32:14.440 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.440 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.440 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.440 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:32:14.440 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:32:14.440 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.440 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.440 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.440 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:32:14.440 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:32:14.440 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.440 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.440 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.440 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:32:14.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:32:14.441 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:32:14.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:32:14.441 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:32:14.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:32:14.441 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:32:14.442 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:32:14.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.442 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.442 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.442 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:32:14.442 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:32:14.442 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.442 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.442 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.442 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.442 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:14.442 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:32:14.442 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 309.074393026, 2023-12-06T18:32:14.442 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:32:14.443 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.443 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.443 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.443 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.443 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.443 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:32:14.443 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.443 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.443 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.443 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.443 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:32:14.443 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.443 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.443 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.443 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.443 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:32:14.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.444 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.444 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:32:14.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:32:14.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.444 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.444 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:32:14.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:32:14.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.445 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.445 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:32:14.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:32:14.445 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.445 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.445 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.445 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:32:14.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:32:14.445 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.445 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.445 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.445 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:32:14.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:32:14.445 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.446 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.446 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:32:14.446 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:32:14.446 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.446 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.446 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:32:14.446 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:32:14.446 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.446 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.446 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:32:14.446 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:32:14.446 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.447 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.447 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.447 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:32:14.447 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:32:14.447 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.447 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.447 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.447 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:32:14.447 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:32:14.447 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.447 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.447 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.447 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:32:14.447 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:32:14.447 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.448 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.448 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.448 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.448 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.448 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:14.448 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:32:14.448 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 309.07373759799998, 2023-12-06T18:32:14.448 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:32:14.448 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.448 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.448 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.448 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.448 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.448 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:32:14.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.449 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:32:14.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:32:14.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:32:14.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:32:14.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.450 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.450 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:32:14.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:32:14.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.450 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.450 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:32:14.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:32:14.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.450 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.450 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:32:14.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:32:14.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.451 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.451 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:32:14.451 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:32:14.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.451 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.451 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:32:14.451 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:32:14.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.451 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.451 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:32:14.451 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:32:14.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:32:14.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:32:14.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:32:14.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:32:14.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:32:14.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:32:14.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.453 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.453 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.453 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:32:14.453 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:32:14.453 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.453 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.453 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.453 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.453 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.453 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:14.453 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:32:14.453 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 309.03308305100001, 2023-12-06T18:32:14.453 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:32:14.453 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.453 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:14.454 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:14.454 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:14.454 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:14.454 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:32:14.454 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.454 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.454 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.454 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.454 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:32:14.454 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.454 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.454 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.454 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.454 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:32:14.454 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:32:14.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:32:14.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:32:14.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:32:14.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:32:14.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:32:14.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.456 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:32:14.456 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:32:14.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.456 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:32:14.456 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:32:14.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.456 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:32:14.456 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:32:14.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:14.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:32:14.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:32:14.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:14.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:32:14.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:32:14.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:14.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:32:14.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:32:14.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:14.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:32:14.458 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:32:14.458 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.458 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.458 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.458 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:32:14.458 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:32:14.458 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.458 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.458 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.458 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.458 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.458 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:14.460 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:32:14.460 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:14.456+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:14.561 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:14.558+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:14.585 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:32:14.581+0000 7f46b2672640 -1 mgr.server reply reply (1) Operation not permitted no change 2023-12-06T18:32:14.585 INFO:teuthology.orchestra.run.smithi061.stdout:moved 0 / 18 (0%) 2023-12-06T18:32:14.585 INFO:teuthology.orchestra.run.smithi061.stdout:avg 2.25 2023-12-06T18:32:14.585 INFO:teuthology.orchestra.run.smithi061.stdout:stddev 1.39194 -> 1.39194 (expected baseline 1.40312) 2023-12-06T18:32:14.585 INFO:teuthology.orchestra.run.smithi061.stdout:min osd.0 with 1 -> 1 pgs (0.444444 -> 0.444444 * mean) 2023-12-06T18:32:14.585 INFO:teuthology.orchestra.run.smithi061.stdout:max osd.5 with 5 -> 5 pgs (2.22222 -> 2.22222 * mean) 2023-12-06T18:32:14.585 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:32:14.586 INFO:teuthology.orchestra.run.smithi061.stdout:oload 110 2023-12-06T18:32:14.586 INFO:teuthology.orchestra.run.smithi061.stdout:max_change 0.05 2023-12-06T18:32:14.586 INFO:teuthology.orchestra.run.smithi061.stdout:max_change_osds 4 2023-12-06T18:32:14.586 INFO:teuthology.orchestra.run.smithi061.stdout:average_utilization 0.0003 2023-12-06T18:32:14.586 INFO:teuthology.orchestra.run.smithi061.stdout:overload_utilization 0.0003 2023-12-06T18:32:14.586 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7 weight 0.8649 -> 0.8150 2023-12-06T18:32:14.586 INFO:teuthology.orchestra.run.smithi061.stderr:no change 2023-12-06T18:32:14.598 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2023-12-06T18:32:14.599 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2023-12-06T18:32:14.640 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:14.640 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:14.640 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:14.640 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:14.650 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:32:14.692 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:14.687+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:14.763 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:14.759+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:14.837 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:14.837 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:14.837 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:14.837 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:14.838 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:14.847 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:32:14.864 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:14.861+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:14.965 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:14.962+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:14.989 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:14.989 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:14.989 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:14.989 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:32:14.989 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.989 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:32:14.990 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:14.990 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 293.173353416, 2023-12-06T18:32:14.990 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:32:14.990 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:14.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:32:14.990 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:14.990 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.990 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:14.990 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:14.990 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.990 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.990 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.990 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:14.990 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:14.991 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:14.991 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.991 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.991 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:14.991 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:32:14.991 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:32:14.991 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.991 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.991 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:14.991 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:32:14.991 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:32:14.991 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.991 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.991 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:14.991 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:32:14.991 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:32:14.992 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.992 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.992 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:14.992 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:32:14.992 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:32:14.992 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.992 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.992 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:14.992 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:32:14.992 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:32:14.992 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.992 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.992 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:14.992 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:32:14.992 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:32:14.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:14.993 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:14.993 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:14.993 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:32:14.993 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:32:14.993 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.993 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.993 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.993 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:14.993 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:14.993 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:14.998 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:32:15.058 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '20 of 30' 2023-12-06T18:32:15.102 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:15.098+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:15.147 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:15.148 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:15.148 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:15.148 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:15.158 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:32:15.200 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:15.196+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:15.268 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:15.265+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:15.341 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:15.342 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:15.342 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:15.342 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:15.342 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:15.350 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:32:15.395 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:15.391+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:15.470 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:15.467+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:15.491 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:15.491 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:15.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:15.491 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:32:15.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:15.491 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:32:15.492 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:32:15.492 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 174.39194949200001, 2023-12-06T18:32:15.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:32:15.492 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:15.492 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:32:15.492 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:15.492 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:32:15.492 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:15.492 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:32:15.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:15.492 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:15.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:15.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:15.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:32:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:32:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:32:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:32:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:32:15.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:32:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:32:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:32:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:32:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:32:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:15.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:32:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:32:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:32:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:32:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:32:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:32:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 114.37369219, 2023-12-06T18:32:15.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:32:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:32:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:32:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:32:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:32:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:15.496 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:32:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:32:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:32:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:32:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:32:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:15.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:32:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:32:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:32:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:32:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:32:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:32:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:15.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:32:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:32:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:15.499 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:15.499 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:15.500 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:32:15.573 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:15.570+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:15.639 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:15.640 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:15.640 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:15.640 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:15.649 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:32:15.674 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:15.671+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:15.775 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:15.772+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:15.818 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:15.818 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:15.818 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:15.818 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:15.818 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:15.826 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:32:15.876 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:15.873+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:15.958 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:15.958 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:15.958 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:15.958 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:15.958 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:15.958 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:32:15.958 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:15.958 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 234.46469883, 2023-12-06T18:32:15.959 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:32:15.959 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:15.959 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:32:15.959 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:32:15.959 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:32:15.959 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:15.959 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:32:15.959 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:15.959 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:15.959 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:15.959 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:15.959 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:15.959 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:15.959 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:15.959 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:15.959 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:15.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:15.960 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:15.960 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:15.960 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:15.960 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:15.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:32:15.960 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:32:15.960 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:15.960 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:15.960 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:15.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:32:15.960 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:32:15.960 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:15.960 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:15.960 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:15.960 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:32:15.960 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:32:15.960 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:15.961 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:15.961 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:15.961 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:32:15.961 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:32:15.961 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:15.961 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:15.961 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:15.961 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:32:15.961 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:32:15.961 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:15.961 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:15.961 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:15.961 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:32:15.961 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:32:15.961 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:15.961 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:15.961 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:15.962 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:15.962 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:15.962 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:15.967 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:32:15.978 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:15.974+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:16.025 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-12-06T18:32:16.079 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:16.075+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:16.093 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '21 of 30' 2023-12-06T18:32:16.135 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:16.135 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:16.135 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:16.135 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:16.144 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:32:16.167 INFO:teuthology.orchestra.run.smithi061.stdout:moved 0 / 18 (0%) 2023-12-06T18:32:16.167 INFO:teuthology.orchestra.run.smithi061.stdout:avg 2.25 2023-12-06T18:32:16.167 INFO:teuthology.orchestra.run.smithi061.stdout:stddev 1.39194 -> 1.39194 (expected baseline 1.40312) 2023-12-06T18:32:16.168 INFO:teuthology.orchestra.run.smithi061.stdout:min osd.0 with 1 -> 1 pgs (0.444444 -> 0.444444 * mean) 2023-12-06T18:32:16.168 INFO:teuthology.orchestra.run.smithi061.stdout:max osd.5 with 5 -> 5 pgs (2.22222 -> 2.22222 * mean) 2023-12-06T18:32:16.168 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:32:16.168 INFO:teuthology.orchestra.run.smithi061.stdout:oload 110 2023-12-06T18:32:16.168 INFO:teuthology.orchestra.run.smithi061.stdout:max_change 3 2023-12-06T18:32:16.168 INFO:teuthology.orchestra.run.smithi061.stdout:max_change_osds 4 2023-12-06T18:32:16.168 INFO:teuthology.orchestra.run.smithi061.stdout:average_utilization 0.0003 2023-12-06T18:32:16.168 INFO:teuthology.orchestra.run.smithi061.stdout:overload_utilization 0.0003 2023-12-06T18:32:16.168 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7 weight 0.8649 -> 0.7477 2023-12-06T18:32:16.182 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:16.179+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:16.253 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:32:16.271 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:16.271 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:16.271 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:16.271 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:16.271 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:16.281 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:32:16.282 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:16.279+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:16.395 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:16.391+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:16.449 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:16.449 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:16.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:16.449 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:16.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.449 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:32:16.449 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:16.449 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 319.83353439899997, 2023-12-06T18:32:16.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:32:16.450 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.450 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.450 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:16.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:16.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:32:16.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:32:16.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:32:16.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:32:16.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:32:16.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:32:16.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:32:16.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:32:16.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:32:16.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:32:16.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:32:16.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:32:16.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:32:16.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:32:16.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:32:16.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:32:16.453 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.453 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.453 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.453 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:32:16.453 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:16.453 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 318.81070739, 2023-12-06T18:32:16.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:32:16.454 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.454 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.454 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:16.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:16.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:32:16.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:32:16.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:32:16.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:32:16.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:32:16.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:32:16.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:32:16.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:32:16.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:32:16.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:32:16.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:32:16.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:32:16.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:32:16.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:32:16.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:32:16.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:32:16.458 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.458 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.458 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.458 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:32:16.458 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:16.458 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 318.79866075199999, 2023-12-06T18:32:16.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:32:16.458 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.458 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.458 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:16.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:16.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:32:16.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:32:16.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:32:16.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:32:16.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:32:16.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:32:16.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:32:16.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:32:16.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:32:16.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:32:16.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:32:16.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:32:16.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:32:16.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:32:16.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:32:16.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:32:16.462 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.462 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.462 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.462 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:32:16.462 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:16.462 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 318.77726014500001, 2023-12-06T18:32:16.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:32:16.462 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.462 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.462 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:16.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:16.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:32:16.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:32:16.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:32:16.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:32:16.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:32:16.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:32:16.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:32:16.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:32:16.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:32:16.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:32:16.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:32:16.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:32:16.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:32:16.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:32:16.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:32:16.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:32:16.466 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.466 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.466 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.466 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:32:16.466 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:16.466 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 318.77608082199998, 2023-12-06T18:32:16.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:32:16.466 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.466 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.466 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:16.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:16.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:32:16.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:32:16.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:32:16.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:32:16.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:32:16.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:32:16.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.468 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:32:16.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:32:16.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:32:16.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:32:16.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:32:16.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:32:16.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:32:16.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:32:16.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.470 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.470 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:32:16.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:32:16.470 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.470 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.470 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.470 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:32:16.470 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:16.470 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 318.77550884099998, 2023-12-06T18:32:16.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:32:16.470 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.470 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.470 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:16.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:16.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:32:16.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:32:16.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:32:16.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:32:16.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:32:16.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:32:16.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:32:16.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:32:16.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:32:16.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:32:16.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:32:16.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:32:16.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:32:16.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:32:16.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:32:16.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:32:16.474 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.474 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.474 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.474 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:32:16.474 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:16.474 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 318.32967337100001, 2023-12-06T18:32:16.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:32:16.474 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.474 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.474 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:16.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:16.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:32:16.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:32:16.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:32:16.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:32:16.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:32:16.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:32:16.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:32:16.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:32:16.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:32:16.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:32:16.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:32:16.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:32:16.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:32:16.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:32:16.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:32:16.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:32:16.478 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.478 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.478 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.478 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:32:16.478 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:16.478 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 318.32807351500003, 2023-12-06T18:32:16.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:32:16.479 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.479 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.479 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:16.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:16.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:32:16.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:32:16.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:32:16.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:32:16.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.480 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.480 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:32:16.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:32:16.480 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.480 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:32:16.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:32:16.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:32:16.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:32:16.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.481 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.481 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:32:16.481 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:32:16.481 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.481 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:32:16.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:32:16.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.482 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.482 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.482 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:32:16.482 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:32:16.482 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.482 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.482 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.482 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.483 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:32:16.483 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:16.483 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 317.94886789899999, 2023-12-06T18:32:16.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:32:16.483 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.483 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.483 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:16.483 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.483 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.483 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.483 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.483 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:16.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:32:16.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:32:16.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:32:16.484 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:32:16.484 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.484 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.484 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.484 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:32:16.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:32:16.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:32:16.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:32:16.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:32:16.485 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:32:16.485 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.485 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.485 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.485 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:32:16.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:32:16.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:32:16.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:32:16.486 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.486 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.486 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.486 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:32:16.486 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:32:16.486 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.486 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.486 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.487 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:32:16.487 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:16.487 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 317.71136571699998, 2023-12-06T18:32:16.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:32:16.487 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.487 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.487 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.487 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.487 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:16.487 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.487 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.487 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.487 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:16.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:32:16.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:32:16.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:32:16.488 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:32:16.488 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.488 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.488 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.488 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:32:16.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:32:16.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:32:16.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:32:16.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:32:16.489 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:32:16.489 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.489 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.489 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.489 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:32:16.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:32:16.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:32:16.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:32:16.490 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.490 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.490 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.490 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:32:16.490 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:32:16.490 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.490 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.490 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.491 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.491 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:32:16.491 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:16.491 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 317.71063771799999, 2023-12-06T18:32:16.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:32:16.491 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.491 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.491 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.491 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.491 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.491 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:16.491 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.491 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:16.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:32:16.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:32:16.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.492 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.492 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:32:16.492 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:32:16.492 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.492 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:32:16.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:32:16.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:32:16.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:32:16.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.493 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.493 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:32:16.493 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:32:16.493 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.493 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:32:16.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:32:16.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:32:16.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:32:16.494 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.494 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.494 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.494 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:32:16.494 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:32:16.494 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.495 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.495 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.495 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.495 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:32:16.495 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:16.495 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 317.70562530799998, 2023-12-06T18:32:16.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:32:16.495 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.495 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.495 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.495 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.495 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.495 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:16.495 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:16.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:32:16.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:32:16.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.496 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.496 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.496 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:32:16.496 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:32:16.496 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:32:16.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:32:16.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:32:16.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:32:16.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.497 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.497 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.497 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:32:16.497 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:32:16.497 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:32:16.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:32:16.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:32:16.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:32:16.498 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.498 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.498 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.498 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:32:16.498 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:32:16.499 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.499 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.499 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.499 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.499 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:32:16.499 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:16.499 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 317.70009822899999, 2023-12-06T18:32:16.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:32:16.499 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.499 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.499 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.499 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.499 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.499 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:16.499 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:16.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:32:16.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:32:16.500 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.500 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.500 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.500 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:32:16.500 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:32:16.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:32:16.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:32:16.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:32:16.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:32:16.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.501 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.501 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.501 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:32:16.501 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:32:16.501 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:32:16.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:32:16.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:32:16.502 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:32:16.502 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.502 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.502 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.502 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:32:16.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:32:16.503 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.503 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.503 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.503 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.503 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:32:16.503 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:16.503 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 317.69958739200001, 2023-12-06T18:32:16.503 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:32:16.503 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.503 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.503 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.503 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.503 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:16.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:16.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:32:16.504 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:32:16.504 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.504 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.504 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.504 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:32:16.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:32:16.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:32:16.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:32:16.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:32:16.505 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:32:16.505 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.505 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.505 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.505 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:32:16.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:32:16.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:32:16.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:32:16.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:32:16.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:32:16.506 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.506 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.506 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.506 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:32:16.506 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:32:16.507 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.507 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.507 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.507 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.507 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:32:16.507 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:16.507 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 317.69912813100001, 2023-12-06T18:32:16.507 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:32:16.507 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.507 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.507 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.507 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.507 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.507 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:16.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:16.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:32:16.508 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:32:16.508 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.508 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.508 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.508 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:32:16.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:32:16.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:32:16.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:32:16.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:32:16.509 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:32:16.509 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.509 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.509 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.509 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:32:16.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:32:16.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:32:16.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:32:16.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.510 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:32:16.510 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:32:16.510 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.510 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.510 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.511 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:32:16.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:32:16.511 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.511 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.511 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.511 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.511 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:32:16.511 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:16.511 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 311.11921232399999, 2023-12-06T18:32:16.511 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:32:16.511 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.511 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.511 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.511 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.511 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:16.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:16.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:32:16.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:32:16.512 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.512 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:32:16.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:32:16.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:32:16.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:32:16.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:32:16.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:32:16.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:32:16.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:32:16.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:32:16.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:32:16.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:32:16.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:32:16.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:32:16.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:32:16.515 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.515 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.515 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.515 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:32:16.515 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:16.515 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 311.11297451899998, 2023-12-06T18:32:16.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:32:16.515 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.515 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.515 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:16.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:16.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:32:16.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:32:16.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:32:16.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:32:16.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:32:16.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:32:16.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:32:16.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:32:16.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:32:16.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:32:16.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:32:16.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:32:16.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:32:16.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:32:16.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:32:16.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:32:16.519 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.519 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.519 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.519 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:32:16.519 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:16.519 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 311.10691143999998, 2023-12-06T18:32:16.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:32:16.519 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.519 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.519 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:16.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:16.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:32:16.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:32:16.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:32:16.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:32:16.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:32:16.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:32:16.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:32:16.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:32:16.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.522 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.522 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:32:16.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:32:16.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.522 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.522 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:32:16.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:32:16.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.522 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.522 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:32:16.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:32:16.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:32:16.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:32:16.523 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.523 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.523 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.523 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:32:16.523 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:16.523 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 310.96952435499998, 2023-12-06T18:32:16.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:32:16.523 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.523 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.524 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:16.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:16.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:32:16.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:32:16.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:32:16.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:32:16.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:32:16.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:32:16.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:32:16.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:32:16.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.526 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.526 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:32:16.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:32:16.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.526 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.526 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:32:16.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:32:16.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.526 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:32:16.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:32:16.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:32:16.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:32:16.527 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.527 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.527 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.527 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:32:16.527 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:16.527 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 310.962853628, 2023-12-06T18:32:16.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:32:16.528 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:16.528 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:16.528 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:16.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:16.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:16.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:16.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:16.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:16.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:16.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:32:16.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:32:16.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:16.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:32:16.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:32:16.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:16.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:32:16.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:32:16.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:16.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:32:16.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:32:16.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.530 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:16.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:32:16.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:32:16.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.530 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:16.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:32:16.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:32:16.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:16.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:32:16.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:32:16.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:16.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:16.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:16.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:32:16.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:32:16.531 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.531 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.531 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.531 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:16.531 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:16.531 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:16.533 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:32:16.533 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:16.529+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:16.634 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:16.631+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:16.690 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:32:16.690 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:32:16.705 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:16.705 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:16.705 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:16.705 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:16.714 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:32:16.735 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:16.731+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:16.794 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:32:16.794 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:32:16.836 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:16.832+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:16.881 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:16.882 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:16.882 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:16.882 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:16.882 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:16.890 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:32:16.902 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:32:16.903 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:32:16.936 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:16.933+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:16.994 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:32:16.994 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:32:17.024 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:17.024 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:17.024 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:17.024 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:32:17.024 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:17.033 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:32:17.037 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:17.033+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:17.087 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:32:17.087 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:32:17.138 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:17.135+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:17.191 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:32:17.191 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:32:17.206 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:17.206 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:17.206 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:17.206 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:17.217 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:32:17.238 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:17.235+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:17.249 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is set 2023-12-06T18:32:17.280 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:32:17.280 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:32:17.340 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:17.336+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:17.368 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:32:17.368 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:32:17.405 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:17.405 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:17.414 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:32:17.459 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:17.455+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:17.543 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:17.539+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:17.554 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:17.555 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:17.565 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:32:17.643 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:17.640+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:17.701 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:17.701 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:17.711 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:32:17.755 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:17.751+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:17.844 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:17.841+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:17.846 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:17.846 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:17.856 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:32:17.945 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:17.941+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:18.033 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:18.033 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:18.042 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:32:18.084 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:18.080+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:18.147 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:18.144+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:18.179 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '22 of 30' 2023-12-06T18:32:18.228 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:18.228 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:18.238 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:32:18.248 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:18.244+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:18.349 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:18.345+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:18.388 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:18.388 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:18.397 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:32:18.482 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:18.478+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:18.551 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:18.547+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:18.574 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:18.574 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:18.584 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:32:18.668 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:18.664+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:18.753 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:18.749+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:18.761 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:18.761 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:18.770 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:32:18.855 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:18.851+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:18.949 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:18.949 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:18.954 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:18.951+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:18.959 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:32:19.054 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:19.051+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:19.155 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:19.151+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:19.160 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:19.160 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:19.170 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:32:19.207 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '23 of 30' 2023-12-06T18:32:19.258 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:19.255+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:19.264 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-12-06T18:32:19.359 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:19.356+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:19.381 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:32:19.394 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:19.395 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:19.405 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:32:19.490 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:19.486+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:19.562 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:19.559+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:19.604 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:19.604 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:19.614 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:32:19.699 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:19.695+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:19.765 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:19.761+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:19.817 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:19.817 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:19.826 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:32:19.849 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:32:19.849 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:32:19.871 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:19.868+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:19.966 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:19.963+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:19.969 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:19.969 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:19.980 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:32:19.980 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:32:19.981 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:32:20.067 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:20.064+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:20.071 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:32:20.071 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:32:20.148 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:20.148 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:20.157 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:32:20.169 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:32:20.170 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:32:20.200 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:20.195+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:20.202 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '24 of 30' 2023-12-06T18:32:20.220 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is set 2023-12-06T18:32:20.259 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:32:20.259 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:32:20.269 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:20.266+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:20.330 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:32:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:20.330 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 238.836840408, 2023-12-06T18:32:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:32:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:32:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:32:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:32:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:32:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.331 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:32:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:32:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:32:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:32:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.332 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:32:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:32:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:32:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:32:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:32:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:32:20.333 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:32:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:32:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.334 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.334 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:20.339 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:32:20.349 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:32:20.349 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:32:20.382 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:20.379+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:20.440 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:32:20.440 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:32:20.470 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:20.467+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:20.473 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:20.473 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:20.473 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:20.473 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:20.483 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:32:20.527 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:32:20.528 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:32:20.583 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:20.579+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:20.660 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:20.661 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:20.661 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:20.661 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:20.661 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:20.670 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:32:20.673 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:20.669+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:20.774 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:20.770+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:20.836 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:20.836 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:20.836 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:20.836 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:20.836 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.836 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:32:20.836 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:20.837 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 324.220807224, 2023-12-06T18:32:20.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:32:20.837 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.837 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.837 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.837 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.837 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.837 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:20.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.837 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.837 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.837 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:20.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.838 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.838 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:32:20.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:32:20.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.838 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.838 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:32:20.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:32:20.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.839 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.839 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.839 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:32:20.839 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:32:20.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.839 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.839 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.839 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:32:20.839 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:32:20.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.839 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.839 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.840 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:32:20.840 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:32:20.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.840 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.840 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.840 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:32:20.840 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:32:20.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.840 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.840 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.840 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:32:20.840 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:32:20.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.841 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.841 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:32:20.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:32:20.841 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.841 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.841 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.841 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:32:20.841 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:20.841 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 323.19798021499997, 2023-12-06T18:32:20.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:32:20.842 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.842 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.842 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:20.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:20.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:32:20.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:32:20.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:32:20.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:32:20.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:32:20.844 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:32:20.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:32:20.844 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:32:20.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:32:20.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:32:20.845 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.845 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.845 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.845 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:32:20.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:32:20.845 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.845 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.845 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.845 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:32:20.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:32:20.845 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.845 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.845 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.846 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:32:20.846 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:32:20.846 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.846 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.846 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.846 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.846 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.846 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:32:20.846 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:20.846 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 323.18593357700001, 2023-12-06T18:32:20.846 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:32:20.846 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.846 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.847 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:20.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:20.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:32:20.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:32:20.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:32:20.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:32:20.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:32:20.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:32:20.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.849 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.849 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:32:20.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:32:20.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.849 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.849 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:32:20.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:32:20.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.849 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.849 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:32:20.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:32:20.850 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.850 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.850 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:32:20.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:32:20.850 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.850 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.850 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:32:20.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:32:20.850 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.850 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.851 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.851 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:32:20.851 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:20.851 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 323.16453296999998, 2023-12-06T18:32:20.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:32:20.851 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.851 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.851 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.851 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.851 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:20.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:20.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:32:20.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:32:20.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.853 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.853 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:32:20.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:32:20.853 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.853 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.853 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.853 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:32:20.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:32:20.853 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.853 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.853 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.853 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:32:20.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:32:20.853 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.854 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.854 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:32:20.854 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:32:20.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.854 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.854 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:32:20.854 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:32:20.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:32:20.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:32:20.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:32:20.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:32:20.855 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.855 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.855 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.855 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:32:20.856 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:20.856 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 323.16335364700001, 2023-12-06T18:32:20.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:32:20.856 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.856 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.856 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:20.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:20.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:32:20.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:32:20.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:32:20.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:32:20.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:32:20.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:32:20.858 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:32:20.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:32:20.858 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:32:20.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:32:20.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:32:20.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:32:20.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:32:20.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:32:20.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.860 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:32:20.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:32:20.860 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.860 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.860 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.860 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.860 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:32:20.860 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:20.860 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 323.162781666, 2023-12-06T18:32:20.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:32:20.860 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.860 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.861 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:20.861 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:20.861 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.862 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:32:20.862 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:32:20.862 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.862 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.862 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.862 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:32:20.862 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:32:20.862 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.862 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.862 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.862 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:32:20.862 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:32:20.862 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:32:20.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:32:20.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:32:20.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:32:20.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:32:20.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:32:20.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:32:20.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:32:20.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:32:20.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:32:20.864 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.865 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.865 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.865 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:32:20.865 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:20.865 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 322.71694619599998, 2023-12-06T18:32:20.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:32:20.865 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.865 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.865 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:20.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.866 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:20.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.866 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:32:20.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:32:20.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.867 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:32:20.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:32:20.867 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.867 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:32:20.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:32:20.867 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.867 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:32:20.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:32:20.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:32:20.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:32:20.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:32:20.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:32:20.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:32:20.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:32:20.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:32:20.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:32:20.869 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.869 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.869 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.870 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:32:20.870 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:20.870 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 322.71534634, 2023-12-06T18:32:20.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:32:20.870 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.870 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.870 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:20.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.871 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.871 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:20.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.871 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.871 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:32:20.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:32:20.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:32:20.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:32:20.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:32:20.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:32:20.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:32:20.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:32:20.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:32:20.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:32:20.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:32:20.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:32:20.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:32:20.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:32:20.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:32:20.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:32:20.874 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.874 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.875 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.875 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:32:20.875 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:20.875 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 322.33614072400002, 2023-12-06T18:32:20.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:32:20.875 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.875 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.875 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:20.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:20.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:32:20.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:32:20.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:32:20.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:32:20.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:32:20.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:32:20.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:32:20.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:32:20.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:32:20.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:32:20.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:32:20.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:32:20.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:32:20.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:32:20.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:32:20.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:32:20.879 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.879 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.879 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.879 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:32:20.879 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:20.879 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 322.098638542, 2023-12-06T18:32:20.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:32:20.879 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.879 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.879 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:20.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:20.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:32:20.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:32:20.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:32:20.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:32:20.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:32:20.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:32:20.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:32:20.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:32:20.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:32:20.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:32:20.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:32:20.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:32:20.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:32:20.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:32:20.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:32:20.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:32:20.884 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.884 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.884 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.884 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.884 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.884 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:32:20.884 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:20.884 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 322.09791054300001, 2023-12-06T18:32:20.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:32:20.884 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.884 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.885 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:20.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:20.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:32:20.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:32:20.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:32:20.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:32:20.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:32:20.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:32:20.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:32:20.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:32:20.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:32:20.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:32:20.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:32:20.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:32:20.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:32:20.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:32:20.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:32:20.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:32:20.889 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.889 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.889 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.889 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.889 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:32:20.889 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:20.889 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 322.09289813300001, 2023-12-06T18:32:20.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:32:20.889 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.889 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.890 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:20.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:20.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:32:20.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:32:20.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:32:20.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:32:20.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:32:20.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:32:20.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:32:20.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:32:20.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:32:20.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:32:20.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:32:20.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:32:20.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.893 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.893 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:32:20.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:32:20.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.893 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.893 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:32:20.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:32:20.893 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.893 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.893 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.894 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:32:20.894 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:20.894 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 322.08737105400002, 2023-12-06T18:32:20.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:32:20.894 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.894 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.894 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:20.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:20.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:32:20.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:32:20.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:32:20.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:32:20.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:32:20.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:32:20.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:32:20.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:32:20.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.897 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:32:20.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:32:20.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.897 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.897 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:32:20.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:32:20.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.897 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.897 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:32:20.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:32:20.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:32:20.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:32:20.898 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.898 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.898 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.898 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:32:20.901 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:20.901 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 322.08686021699998, 2023-12-06T18:32:20.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:32:20.901 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.901 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.901 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:20.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:20.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:32:20.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:32:20.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:32:20.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:32:20.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:32:20.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:32:20.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:32:20.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:32:20.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:32:20.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:32:20.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:32:20.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:32:20.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:32:20.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:32:20.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:32:20.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:32:20.905 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.905 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.905 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.906 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:32:20.906 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:20.906 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 322.08640095599998, 2023-12-06T18:32:20.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:32:20.906 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.906 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.906 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:20.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:20.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:32:20.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:32:20.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:32:20.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:32:20.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:32:20.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:32:20.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:32:20.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:32:20.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:32:20.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:32:20.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:32:20.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:32:20.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:32:20.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:32:20.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:32:20.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:32:20.910 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.910 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.910 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.910 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:32:20.910 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:20.910 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 315.50648514900001, 2023-12-06T18:32:20.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:32:20.911 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.911 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.911 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:20.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:20.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:32:20.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:32:20.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:32:20.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:32:20.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:32:20.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:32:20.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:32:20.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:32:20.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:32:20.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:32:20.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:32:20.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:32:20.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:32:20.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:32:20.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:32:20.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:32:20.915 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.915 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.915 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.915 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:32:20.915 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:20.915 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 315.500247344, 2023-12-06T18:32:20.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:32:20.916 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.916 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.916 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:20.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:20.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:32:20.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:32:20.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:32:20.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:32:20.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:32:20.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:32:20.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:32:20.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:32:20.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:32:20.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:32:20.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:32:20.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:32:20.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:32:20.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:32:20.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:32:20.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:32:20.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.920 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.921 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:32:20.921 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:20.921 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 315.494184265, 2023-12-06T18:32:20.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:32:20.921 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.921 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.921 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.921 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.921 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.921 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:20.921 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.921 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:20.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:32:20.922 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:32:20.922 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.922 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.922 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.922 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:32:20.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:32:20.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:32:20.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:32:20.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.923 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.923 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:32:20.923 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:32:20.923 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.923 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:32:20.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:32:20.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:32:20.924 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:32:20.924 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.924 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.924 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.924 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:32:20.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:32:20.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.925 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.925 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:32:20.925 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:32:20.925 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.925 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.925 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.925 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.925 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.925 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:32:20.926 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:20.926 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 315.35679718, 2023-12-06T18:32:20.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:32:20.926 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.926 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.926 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.926 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:20.926 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.926 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.926 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.926 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:20.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:32:20.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:32:20.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.927 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.927 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.927 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:32:20.927 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:32:20.927 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:32:20.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:32:20.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:32:20.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:32:20.928 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.928 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.928 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.928 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:32:20.928 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:32:20.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:32:20.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:32:20.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:32:20.929 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:32:20.929 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.929 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.929 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.929 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:32:20.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:32:20.930 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.930 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.930 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.930 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.930 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.930 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:32:20.930 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:20.930 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 315.35012645299997, 2023-12-06T18:32:20.930 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:32:20.930 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.931 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:20.932 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:32:20.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:32:20.932 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:20.932 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:32:20.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:32:20.932 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.932 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.932 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:20.932 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:32:20.932 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:32:20.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:20.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:32:20.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:32:20.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.933 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.933 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:20.933 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:32:20.933 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:32:20.933 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:20.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:32:20.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:32:20.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:20.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:32:20.934 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:32:20.934 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:20.934 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:20.934 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:20.934 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:32:20.935 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:32:20.935 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.935 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.935 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.935 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:20.935 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:20.935 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:20.937 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:32:20.937 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:20.934+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:21.037 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:21.034+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:21.108 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:21.108 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:21.108 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:21.109 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:21.118 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:32:21.159 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:21.156+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:21.183 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 7, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:32:21.183 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:32:21.183 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [0, 1, 2, 3, 4, 6, 7, 5] 2023-12-06T18:32:21.183 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 6 2023-12-06T18:32:21.210 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '25 of 30' 2023-12-06T18:32:21.238 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:21.235+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:21.285 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:21.294 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:32:21.339 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:21.336+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:21.440 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:21.437+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:32:21.465 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:21.474 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:32:21.541 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:21.538+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:21.601 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:21.601 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:21.601 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:21.601 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:21.610 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:32:21.642 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:21.639+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:21.737 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:21.743 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:21.740+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:21.746 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:32:21.832 INFO:tasks.ceph.mon.c.smithi061.stderr:2023-12-06T18:32:21.816+0000 ff5f640 -1 mon.c@2(leader).osd e90 definitely_dead 0 2023-12-06T18:32:21.844 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:21.841+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:21.878 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:21.878 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 120.21674602, 2023-12-06T18:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:21.879 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:21.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:21.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:21.882 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:21.883 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:21.883 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:21.883 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:21.888 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:32:21.946 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:21.942+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:22.033 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:22.033 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:22.033 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:22.033 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:22.042 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:32:22.046 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:22.043+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:22.147 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:22.144+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:22.211 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '26 of 30' 2023-12-06T18:32:22.228 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:22.228 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:22.228 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:22.228 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:22.228 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:22.235 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-12-06T18:32:22.239 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:32:22.248 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:22.245+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:22.260 INFO:teuthology.orchestra.run.smithi061.stderr:marked out osd.6. 2023-12-06T18:32:22.348 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:22.345+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:22.425 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:22.426 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:22.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:22.426 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:32:22.426 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:22.436 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:32:22.449 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:22.445+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:22.541 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:32:22.599 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:22.596+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:22.614 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:22.614 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:22.614 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:22.614 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:22.624 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:32:22.666 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:22.663+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:22.752 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:22.749+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:22.818 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:32:22.819 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:32:22.853 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:22.850+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:22.862 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:22.862 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:22.862 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:22.862 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:22.862 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:22.871 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:32:22.945 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:32:22.945 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:32:22.975 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:22.973+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:23.048 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:32:23.049 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:32:23.054 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:23.052+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:23.057 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:23.057 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:23.067 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:32:23.146 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:32:23.147 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:32:23.155 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:23.153+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:23.248 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '27 of 30' 2023-12-06T18:32:23.250 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:32:23.251 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:32:23.251 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:23.252 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:23.262 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:32:23.262 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:23.259+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:23.347 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is unset 2023-12-06T18:32:23.349 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:32:23.349 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:32:23.358 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:23.356+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:23.459 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:23.456+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:23.462 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:32:23.463 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:32:23.477 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:23.477 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:23.486 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:32:23.552 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:32:23.552 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:32:23.567 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:23.563+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:23.661 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:23.659+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:23.677 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:23.677 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:23.687 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:32:23.772 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:23.769+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:23.863 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:23.860+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:23.867 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:23.867 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:23.876 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:32:23.963 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:23.960+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:24.054 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:24.055 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:24.064 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:24.061+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:24.065 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:32:24.165 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:24.162+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:24.224 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:24.224 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:24.234 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:32:24.243 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '28 of 30' 2023-12-06T18:32:24.286 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:24.282+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:24.363 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:24.363 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:24.367 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:24.365+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:24.372 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:32:24.468 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:24.465+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:24.499 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:24.500 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:24.509 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:32:24.569 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:24.566+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:24.638 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:24.639 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:24.648 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:32:24.670 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:24.668+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:24.772 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:24.768+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:24.817 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:24.817 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:24.827 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:32:24.872 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:24.870+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:24.973 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:24.970+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:24.998 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:24.998 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:25.007 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:32:25.074 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:25.072+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:25.175 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:25.173+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:25.180 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:25.180 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:25.190 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:32:25.276 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:25.273+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:25.298 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '29 of 30' 2023-12-06T18:32:25.320 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:25.320 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:25.329 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:32:25.368 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-12-06T18:32:25.377 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:25.373+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:25.457 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:25.457 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:25.465 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:32:25.506 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:25.503+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:25.570 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:32:25.615 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:25.612+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:25.633 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:25.633 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:25.642 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:32:25.679 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:25.676+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:25.770 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:25.770 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:25.779 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:32:25.780 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:25.778+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:25.881 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:25.878+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:25.912 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:25.912 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:25.921 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:32:25.983 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:25.981+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:26.033 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:32:26.033 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:32:26.039 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:25.770 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:26.043 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:25.774 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:26.074 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:25.805 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:26.084 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:26.081+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:26.089 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:26.090 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:26.098 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:32:26.105 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:05.549 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:26.110 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:05.556 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:26.131 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:32:26.131 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:32:26.139 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:05.584 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:26.185 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:26.182+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:26.242 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:32:26.243 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:32:26.262 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:26.263 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:26.263 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:26.263 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:26.274 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:32:26.286 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:26.283+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:26.345 INFO:tasks.mon_recovery:adding mon b back in 2023-12-06T18:32:26.346 INFO:tasks.ceph.mon.b:Restarting daemon 2023-12-06T18:32:26.346 DEBUG:teuthology.orchestra.run.smithi152:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2023-12-06T18:32:26.348 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:32:26.348 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:32:26.349 INFO:tasks.ceph.mon.b:Started 2023-12-06T18:32:26.349 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2023-12-06T18:32:26.349 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:32:26.393 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:26.390+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:26.438 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:32:26.439 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:32:26.473 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:26.473 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:26.473 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:26.473 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:26.473 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:26.484 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:32:26.528 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:26.524+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:26.529 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:32:26.529 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:32:26.589 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:26.586+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:26.617 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:32:26.617 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:32:26.667 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:26.668 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:26.668 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:26.668 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:32:26.668 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:26.677 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:32:26.719 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:26.716+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:26.720 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:32:26.721 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:32:26.791 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:26.788+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:26.864 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:26.864 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:26.864 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:26.864 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:26.874 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:32:26.891 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:26.888+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:26.993 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:26.990+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:27.069 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:27.070 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:27.070 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:27.070 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:27.070 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:27.082 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:32:27.094 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:27.090+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:27.095 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is unset 2023-12-06T18:32:27.142 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":40,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":35,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:32:27.194 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:27.191+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:27.280 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 7, 5] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:32:27.280 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:32:27.281 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:32:27.301 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:27.302 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:27.302 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:27.302 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:32:27.302 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.302 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:32:27.302 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:32:27.302 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 331.88900894599999, 2023-12-06T18:32:27.302 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:32:27.302 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.303 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.303 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.303 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.303 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.303 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:32:27.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.303 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.303 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:32:27.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.303 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:32:27.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:32:27.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:32:27.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:32:27.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:32:27.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:32:27.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:32:27.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:32:27.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:32:27.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.305 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.305 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:32:27.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:32:27.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.306 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:32:27.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:32:27.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:32:27.306 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.306 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:32:27.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:32:27.306 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.306 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:32:27.307 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:32:27.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.307 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.307 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.307 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:32:27.307 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:32:27.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.307 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.307 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.307 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:32:27.307 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:32:27.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.307 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.308 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.308 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:32:27.308 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:32:27.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.309 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.309 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:32:27.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:32:27.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.309 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.309 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:32:27.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:32:27.309 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.309 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.309 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.310 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.310 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:27.310 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:32:27.310 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 329.78043570699998, 2023-12-06T18:32:27.310 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:32:27.310 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.310 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.310 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.310 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.310 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.310 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:32:27.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.311 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.311 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:32:27.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.311 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:32:27.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.311 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:32:27.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:32:27.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:32:27.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:32:27.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:32:27.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:32:27.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:32:27.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:32:27.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.313 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:32:27.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:32:27.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.313 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:32:27.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:32:27.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:32:27.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:32:27.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:32:27.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:32:27.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:32:27.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:32:27.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.315 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:32:27.315 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:32:27.315 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.315 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.315 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.315 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:32:27.315 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:32:27.315 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.315 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.315 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.315 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.315 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.315 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:27.315 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:32:27.316 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 329.779690475, 2023-12-06T18:32:27.316 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:32:27.316 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.316 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.316 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.316 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.316 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.316 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:32:27.316 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.316 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.316 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.316 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.317 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:32:27.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.317 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.317 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:32:27.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.317 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.317 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:32:27.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:32:27.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:32:27.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:32:27.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:32:27.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:32:27.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:32:27.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:32:27.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:32:27.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:32:27.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:32:27.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:32:27.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:32:27.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:32:27.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:32:27.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:32:27.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:32:27.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:32:27.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.321 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:32:27.321 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:32:27.321 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.321 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.321 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.321 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:32:27.321 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:32:27.321 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.321 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.321 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.321 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.321 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.321 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:27.321 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:32:27.322 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 329.77900671999998, 2023-12-06T18:32:27.322 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:32:27.322 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.322 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.322 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.322 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.322 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.322 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:32:27.322 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.322 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.322 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.322 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.322 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:32:27.322 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.323 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.323 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.323 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:32:27.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.323 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.323 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.323 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:32:27.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:32:27.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.323 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.323 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.324 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:32:27.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:32:27.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.324 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:32:27.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:32:27.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.324 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:32:27.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:32:27.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:32:27.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:32:27.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:32:27.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:32:27.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:32:27.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:32:27.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:32:27.326 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:32:27.326 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.326 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.326 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.326 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:32:27.326 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:32:27.326 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.326 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.326 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.326 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:32:27.326 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:32:27.326 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.326 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.327 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.327 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:32:27.327 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:32:27.327 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.327 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.327 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.327 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.327 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.327 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:27.327 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:32:27.327 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 329.77343346499998, 2023-12-06T18:32:27.327 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:32:27.327 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.328 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.328 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.328 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.328 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.328 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:32:27.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.328 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.328 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.328 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.328 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:32:27.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.329 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:32:27.329 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.329 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.329 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:32:27.329 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:32:27.329 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.329 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:32:27.329 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:32:27.329 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:32:27.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:32:27.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:32:27.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:32:27.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:32:27.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:32:27.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.331 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.331 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.331 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:32:27.331 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:32:27.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.331 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.331 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.331 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:32:27.331 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:32:27.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.331 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.331 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.331 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:32:27.331 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:32:27.332 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.332 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.332 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.332 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:32:27.332 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:32:27.332 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.332 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.332 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.332 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:32:27.332 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:32:27.332 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.332 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.332 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.332 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:32:27.332 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:32:27.333 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.333 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.333 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.333 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.333 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.333 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:27.333 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:32:27.333 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 329.77275982499998, 2023-12-06T18:32:27.333 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:32:27.333 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.333 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.333 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.333 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.333 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.333 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:32:27.333 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.334 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.334 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.334 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.334 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:32:27.334 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.334 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.334 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.334 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.334 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:32:27.334 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.334 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.334 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.334 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.334 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:32:27.334 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:32:27.335 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.335 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.335 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.335 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:32:27.335 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:32:27.335 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.335 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.335 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.335 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:32:27.335 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:32:27.335 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.335 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.335 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.335 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:32:27.335 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:32:27.336 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.336 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.336 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.336 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:32:27.336 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:32:27.336 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.336 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.336 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.336 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:32:27.336 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:32:27.336 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.336 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.336 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.336 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:32:27.337 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:32:27.337 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.337 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.337 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.337 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:32:27.337 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:32:27.337 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.337 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.337 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.337 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:32:27.337 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:32:27.337 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.337 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.337 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.338 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:32:27.338 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:32:27.338 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.338 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.338 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.338 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:32:27.338 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:32:27.338 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.338 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.338 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.338 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.338 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.338 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:27.338 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:32:27.338 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 329.772097626, 2023-12-06T18:32:27.339 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:32:27.339 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.339 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.339 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.339 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.339 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.339 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:32:27.339 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.339 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.339 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.339 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.339 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:32:27.339 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.339 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.340 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.340 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.340 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:32:27.340 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.340 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.340 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.340 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.340 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:32:27.340 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:32:27.340 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.340 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.340 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.340 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:32:27.340 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:32:27.341 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.341 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.341 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.341 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:32:27.341 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:32:27.341 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.341 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.341 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.341 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:32:27.341 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:32:27.341 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.341 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.341 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.341 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:32:27.342 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:32:27.342 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.342 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.342 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.342 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:32:27.342 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:32:27.342 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.342 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.342 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.342 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:32:27.342 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:32:27.342 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.343 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.343 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.343 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:32:27.343 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:32:27.343 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.343 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.343 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.343 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:32:27.343 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:32:27.343 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.343 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.343 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.343 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:32:27.343 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:32:27.344 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.344 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.344 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.344 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:32:27.344 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:32:27.344 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.344 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.344 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.344 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.344 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.344 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:27.344 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:32:27.344 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 329.69938819100003, 2023-12-06T18:32:27.344 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:32:27.344 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.344 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.345 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.345 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.345 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.345 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:32:27.345 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.345 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.345 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.345 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.345 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:32:27.345 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.345 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.345 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.345 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.345 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:32:27.346 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.346 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.346 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.346 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.346 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:32:27.346 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:32:27.346 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.346 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.346 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.346 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:32:27.346 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:32:27.346 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.346 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.346 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.347 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:32:27.347 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:32:27.347 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.347 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.347 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.347 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:32:27.347 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:32:27.347 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.347 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.347 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.347 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:32:27.347 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:32:27.347 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.347 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.347 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.348 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:32:27.348 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:32:27.348 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.348 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.348 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.348 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:32:27.348 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:32:27.348 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.348 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.348 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.348 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:32:27.348 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:32:27.348 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.348 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.348 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.348 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:32:27.349 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:32:27.349 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.349 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.349 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.349 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:32:27.349 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:32:27.349 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.349 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.349 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.349 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:32:27.349 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:32:27.349 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.349 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.349 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.349 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.350 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.350 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:27.350 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:32:27.350 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 329.69870648199998, 2023-12-06T18:32:27.350 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:32:27.350 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.350 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.350 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.350 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.350 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.350 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:32:27.350 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.350 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.350 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.350 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.351 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:32:27.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.351 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:32:27.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.351 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:32:27.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:32:27.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:32:27.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:32:27.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:32:27.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:32:27.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:32:27.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:32:27.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:32:27.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:32:27.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:32:27.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:32:27.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:32:27.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:32:27.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:32:27.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:32:27.354 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.354 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.354 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.354 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:32:27.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:32:27.354 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.354 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.354 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.354 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:32:27.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:32:27.354 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.354 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.354 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.355 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:32:27.355 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:32:27.355 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.355 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.355 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.355 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.355 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.355 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:27.355 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:32:27.355 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 328.61990239400001, 2023-12-06T18:32:27.355 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:32:27.355 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.355 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.355 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.355 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.355 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.356 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:32:27.356 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.356 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.356 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.356 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.356 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:32:27.356 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.356 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.356 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.356 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.356 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:32:27.356 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.356 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.356 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.356 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:32:27.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:32:27.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.357 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:32:27.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:32:27.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.357 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:32:27.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:32:27.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.357 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.358 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:32:27.358 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:32:27.358 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.358 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.358 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.358 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:32:27.358 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:32:27.358 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.358 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.358 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.358 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:32:27.358 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:32:27.358 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.358 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.359 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.359 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:32:27.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:32:27.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.359 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.359 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.359 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:32:27.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:32:27.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.359 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.359 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.359 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:32:27.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:32:27.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.359 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.360 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.360 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:32:27.360 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:32:27.360 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.360 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.360 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.360 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:32:27.360 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:32:27.360 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.360 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.360 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.360 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.360 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.360 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:27.361 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:32:27.361 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 328.61848581999999, 2023-12-06T18:32:27.361 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:32:27.361 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.361 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.361 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.361 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.361 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.361 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:32:27.361 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.361 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.361 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.361 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.361 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:32:27.361 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.361 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.362 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.362 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:32:27.362 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.362 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.362 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.362 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:32:27.362 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:32:27.362 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.362 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.362 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:32:27.362 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:32:27.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:32:27.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:32:27.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:32:27.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:32:27.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:32:27.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:32:27.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:32:27.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:32:27.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:32:27.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:32:27.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:32:27.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:32:27.365 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.365 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:32:27.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:32:27.365 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.365 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:32:27.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:32:27.365 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.365 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:32:27.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:32:27.366 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.366 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.366 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.366 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.366 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.366 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:27.366 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:32:27.366 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 328.61740069500001, 2023-12-06T18:32:27.366 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:32:27.366 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.366 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.366 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.366 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.366 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.366 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:32:27.367 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.367 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.367 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.367 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.367 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:32:27.367 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.367 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.367 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.367 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.367 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:32:27.367 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.367 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.367 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.367 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.367 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:32:27.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:32:27.368 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.368 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.368 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:32:27.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:32:27.368 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.368 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.368 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:32:27.368 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:32:27.368 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.368 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.368 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.369 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:32:27.369 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:32:27.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.369 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.369 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.369 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:32:27.369 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:32:27.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.369 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.369 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.369 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:32:27.369 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:32:27.369 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.370 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.370 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.370 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:32:27.370 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:32:27.370 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.370 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.370 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.370 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:32:27.370 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:32:27.370 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.370 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.370 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.371 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:32:27.371 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:32:27.371 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.371 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.371 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.371 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:32:27.371 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:32:27.371 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.371 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.371 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.371 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:32:27.371 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:32:27.371 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.371 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.372 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.372 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.372 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.372 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:27.372 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:32:27.372 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 328.58225159300002, 2023-12-06T18:32:27.372 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:32:27.372 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.372 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.372 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.372 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.372 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.372 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:32:27.373 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.373 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.373 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.373 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:32:27.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.373 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.373 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.373 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:32:27.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.373 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.373 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.373 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:32:27.374 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:32:27.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:32:27.374 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:32:27.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:32:27.374 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:32:27.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:32:27.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:32:27.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:32:27.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:32:27.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:32:27.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:32:27.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:32:27.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:32:27.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:32:27.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:32:27.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:32:27.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:32:27.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.377 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:32:27.377 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:32:27.377 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.377 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.377 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:32:27.377 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:32:27.377 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.377 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.377 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.377 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.377 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:27.378 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:32:27.378 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 322.03252106799999, 2023-12-06T18:32:27.378 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:32:27.378 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.378 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.378 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.378 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.378 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.378 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:32:27.378 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.378 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.378 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.378 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.379 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:32:27.379 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.379 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.379 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.379 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.379 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:32:27.379 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.379 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.379 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.379 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.379 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:32:27.379 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:32:27.379 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.379 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.379 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.380 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:32:27.380 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:32:27.380 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.380 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.380 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.380 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:32:27.380 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:32:27.380 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.380 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.380 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.380 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:32:27.380 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:32:27.380 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.380 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.381 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.381 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:32:27.381 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:32:27.381 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.381 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.381 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.381 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:32:27.381 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:32:27.381 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.381 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.381 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.381 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:32:27.381 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:32:27.381 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.381 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.381 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.382 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:32:27.382 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:32:27.382 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.382 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.382 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.382 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:32:27.382 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:32:27.382 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.382 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.382 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.382 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:32:27.382 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:32:27.382 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.382 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.383 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.383 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:32:27.383 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:32:27.383 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.383 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.383 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.383 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.383 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.383 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:27.383 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:32:27.383 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 322.03171545999999, 2023-12-06T18:32:27.383 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:32:27.383 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.383 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.383 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.384 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.384 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.384 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:32:27.384 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.384 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.384 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.384 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.384 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:32:27.384 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.384 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.384 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.384 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.384 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:32:27.384 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.384 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.384 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.385 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.385 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:32:27.385 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:32:27.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.385 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.385 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.385 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:32:27.385 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:32:27.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.385 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.385 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.385 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:32:27.385 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:32:27.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.385 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.385 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.385 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:32:27.386 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:32:27.386 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.386 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.386 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.386 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:32:27.386 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:32:27.386 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.386 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.386 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.386 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:32:27.386 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:32:27.386 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.386 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.386 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.386 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:32:27.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:32:27.387 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.387 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.387 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.387 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:32:27.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:32:27.387 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.387 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.387 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.387 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:32:27.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:32:27.387 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.387 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.387 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.387 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:32:27.388 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:32:27.388 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.388 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.388 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.388 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:32:27.388 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:32:27.388 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.388 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.388 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.388 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.388 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.388 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:27.388 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:32:27.388 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 322.031024651, 2023-12-06T18:32:27.388 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:32:27.389 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.389 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.389 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.389 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.389 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.389 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:32:27.389 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.389 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.389 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.389 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.389 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:32:27.389 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.389 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.389 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.389 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.390 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:32:27.390 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.390 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.390 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.390 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.390 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:32:27.390 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:32:27.390 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.390 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.390 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.390 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:32:27.390 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:32:27.390 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.390 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.390 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.391 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:32:27.391 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:32:27.391 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.391 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.391 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.391 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:32:27.391 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:32:27.391 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.391 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.391 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.391 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:32:27.391 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:32:27.391 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.392 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.392 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.392 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:32:27.392 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:32:27.392 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.392 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.392 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.392 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:32:27.392 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:32:27.392 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.392 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.392 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.392 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:32:27.392 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:32:27.392 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.393 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.393 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.393 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:32:27.393 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:32:27.393 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.393 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.393 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.393 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:32:27.393 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:32:27.393 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.393 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.393 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.393 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:32:27.393 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:32:27.393 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.393 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.394 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.394 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.394 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.394 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:27.394 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:32:27.394 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 322.02672714300002, 2023-12-06T18:32:27.394 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:32:27.394 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.394 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.394 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.394 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.394 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.394 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:32:27.394 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.395 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.395 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.395 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.395 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:32:27.395 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.395 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.395 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.395 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.395 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:32:27.395 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.395 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.395 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.395 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.396 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:32:27.396 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:32:27.396 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.396 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.396 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.396 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:32:27.396 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:32:27.396 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.396 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.396 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.396 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:32:27.396 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:32:27.396 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.397 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.397 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.397 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:32:27.397 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:32:27.397 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.397 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.397 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.397 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:32:27.397 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:32:27.397 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.397 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.397 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.397 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:32:27.397 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:32:27.398 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.398 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.398 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.398 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:32:27.398 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:32:27.398 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.398 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.398 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.398 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:32:27.398 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:32:27.398 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.398 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.398 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.398 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:32:27.399 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:32:27.399 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.399 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.399 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.399 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:32:27.399 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:32:27.399 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.399 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.399 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.399 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:32:27.399 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:32:27.399 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.399 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.399 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.399 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.400 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.400 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:27.400 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:32:27.400 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 322.02605946800003, 2023-12-06T18:32:27.400 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:32:27.400 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.400 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.400 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.400 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.400 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.400 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:32:27.400 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.400 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.400 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.400 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.401 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:32:27.401 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.401 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.401 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.401 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.401 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:32:27.401 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.401 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.401 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.401 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.401 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:32:27.401 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:32:27.401 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.401 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.401 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.402 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:32:27.402 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:32:27.402 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.402 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.402 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.402 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:32:27.402 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:32:27.402 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.402 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.402 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.402 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:32:27.402 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:32:27.402 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.402 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.403 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.403 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:32:27.403 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:32:27.403 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.403 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.403 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:32:27.403 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:32:27.403 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.403 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.403 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:32:27.403 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:32:27.403 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.404 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:32:27.404 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:32:27.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.404 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.404 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:32:27.404 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:32:27.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.404 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.404 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:32:27.404 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:32:27.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.405 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.405 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.405 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:32:27.405 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:32:27.405 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.405 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.405 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.405 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.405 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.405 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:27.405 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:32:27.405 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 322.02540404000001, 2023-12-06T18:32:27.405 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:32:27.405 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.406 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.406 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.406 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.406 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.406 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:32:27.406 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.406 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.406 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.406 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.406 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:32:27.406 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.406 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.406 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.406 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:32:27.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.407 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.407 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:32:27.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:32:27.407 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.407 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:32:27.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:32:27.407 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.407 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.408 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:32:27.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:32:27.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.408 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.408 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:32:27.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:32:27.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.408 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.408 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:32:27.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:32:27.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.408 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.408 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:32:27.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:32:27.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.409 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.409 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.409 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:32:27.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:32:27.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.409 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.409 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.409 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:32:27.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:32:27.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.409 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.409 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.409 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:32:27.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:32:27.410 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.410 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.410 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.410 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:32:27.410 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:32:27.410 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.410 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.410 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.410 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:32:27.410 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:32:27.410 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.410 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.410 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.410 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.411 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.411 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:27.411 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:32:27.411 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 321.98474949299998, 2023-12-06T18:32:27.411 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:32:27.411 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.411 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:27.411 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:27.411 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:27.411 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:27.411 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:32:27.411 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.411 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.411 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.411 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:32:27.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.412 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:32:27.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.412 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:32:27.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:32:27.412 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.413 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:32:27.413 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:32:27.413 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.413 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.413 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.413 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:32:27.413 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:32:27.413 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.413 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.413 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.413 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:32:27.413 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:32:27.413 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.413 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.413 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:32:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:32:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:32:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:32:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:32:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:32:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:27.414 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:32:27.415 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:32:27.415 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.415 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.415 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:27.415 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:32:27.415 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:32:27.415 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.415 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.415 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:27.415 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:32:27.415 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:32:27.415 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.415 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.415 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.415 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:32:27.415 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:32:27.415 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.416 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.416 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.416 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.416 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.416 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:27.416 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:27.299+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:27.417 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:32:27.459 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:27.456+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:27.519 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:27.516+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:27.604 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:27.604 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:27.605 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:27.605 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:27.613 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:32:27.658 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:27.655+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:27.721 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:27.717+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:27.756 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:27.756 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:27.756 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:27.756 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:27.756 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:27.766 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:32:27.851 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:27.848+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:27.878 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:32:27.878 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:32:27.895 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2023-12-06T18:32:27.016390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":199708,"kb_used_data":5364,"kb_used_omap":13,"kb_used_meta":190002,"kb_avail":660403172,"statfs":{"total":676457349120,"available":676252848128,"internally_reserved":0,"allocated":5492736,"data_stored":2568786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13963,"internal_metadata":194562421},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.739742"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-06T18:32:22.416006+0000","last_change":"2023-12-06T18:29:09.051293+0000","last_active":"2023-12-06T18:32:22.416006+0000","last_peered":"2023-12-06T18:32:22.416006+0000","last_clean":"2023-12-06T18:32:22.416006+0000","last_became_active":"2023-12-06T18:29:09.045331+0000","last_became_peered":"2023-12-06T18:29:09.045331+0000","last_unstale":"2023-12-06T18:32:22.416006+0000","last_undegraded":"2023-12-06T18:32:22.416006+0000","last_fullsized":"2023-12-06T18:32:22.416006+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T02:46:01.854508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":76,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-06T18:32:23.314540+0000","last_change":"2023-12-06T18:27:04.883840+0000","last_active":"2023-12-06T18:32:23.314540+0000","last_peered":"2023-12-06T18:32:23.314540+0000","last_clean":"2023-12-06T18:32:23.314540+0000","last_became_active":"2023-12-06T18:27:04.858001+0000","last_became_peered":"2023-12-06T18:27:04.858001+0000","last_unstale":"2023-12-06T18:32:23.314540+0000","last_undegraded":"2023-12-06T18:32:23.314540+0000","last_fullsized":"2023-12-06T18:32:23.314540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:44:02.916244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":76,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-06T18:32:17.548869+0000","last_change":"2023-12-06T18:27:04.336778+0000","last_active":"2023-12-06T18:32:17.548869+0000","last_peered":"2023-12-06T18:32:17.548869+0000","last_clean":"2023-12-06T18:32:17.548869+0000","last_became_active":"2023-12-06T18:27:04.321269+0000","last_became_peered":"2023-12-06T18:27:04.321269+0000","last_unstale":"2023-12-06T18:32:17.548869+0000","last_undegraded":"2023-12-06T18:32:17.548869+0000","last_fullsized":"2023-12-06T18:32:17.548869+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T22:25:15.703579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":60,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-06T18:32:23.586404+0000","last_change":"2023-12-06T18:32:23.586404+0000","last_active":"2023-12-06T18:32:23.586404+0000","last_peered":"2023-12-06T18:32:23.586404+0000","last_clean":"2023-12-06T18:32:23.586404+0000","last_became_active":"2023-12-06T18:32:23.580588+0000","last_became_peered":"2023-12-06T18:32:23.580588+0000","last_unstale":"2023-12-06T18:32:23.586404+0000","last_undegraded":"2023-12-06T18:32:23.586404+0000","last_fullsized":"2023-12-06T18:32:23.586404+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:31:58.277721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":76,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-06T18:32:17.550844+0000","last_change":"2023-12-06T18:27:04.408081+0000","last_active":"2023-12-06T18:32:17.550844+0000","last_peered":"2023-12-06T18:32:17.550844+0000","last_clean":"2023-12-06T18:32:17.550844+0000","last_became_active":"2023-12-06T18:27:04.399765+0000","last_became_peered":"2023-12-06T18:27:04.399765+0000","last_unstale":"2023-12-06T18:32:17.550844+0000","last_undegraded":"2023-12-06T18:32:17.550844+0000","last_fullsized":"2023-12-06T18:32:17.550844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:40:33.570114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-06T18:32:23.038983+0000","last_change":"2023-12-06T18:30:08.762262+0000","last_active":"2023-12-06T18:32:23.038983+0000","last_peered":"2023-12-06T18:32:23.038983+0000","last_clean":"2023-12-06T18:32:23.038983+0000","last_became_active":"2023-12-06T18:30:08.733421+0000","last_became_peered":"2023-12-06T18:30:08.733421+0000","last_unstale":"2023-12-06T18:32:23.038983+0000","last_undegraded":"2023-12-06T18:32:23.038983+0000","last_fullsized":"2023-12-06T18:32:23.038983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:43:31.796397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":90,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-06T18:32:17.544477+0000","last_change":"2023-12-06T18:28:47.876556+0000","last_active":"2023-12-06T18:32:17.544477+0000","last_peered":"2023-12-06T18:32:17.544477+0000","last_clean":"2023-12-06T18:32:17.544477+0000","last_became_active":"2023-12-06T18:28:47.865169+0000","last_became_peered":"2023-12-06T18:28:47.865169+0000","last_unstale":"2023-12-06T18:32:17.544477+0000","last_undegraded":"2023-12-06T18:32:17.544477+0000","last_fullsized":"2023-12-06T18:32:17.544477+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:16:40.095542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-06T18:32:22.298466+0000","last_change":"2023-12-06T18:31:57.187068+0000","last_active":"2023-12-06T18:32:22.298466+0000","last_peered":"2023-12-06T18:32:22.298466+0000","last_clean":"2023-12-06T18:32:22.298466+0000","last_became_active":"2023-12-06T18:31:57.175444+0000","last_became_peered":"2023-12-06T18:31:57.175444+0000","last_unstale":"2023-12-06T18:32:22.298466+0000","last_undegraded":"2023-12-06T18:32:22.298466+0000","last_fullsized":"2023-12-06T18:32:22.298466+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:25:09.547812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":212,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-12-06T18:32:22.296266+0000","last_change":"2023-12-06T18:28:48.207134+0000","last_active":"2023-12-06T18:32:22.296266+0000","last_peered":"2023-12-06T18:32:22.296266+0000","last_clean":"2023-12-06T18:32:22.296266+0000","last_became_active":"2023-12-06T18:28:47.875258+0000","last_became_peered":"2023-12-06T18:28:47.875258+0000","last_unstale":"2023-12-06T18:32:22.296266+0000","last_undegraded":"2023-12-06T18:32:22.296266+0000","last_fullsized":"2023-12-06T18:32:22.296266+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:14:32.667039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1204224,"data_stored":1193520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411396,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":246361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:32:03 2023","interfaces":[{"interface":"back","average":{"1min":11.704000000000001,"5min":10.824,"15min":10.577999999999999},"min":{"1min":3.835,"5min":3.835,"15min":3.835},"max":{"1min":24.553000000000001,"5min":108.57599999999999,"15min":108.57599999999999},"last":7.1280000000000001},{"interface":"front","average":{"1min":9.4469999999999992,"5min":11.196,"15min":12.238},"min":{"1min":4.9619999999999997,"5min":3.7200000000000002,"15min":3.7200000000000002},"max":{"1min":20.617000000000001,"5min":128.09399999999999,"15min":128.09399999999999},"last":9.1010000000000009}]},{"osd":1,"last update":"Wed Dec 6 18:32:03 2023","interfaces":[{"interface":"back","average":{"1min":12.32,"5min":12.087,"15min":12.733000000000001},"min":{"1min":7.048,"5min":6.2539999999999996,"15min":6.2539999999999996},"max":{"1min":25.673999999999999,"5min":70.072000000000003,"15min":70.072000000000003},"last":15.882999999999999},{"interface":"front","average":{"1min":13.257,"5min":13.238,"15min":15.202},"min":{"1min":8.1669999999999998,"5min":6.444,"15min":6.444},"max":{"1min":19.352,"5min":32.920000000000002,"15min":32.920000000000002},"last":15.093999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:32:03 2023","interfaces":[{"interface":"back","average":{"1min":16.815999999999999,"5min":15.079000000000001,"15min":14.851000000000001},"min":{"1min":9.3119999999999994,"5min":7.5149999999999997,"15min":7.5149999999999997},"max":{"1min":53.505000000000003,"5min":75.125,"15min":75.125},"last":10.896000000000001},{"interface":"front","average":{"1min":17.341999999999999,"5min":16.446999999999999,"15min":17.666},"min":{"1min":8.6229999999999993,"5min":7.5419999999999998,"15min":7.5419999999999998},"max":{"1min":25.427,"5min":69.403999999999996,"15min":69.403999999999996},"last":12.510999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:32:08 2023","interfaces":[{"interface":"back","average":{"1min":14.451000000000001,"5min":12.612,"15min":12.106},"min":{"1min":10.464,"5min":5.8879999999999999,"15min":5.8879999999999999},"max":{"1min":19.103999999999999,"5min":27.518000000000001,"15min":27.518000000000001},"last":17.562000000000001},{"interface":"front","average":{"1min":11.693,"5min":14.260999999999999,"15min":15.484},"min":{"1min":7.2910000000000004,"5min":6.3099999999999996,"15min":6.3099999999999996},"max":{"1min":16.655999999999999,"5min":71.718000000000004,"15min":71.718000000000004},"last":13.006}]},{"osd":5,"last update":"Wed Dec 6 18:32:03 2023","interfaces":[{"interface":"back","average":{"1min":18.773,"5min":18.541,"15min":17.984999999999999},"min":{"1min":13.404,"5min":8.3179999999999996,"15min":8.3179999999999996},"max":{"1min":28.771999999999998,"5min":75.751999999999995,"15min":75.751999999999995},"last":26.280999999999999},{"interface":"front","average":{"1min":16.209,"5min":17.341999999999999,"15min":18},"min":{"1min":8.391,"5min":7.9269999999999996,"15min":7.9269999999999996},"max":{"1min":24.571999999999999,"5min":70.933000000000007,"15min":70.933000000000007},"last":28.039999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:32:03 2023","interfaces":[{"interface":"back","average":{"1min":13.712,"5min":14.553000000000001,"15min":15.821},"min":{"1min":10.443,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":18.664999999999999,"5min":29.245000000000001,"15min":29.245000000000001},"last":19.911999999999999},{"interface":"front","average":{"1min":18.916,"5min":19.683,"15min":20.960000000000001},"min":{"1min":14.132,"5min":10.973000000000001,"15min":10.973000000000001},"max":{"1min":26.658999999999999,"5min":74.224000000000004,"15min":74.224000000000004},"last":19.059000000000001}]},{"osd":7,"last update":"Wed Dec 6 18:32:08 2023","interfaces":[{"interface":"back","average":{"1min":15.465999999999999,"5min":20.658999999999999,"15min":21.524999999999999},"min":{"1min":10.867000000000001,"5min":10.867000000000001,"15min":10.867000000000001},"max":{"1min":23.710999999999999,"5min":122.93000000000001,"15min":122.93000000000001},"last":21.827000000000002},{"interface":"front","average":{"1min":20.474,"5min":23.082000000000001,"15min":23.515999999999998},"min":{"1min":15.012,"5min":13.196,"15min":13.196},"max":{"1min":29.952999999999999,"5min":124.473,"15min":124.473},"last":22.896000000000001}]}]},{"osd":2,"up_from":16,"seq":68719476804,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":675840,"data_stored":255237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:32:05 2023","interfaces":[{"interface":"back","average":{"1min":12.933,"5min":10.17,"15min":11.068},"min":{"1min":4.1639999999999997,"5min":4.1639999999999997,"15min":4.1639999999999997},"max":{"1min":39.220999999999997,"5min":39.220999999999997,"15min":39.220999999999997},"last":7.2830000000000004},{"interface":"front","average":{"1min":12.875,"5min":10.885,"15min":11.212999999999999},"min":{"1min":5.4699999999999998,"5min":4.3280000000000003,"15min":4.3280000000000003},"max":{"1min":44.691000000000003,"5min":44.691000000000003,"15min":44.691000000000003},"last":9.9860000000000007}]},{"osd":1,"last update":"Wed Dec 6 18:32:05 2023","interfaces":[{"interface":"back","average":{"1min":12.743,"5min":11.545999999999999,"15min":11.602},"min":{"1min":7.0570000000000004,"5min":5.9139999999999997,"15min":5.9139999999999997},"max":{"1min":27.573,"5min":42.920999999999999,"15min":42.920999999999999},"last":9.234},{"interface":"front","average":{"1min":15.856999999999999,"5min":13.705,"15min":13.909000000000001},"min":{"1min":9.4440000000000008,"5min":6.1479999999999997,"15min":6.1479999999999997},"max":{"1min":31.742000000000001,"5min":34.607999999999997,"15min":34.607999999999997},"last":12.446}]},{"osd":3,"last update":"Wed Dec 6 18:32:05 2023","interfaces":[{"interface":"back","average":{"1min":15.395,"5min":15.23,"15min":18.334},"min":{"1min":7.359,"5min":6.1260000000000003,"15min":6.1260000000000003},"max":{"1min":21.619,"5min":112.129,"15min":112.129},"last":13.679},{"interface":"front","average":{"1min":17.466999999999999,"5min":17.829999999999998,"15min":22.760999999999999},"min":{"1min":11.343,"5min":7.5510000000000002,"15min":7.5510000000000002},"max":{"1min":29.901,"5min":181.601,"15min":181.601},"last":18.091000000000001}]},{"osd":4,"last update":"Wed Dec 6 18:32:05 2023","interfaces":[{"interface":"back","average":{"1min":17.382999999999999,"5min":15.638999999999999,"15min":17.507000000000001},"min":{"1min":9.0120000000000005,"5min":6.609,"15min":6.609},"max":{"1min":35.594999999999999,"5min":97.088999999999999,"15min":97.088999999999999},"last":11.878},{"interface":"front","average":{"1min":14.834,"5min":15.625999999999999,"15min":19.867999999999999},"min":{"1min":7.5919999999999996,"5min":6.0720000000000001,"15min":6.0720000000000001},"max":{"1min":36.552,"5min":98.069000000000003,"15min":98.069000000000003},"last":12.779999999999999}]},{"osd":5,"last update":"Wed Dec 6 18:32:05 2023","interfaces":[{"interface":"back","average":{"1min":19.129000000000001,"5min":17.622,"15min":18.07},"min":{"1min":10.382999999999999,"5min":7.8170000000000002,"15min":7.8170000000000002},"max":{"1min":34.204000000000001,"5min":57.628999999999998,"15min":57.628999999999998},"last":14.420999999999999},{"interface":"front","average":{"1min":17.626000000000001,"5min":15.821,"15min":15.833},"min":{"1min":10.417,"5min":8.5990000000000002,"15min":8.5990000000000002},"max":{"1min":30.733000000000001,"5min":47.749000000000002,"15min":47.749000000000002},"last":18.805}]},{"osd":6,"last update":"Wed Dec 6 18:32:05 2023","interfaces":[{"interface":"back","average":{"1min":20.288,"5min":24.042999999999999,"15min":39.695},"min":{"1min":9.8629999999999995,"5min":8.7289999999999992,"15min":8.7289999999999992},"max":{"1min":32.420000000000002,"5min":498.38499999999999,"15min":498.38499999999999},"last":16.768999999999998},{"interface":"front","average":{"1min":21.768000000000001,"5min":25.242000000000001,"15min":41.494999999999997},"min":{"1min":11.507999999999999,"5min":9.7010000000000005,"15min":9.7010000000000005},"max":{"1min":38.018000000000001,"5min":496.17399999999998,"15min":496.17399999999998},"last":16.135999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:32:10 2023","interfaces":[{"interface":"back","average":{"1min":18.745999999999999,"5min":18.887,"15min":18.911000000000001},"min":{"1min":10.064,"5min":10.064,"15min":10.064},"max":{"1min":30.503,"5min":49.456000000000003,"15min":49.456000000000003},"last":15.541},{"interface":"front","average":{"1min":21.265999999999998,"5min":20.378,"15min":20.23},"min":{"1min":12.288,"5min":12.288,"15min":12.288},"max":{"1min":35.595999999999997,"5min":40.656999999999996,"15min":40.656999999999996},"last":19.443000000000001}]}]},{"osd":1,"up_from":16,"seq":68719476805,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343764,"statfs":{"total":96636764160,"available":96608014336,"internally_reserved":0,"allocated":1077248,"data_stored":653077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:32:02 2023","interfaces":[{"interface":"back","average":{"1min":12.363,"5min":13.302,"15min":20.675999999999998},"min":{"1min":6.4130000000000003,"5min":5.0890000000000004,"15min":5.0890000000000004},"max":{"1min":34.286000000000001,"5min":238.87100000000001,"15min":238.87100000000001},"last":10.212999999999999},{"interface":"front","average":{"1min":10.032,"5min":17.972999999999999,"15min":39.481999999999999},"min":{"1min":6.5099999999999998,"5min":4.4269999999999996,"15min":4.4269999999999996},"max":{"1min":25.722000000000001,"5min":661.33600000000001,"15min":661.33600000000001},"last":13.433999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:32:02 2023","interfaces":[{"interface":"back","average":{"1min":12.92,"5min":17.893000000000001,"15min":30.646999999999998},"min":{"1min":8.0030000000000001,"5min":6.6340000000000003,"15min":6.6340000000000003},"max":{"1min":31.052,"5min":384.315,"15min":384.315},"last":8.9380000000000006},{"interface":"front","average":{"1min":18.448,"5min":24.774000000000001,"15min":43.735999999999997},"min":{"1min":10.724,"5min":5.7009999999999996,"15min":5.7009999999999996},"max":{"1min":35.524999999999999,"5min":656.98699999999997,"15min":656.98699999999997},"last":15.08}]},{"osd":3,"last update":"Wed Dec 6 18:32:02 2023","interfaces":[{"interface":"back","average":{"1min":13.18,"5min":11.839,"15min":13.013},"min":{"1min":5.9880000000000004,"5min":5.548,"15min":5.548},"max":{"1min":24.472000000000001,"5min":34.057000000000002,"15min":34.057000000000002},"last":19.689},{"interface":"front","average":{"1min":12.675000000000001,"5min":13.606999999999999,"15min":15.683},"min":{"1min":6.7469999999999999,"5min":6.5730000000000004,"15min":6.5730000000000004},"max":{"1min":25.629999999999999,"5min":38.133000000000003,"15min":38.133000000000003},"last":16.460000000000001}]},{"osd":4,"last update":"Wed Dec 6 18:32:02 2023","interfaces":[{"interface":"back","average":{"1min":12.805,"5min":19.073,"15min":37.301000000000002},"min":{"1min":7.2750000000000004,"5min":6.2930000000000001,"15min":6.2930000000000001},"max":{"1min":21.841000000000001,"5min":447.67200000000003,"15min":447.67200000000003},"last":13.805},{"interface":"front","average":{"1min":17.407,"5min":24.321000000000002,"15min":44.558},"min":{"1min":11.444000000000001,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":32.219999999999999,"5min":630.572,"15min":630.572},"last":17.300999999999998}]},{"osd":5,"last update":"Wed Dec 6 18:32:02 2023","interfaces":[{"interface":"back","average":{"1min":13.874000000000001,"5min":22.475999999999999,"15min":39.624000000000002},"min":{"1min":7.0540000000000003,"5min":7.0540000000000003,"15min":7.0540000000000003},"max":{"1min":21.376999999999999,"5min":389.55000000000001,"15min":389.55000000000001},"last":24.172000000000001},{"interface":"front","average":{"1min":19.454000000000001,"5min":25.765000000000001,"15min":51.843000000000004},"min":{"1min":13.272,"5min":7.0410000000000004,"15min":7.0410000000000004},"max":{"1min":26.841000000000001,"5min":658.48599999999999,"15min":658.48599999999999},"last":22.484000000000002}]},{"osd":6,"last update":"Wed Dec 6 18:32:02 2023","interfaces":[{"interface":"back","average":{"1min":14.039999999999999,"5min":19.462,"15min":33.225999999999999},"min":{"1min":7.5899999999999999,"5min":7.5899999999999999,"15min":7.5899999999999999},"max":{"1min":30.204000000000001,"5min":439.56400000000002,"15min":439.56400000000002},"last":21.391999999999999},{"interface":"front","average":{"1min":19.829000000000001,"5min":23.744,"15min":35.177999999999997},"min":{"1min":12.763999999999999,"5min":10.641999999999999,"15min":10.641999999999999},"max":{"1min":25.501999999999999,"5min":386.71499999999997,"15min":386.71499999999997},"last":15.721}]},{"osd":7,"last update":"Wed Dec 6 18:32:11 2023","interfaces":[{"interface":"back","average":{"1min":15.515000000000001,"5min":18.134,"15min":18.57},"min":{"1min":10.276,"5min":10.276,"15min":10.276},"max":{"1min":28.106000000000002,"5min":97.561999999999998,"15min":97.561999999999998},"last":25.713999999999999},{"interface":"front","average":{"1min":19.981999999999999,"5min":18.113,"15min":17.800999999999998},"min":{"1min":8.7219999999999995,"5min":8.7219999999999995,"15min":8.7219999999999995},"max":{"1min":33.448,"5min":45.402999999999999,"15min":45.402999999999999},"last":27.553999999999998}]}]},{"osd":0,"up_from":16,"seq":68719476806,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":675840,"data_stored":255237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 6 18:32:06 2023","interfaces":[{"interface":"back","average":{"1min":10.782,"5min":12.151999999999999,"15min":16.414999999999999},"min":{"1min":5.694,"5min":3.2360000000000002,"15min":3.2360000000000002},"max":{"1min":42.630000000000003,"5min":116.593,"15min":116.593},"last":7.1150000000000002},{"interface":"front","average":{"1min":12.417999999999999,"5min":13.814,"15min":17.207000000000001},"min":{"1min":4.9039999999999999,"5min":4.9039999999999999,"15min":4.9039999999999999},"max":{"1min":45.698,"5min":117.896,"15min":117.896},"last":13.819000000000001}]},{"osd":2,"last update":"Wed Dec 6 18:32:06 2023","interfaces":[{"interface":"back","average":{"1min":15.367000000000001,"5min":14.379,"15min":13.718999999999999},"min":{"1min":7.7939999999999996,"5min":5.694,"15min":5.694},"max":{"1min":26.861999999999998,"5min":43.752000000000002,"15min":43.752000000000002},"last":10.576000000000001},{"interface":"front","average":{"1min":15.535,"5min":12.916,"15min":11.619999999999999},"min":{"1min":6.3959999999999999,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":49.670000000000002,"5min":49.670000000000002,"15min":49.670000000000002},"last":8.3640000000000008}]},{"osd":3,"last update":"Wed Dec 6 18:32:11 2023","interfaces":[{"interface":"back","average":{"1min":15.052,"5min":14.375,"15min":16.021000000000001},"min":{"1min":8.8390000000000004,"5min":6.1379999999999999,"15min":6.1379999999999999},"max":{"1min":34.738,"5min":46.689999999999998,"15min":46.689999999999998},"last":20.533000000000001},{"interface":"front","average":{"1min":16.100999999999999,"5min":15.869999999999999,"15min":16.143000000000001},"min":{"1min":7.8419999999999996,"5min":7.8419999999999996,"15min":7.8419999999999996},"max":{"1min":45.320999999999998,"5min":50.545999999999999,"15min":50.545999999999999},"last":11.917}]},{"osd":4,"last update":"Wed Dec 6 18:32:06 2023","interfaces":[{"interface":"back","average":{"1min":14.795,"5min":14.202,"15min":18.988},"min":{"1min":7.4889999999999999,"5min":5.9349999999999996,"15min":5.9349999999999996},"max":{"1min":37.593000000000004,"5min":82.432000000000002,"15min":82.432000000000002},"last":16.777000000000001},{"interface":"front","average":{"1min":19.280000000000001,"5min":18.550000000000001,"15min":20.053000000000001},"min":{"1min":9.8829999999999991,"5min":6.2629999999999999,"15min":6.2629999999999999},"max":{"1min":37.482999999999997,"5min":81.239999999999995,"15min":81.239999999999995},"last":21.440999999999999}]},{"osd":5,"last update":"Wed Dec 6 18:32:06 2023","interfaces":[{"interface":"back","average":{"1min":16.632000000000001,"5min":16.692,"15min":17.318000000000001},"min":{"1min":8.9459999999999997,"5min":8.0099999999999998,"15min":8.0099999999999998},"max":{"1min":34.091999999999999,"5min":71.367000000000004,"15min":71.367000000000004},"last":18.193999999999999},{"interface":"front","average":{"1min":15.298,"5min":16.262,"15min":17.556000000000001},"min":{"1min":7.1779999999999999,"5min":7.1779999999999999,"15min":7.1779999999999999},"max":{"1min":34.893999999999998,"5min":72.307000000000002,"15min":72.307000000000002},"last":11.314}]},{"osd":6,"last update":"Wed Dec 6 18:32:06 2023","interfaces":[{"interface":"back","average":{"1min":19.879000000000001,"5min":19.247,"15min":19.087},"min":{"1min":11.401,"5min":7.7560000000000002,"15min":7.7560000000000002},"max":{"1min":41.682000000000002,"5min":62.932000000000002,"15min":62.932000000000002},"last":23.079000000000001},{"interface":"front","average":{"1min":18.619,"5min":17.544,"15min":18.878},"min":{"1min":9.3520000000000003,"5min":9.3520000000000003,"15min":9.3520000000000003},"max":{"1min":40.893999999999998,"5min":52.627000000000002,"15min":52.627000000000002},"last":19.201000000000001}]},{"osd":7,"last update":"Wed Dec 6 18:32:11 2023","interfaces":[{"interface":"back","average":{"1min":23.013000000000002,"5min":19.899000000000001,"15min":19.379999999999999},"min":{"1min":12.009,"5min":12.009,"15min":12.009},"max":{"1min":58.021000000000001,"5min":58.021000000000001,"15min":58.021000000000001},"last":25.167999999999999},{"interface":"front","average":{"1min":21.065999999999999,"5min":18.530000000000001,"15min":18.108000000000001},"min":{"1min":11.515000000000001,"5min":11.515000000000001,"15min":11.515000000000001},"max":{"1min":40.978999999999999,"5min":40.978999999999999,"15min":40.978999999999999},"last":15.882}]}]},{"osd":5,"up_from":15,"seq":64424509510,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343764,"statfs":{"total":96636764160,"available":96608014336,"internally_reserved":0,"allocated":1077248,"data_stored":653077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:32:10 2023","interfaces":[{"interface":"back","average":{"1min":9.5280000000000005,"5min":10.005000000000001,"15min":14.375999999999999},"min":{"1min":4.2110000000000003,"5min":3.3660000000000001,"15min":3.3660000000000001},"max":{"1min":20.893999999999998,"5min":208.291,"15min":208.291},"last":5.8150000000000004},{"interface":"front","average":{"1min":11.077,"5min":12.297000000000001,"15min":18.088999999999999},"min":{"1min":5.7199999999999998,"5min":4.4299999999999997,"15min":4.4299999999999997},"max":{"1min":21.773,"5min":199.94900000000001,"15min":199.94900000000001},"last":12.535}]},{"osd":1,"last update":"Wed Dec 6 18:32:10 2023","interfaces":[{"interface":"back","average":{"1min":12.890000000000001,"5min":23.846,"15min":52.345999999999997},"min":{"1min":5.085,"5min":5.085,"15min":5.085},"max":{"1min":19.341999999999999,"5min":1211.6010000000001,"15min":1211.6010000000001},"last":13.936999999999999},{"interface":"front","average":{"1min":10.050000000000001,"5min":20.143999999999998,"15min":47.659999999999997},"min":{"1min":6.351,"5min":4.7199999999999998,"15min":4.7199999999999998},"max":{"1min":17.759,"5min":1189.3869999999999,"15min":1189.3869999999999},"last":11.91}]},{"osd":2,"last update":"Wed Dec 6 18:32:10 2023","interfaces":[{"interface":"back","average":{"1min":14.045,"5min":16.213000000000001,"15min":23.106000000000002},"min":{"1min":9.2620000000000005,"5min":6.5129999999999999,"15min":6.5129999999999999},"max":{"1min":20.805,"5min":283.46800000000002,"15min":283.46800000000002},"last":15.085000000000001},{"interface":"front","average":{"1min":17.727,"5min":19.478000000000002,"15min":26.907},"min":{"1min":9.1929999999999996,"5min":8.9350000000000005,"15min":8.9350000000000005},"max":{"1min":33.527000000000001,"5min":296.47800000000001,"15min":296.47800000000001},"last":18.567}]},{"osd":3,"last update":"Wed Dec 6 18:32:10 2023","interfaces":[{"interface":"back","average":{"1min":11.625999999999999,"5min":15.951000000000001,"15min":18.398},"min":{"1min":6.242,"5min":6.242,"15min":6.242},"max":{"1min":20.460000000000001,"5min":47.926000000000002,"15min":47.926000000000002},"last":13.233000000000001},{"interface":"front","average":{"1min":16.221,"5min":13.342000000000001,"15min":14.916},"min":{"1min":12.289999999999999,"5min":5.3979999999999997,"15min":5.3979999999999997},"max":{"1min":28.971,"5min":66.176000000000002,"15min":66.176000000000002},"last":15.763999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:32:10 2023","interfaces":[{"interface":"back","average":{"1min":17.134,"5min":17.401,"15min":19.295000000000002},"min":{"1min":9.7210000000000001,"5min":9.7210000000000001,"15min":9.7210000000000001},"max":{"1min":31.442,"5min":67.248000000000005,"15min":67.248000000000005},"last":16.956},{"interface":"front","average":{"1min":10.519,"5min":10.772,"15min":12.621},"min":{"1min":6.8869999999999996,"5min":5.3300000000000001,"15min":5.3300000000000001},"max":{"1min":18.614000000000001,"5min":77.266999999999996,"15min":77.266999999999996},"last":9.5500000000000007}]},{"osd":6,"last update":"Wed Dec 6 18:32:10 2023","interfaces":[{"interface":"back","average":{"1min":15.331,"5min":13.098000000000001,"15min":15.305999999999999},"min":{"1min":10.071,"5min":5.915,"15min":5.915},"max":{"1min":23.158000000000001,"5min":76.751999999999995,"15min":76.751999999999995},"last":7.4160000000000004},{"interface":"front","average":{"1min":19.021000000000001,"5min":18.861999999999998,"15min":20.622},"min":{"1min":12.337,"5min":8.8390000000000004,"15min":8.8390000000000004},"max":{"1min":32.909999999999997,"5min":57.101999999999997,"15min":57.101999999999997},"last":17.777999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:32:10 2023","interfaces":[{"interface":"back","average":{"1min":17.286000000000001,"5min":14.826000000000001,"15min":14.416},"min":{"1min":10.41,"5min":7.5519999999999996,"15min":7.5519999999999996},"max":{"1min":29.965,"5min":29.965,"15min":29.965},"last":10.875},{"interface":"front","average":{"1min":12.31,"5min":11.972,"15min":11.916},"min":{"1min":8.7509999999999994,"5min":8.2829999999999995,"15min":8.2829999999999995},"max":{"1min":18.251999999999999,"5min":18.251999999999999,"15min":18.251999999999999},"last":10.548}]}]},{"osd":4,"up_from":15,"seq":64424509511,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":671744,"data_stored":254956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:32:04 2023","interfaces":[{"interface":"back","average":{"1min":8.6379999999999999,"5min":10.662000000000001,"15min":13.016},"min":{"1min":4.4370000000000003,"5min":3.7429999999999999,"15min":3.7429999999999999},"max":{"1min":17.593,"5min":53.298999999999999,"15min":53.298999999999999},"last":7.2649999999999997},{"interface":"front","average":{"1min":9.3729999999999993,"5min":9.9890000000000008,"15min":10.707000000000001},"min":{"1min":5.524,"5min":3.9670000000000001,"15min":3.9670000000000001},"max":{"1min":16.498000000000001,"5min":52.567,"15min":52.567},"last":6.399}]},{"osd":1,"last update":"Wed Dec 6 18:32:04 2023","interfaces":[{"interface":"back","average":{"1min":12.893000000000001,"5min":11.975,"15min":13.640000000000001},"min":{"1min":6.1769999999999996,"5min":5.835,"15min":5.835},"max":{"1min":22.509,"5min":49.183,"15min":49.183},"last":15.092000000000001},{"interface":"front","average":{"1min":12.273,"5min":12.824999999999999,"15min":15.102},"min":{"1min":7.5430000000000001,"5min":5.96,"15min":5.96},"max":{"1min":24.081,"5min":45.412999999999997,"15min":45.412999999999997},"last":9.3640000000000008}]},{"osd":2,"last update":"Wed Dec 6 18:32:04 2023","interfaces":[{"interface":"back","average":{"1min":12.390000000000001,"5min":13.938000000000001,"15min":20.734000000000002},"min":{"1min":5.9539999999999997,"5min":5.6390000000000002,"15min":5.6390000000000002},"max":{"1min":44.649000000000001,"5min":85.394000000000005,"15min":85.394000000000005},"last":8.7070000000000007},{"interface":"front","average":{"1min":14.831,"5min":15.615,"15min":19.646999999999998},"min":{"1min":9.0340000000000007,"5min":6.3639999999999999,"15min":6.3639999999999999},"max":{"1min":42.500999999999998,"5min":69.530000000000001,"15min":69.530000000000001},"last":12.414999999999999}]},{"osd":3,"last update":"Wed Dec 6 18:32:12 2023","interfaces":[{"interface":"back","average":{"1min":14.212999999999999,"5min":14.561999999999999,"15min":14.454000000000001},"min":{"1min":9.8339999999999996,"5min":7.008,"15min":7.008},"max":{"1min":18.713000000000001,"5min":26.219000000000001,"15min":26.219000000000001},"last":10.362},{"interface":"front","average":{"1min":12.509,"5min":13.157,"15min":14.773},"min":{"1min":7.4219999999999997,"5min":5.5549999999999997,"15min":5.5549999999999997},"max":{"1min":19.228000000000002,"5min":29.902999999999999,"15min":29.902999999999999},"last":16.649000000000001}]},{"osd":5,"last update":"Wed Dec 6 18:32:04 2023","interfaces":[{"interface":"back","average":{"1min":15.529999999999999,"5min":14.461,"15min":16.603999999999999},"min":{"1min":5.4969999999999999,"5min":5.4969999999999999,"15min":5.4969999999999999},"max":{"1min":53.840000000000003,"5min":78.798000000000002,"15min":78.798000000000002},"last":11.074999999999999},{"interface":"front","average":{"1min":18.231999999999999,"5min":17.484000000000002,"15min":18.596},"min":{"1min":10.002000000000001,"5min":6.4450000000000003,"15min":6.4450000000000003},"max":{"1min":52.781999999999996,"5min":55.814999999999998,"15min":55.814999999999998},"last":14.313000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:32:04 2023","interfaces":[{"interface":"back","average":{"1min":16.608000000000001,"5min":17.411000000000001,"15min":17.890999999999998},"min":{"1min":12.432,"5min":7.7850000000000001,"15min":7.7850000000000001},"max":{"1min":21.513999999999999,"5min":51.149000000000001,"15min":51.149000000000001},"last":18.574000000000002},{"interface":"front","average":{"1min":14.221,"5min":15.055,"15min":16.305},"min":{"1min":8.8460000000000001,"5min":7.8700000000000001,"15min":7.8700000000000001},"max":{"1min":20.815999999999999,"5min":40.426000000000002,"15min":40.426000000000002},"last":17.295999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:32:12 2023","interfaces":[{"interface":"back","average":{"1min":15.081,"5min":18.016999999999999,"15min":18.506},"min":{"1min":8.8800000000000008,"5min":8.8800000000000008,"15min":8.8800000000000008},"max":{"1min":24.949000000000002,"5min":60.390000000000001,"15min":60.390000000000001},"last":13.406000000000001},{"interface":"front","average":{"1min":18.972999999999999,"5min":22.041,"15min":22.553000000000001},"min":{"1min":13.69,"5min":13.69,"15min":13.69},"max":{"1min":40.625999999999998,"5min":57.606999999999999,"15min":57.606999999999999},"last":16.027999999999999}]}]},{"osd":7,"up_from":61,"seq":261993005085,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32852,"kb_used_data":648,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":663552,"data_stored":250841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4395,"internal_metadata":28634837},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:32:07 2023","interfaces":[{"interface":"back","average":{"1min":9.9429999999999996,"5min":10.101000000000001,"15min":10.127000000000001},"min":{"1min":4.3780000000000001,"5min":4.1059999999999999,"15min":4.1059999999999999},"max":{"1min":37.481999999999999,"5min":37.481999999999999,"15min":37.481999999999999},"last":4.4359999999999999},{"interface":"front","average":{"1min":10.396000000000001,"5min":10.891,"15min":10.973000000000001},"min":{"1min":4.4400000000000004,"5min":4.2530000000000001,"15min":4.2530000000000001},"max":{"1min":38.475999999999999,"5min":38.475999999999999,"15min":38.475999999999999},"last":10.984}]},{"osd":1,"last update":"Wed Dec 6 18:32:07 2023","interfaces":[{"interface":"back","average":{"1min":9.5999999999999996,"5min":15.888,"15min":16.936},"min":{"1min":4.8330000000000002,"5min":4.8330000000000002,"15min":4.8330000000000002},"max":{"1min":18.032,"5min":64.765000000000001,"15min":64.765000000000001},"last":30.350000000000001},{"interface":"front","average":{"1min":11.895,"5min":16.867000000000001,"15min":17.695},"min":{"1min":7.4100000000000001,"5min":7.0739999999999998,"15min":7.0739999999999998},"max":{"1min":20.367999999999999,"5min":65.994,"15min":65.994},"last":31.699000000000002}]},{"osd":2,"last update":"Wed Dec 6 18:32:07 2023","interfaces":[{"interface":"back","average":{"1min":13.750999999999999,"5min":17.163,"15min":17.731999999999999},"min":{"1min":7.9909999999999997,"5min":7.9909999999999997,"15min":7.9909999999999997},"max":{"1min":28.07,"5min":46.478000000000002,"15min":46.478000000000002},"last":36.622},{"interface":"front","average":{"1min":12.694000000000001,"5min":13.834,"15min":14.023999999999999},"min":{"1min":6.806,"5min":6.7039999999999997,"15min":6.7039999999999997},"max":{"1min":24.867999999999999,"5min":41.603999999999999,"15min":41.603999999999999},"last":35.576999999999998}]},{"osd":3,"last update":"Wed Dec 6 18:32:07 2023","interfaces":[{"interface":"back","average":{"1min":12.903,"5min":16.390000000000001,"15min":16.971},"min":{"1min":6.8319999999999999,"5min":6.8319999999999999,"15min":6.8319999999999999},"max":{"1min":19.355,"5min":60.463999999999999,"15min":60.463999999999999},"last":9.2880000000000003},{"interface":"front","average":{"1min":14.920999999999999,"5min":19.263000000000002,"15min":19.986999999999998},"min":{"1min":9.7370000000000001,"5min":9.7370000000000001,"15min":9.7370000000000001},"max":{"1min":20.292000000000002,"5min":56.286000000000001,"15min":56.286000000000001},"last":10.327}]},{"osd":4,"last update":"Wed Dec 6 18:32:07 2023","interfaces":[{"interface":"back","average":{"1min":20.628,"5min":21.187999999999999,"15min":21.282},"min":{"1min":8.4960000000000004,"5min":8.4960000000000004,"15min":8.4960000000000004},"max":{"1min":42.698,"5min":57.716999999999999,"15min":57.716999999999999},"last":12.237},{"interface":"front","average":{"1min":19.593,"5min":18.817,"15min":18.687999999999999},"min":{"1min":8.8650000000000002,"5min":8.2729999999999997,"15min":8.2729999999999997},"max":{"1min":53.223999999999997,"5min":67.159000000000006,"15min":67.159000000000006},"last":8.1210000000000004}]},{"osd":5,"last update":"Wed Dec 6 18:32:07 2023","interfaces":[{"interface":"back","average":{"1min":13.090999999999999,"5min":14.409000000000001,"15min":14.629},"min":{"1min":9.0619999999999994,"5min":6.3399999999999999,"15min":6.3399999999999999},"max":{"1min":16.247,"5min":36.317999999999998,"15min":36.317999999999998},"last":20.283000000000001},{"interface":"front","average":{"1min":16.387,"5min":19.221,"15min":19.693000000000001},"min":{"1min":10.183,"5min":10.111000000000001,"15min":10.111000000000001},"max":{"1min":31.968,"5min":70.900000000000006,"15min":70.900000000000006},"last":13.273}]},{"osd":6,"last update":"Wed Dec 6 18:32:07 2023","interfaces":[{"interface":"back","average":{"1min":15.647,"5min":16.550000000000001,"15min":16.699999999999999},"min":{"1min":11.349,"5min":8.7170000000000005,"15min":8.7170000000000005},"max":{"1min":30.103000000000002,"5min":54.393000000000001,"15min":54.393000000000001},"last":22.155000000000001},{"interface":"front","average":{"1min":18.846,"5min":20.512,"15min":20.789999999999999},"min":{"1min":10.911,"5min":10.911,"15min":10.911},"max":{"1min":28.954000000000001,"5min":69.263000000000005,"15min":69.263000000000005},"last":20.902999999999999}]}]},{"osd":6,"up_from":0,"seq":60129542212,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:32:27.923 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:27.920+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:27.953 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:27.954 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:27.954 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:27.954 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:32:27.954 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.954 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:32:27.954 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:27.954 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 306.13778329100001, 2023-12-06T18:32:27.954 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:32:27.954 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:27.954 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:32:27.954 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:27.954 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.955 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:27.955 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:27.955 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.955 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.955 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:27.955 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:27.955 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:27.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.955 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.955 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:27.955 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:32:27.955 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:32:27.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.955 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.955 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:27.956 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:32:27.956 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:32:27.956 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.956 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.956 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:27.956 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:32:27.956 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:32:27.956 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.956 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.956 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:27.956 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:32:27.956 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:32:27.956 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.956 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.956 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:27.957 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:32:27.957 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:32:27.957 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.957 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.957 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:27.957 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:32:27.957 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:32:27.957 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:27.957 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:27.957 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:27.957 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:32:27.957 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:32:27.957 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.957 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.957 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.958 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:27.958 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:27.958 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:27.964 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:32:28.025 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:28.021+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:28.102 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:28.102 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:28.102 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:28.102 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:28.112 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:32:28.125 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:28.121+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:28.227 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:28.222+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:28.250 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:28.251 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:28.251 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:28.251 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:28.251 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:28.260 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:32:28.328 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:28.325+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:28.405 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:28.405 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:28.405 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:28.405 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:32:28.405 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.405 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:32:28.405 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:32:28.406 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 187.303433388, 2023-12-06T18:32:28.406 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:32:28.406 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:28.406 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:32:28.406 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:28.406 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:32:28.406 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:28.406 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:32:28.406 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.406 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:28.406 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.406 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:28.406 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:32:28.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:28.407 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.407 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:28.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:32:28.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:28.407 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.407 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:28.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:32:28.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:32:28.407 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.407 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:28.408 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:32:28.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:32:28.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.408 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:28.408 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:32:28.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:32:28.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.408 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:28.408 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:32:28.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:32:28.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.408 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:28.408 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:32:28.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:32:28.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.409 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.409 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:28.409 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:32:28.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:32:28.409 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:28.409 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:28.409 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:28.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.409 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.409 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:32:28.409 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:32:28.409 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 127.28517608600001, 2023-12-06T18:32:28.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:32:28.410 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:28.410 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:32:28.410 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:28.410 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:32:28.410 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:28.410 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:32:28.410 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.410 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:28.410 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.410 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:28.410 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:32:28.410 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:28.410 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.410 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.411 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:28.411 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:32:28.411 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:28.411 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.411 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.411 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:28.411 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:32:28.411 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:32:28.411 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.411 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.411 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:28.411 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:32:28.411 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:32:28.411 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.411 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:28.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:32:28.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:32:28.412 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:28.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:32:28.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:32:28.412 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:28.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:32:28.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:32:28.412 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.413 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:28.413 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:32:28.413 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:32:28.413 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:28.413 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:28.413 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:28.413 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.413 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.413 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:11 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2023-12-06T18:32:28.413 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:32:28.413 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 7.2607549689999997, 2023-12-06T18:32:28.413 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0086075449999999994, 2023-12-06T18:32:28.413 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:28.413 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:32:28.413 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:28.413 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:32:28.414 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:28.414 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 11 2023-12-06T18:32:28.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.414 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:28.414 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.414 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:28.414 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:32:28.414 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:28.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.414 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.414 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:28.414 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:32:28.414 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:28.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.414 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.415 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:28.415 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135441+0000", 2023-12-06T18:32:28.415 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.8695999999999999e-05 2023-12-06T18:32:28.415 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.415 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.415 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:28.415 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135650+0000", 2023-12-06T18:32:28.415 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020883699999999999 2023-12-06T18:32:28.415 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.415 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.415 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:28.415 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135863+0000", 2023-12-06T18:32:28.415 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000213931 2023-12-06T18:32:28.415 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.415 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.416 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:28.416 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.136054+0000", 2023-12-06T18:32:28.416 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000190416 2023-12-06T18:32:28.416 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.416 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.416 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:28.416 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.142056+0000", 2023-12-06T18:32:28.416 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0060019330000000001 2023-12-06T18:32:28.416 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:28.416 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:28.416 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:28.416 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.144030+0000", 2023-12-06T18:32:28.416 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0019737320000000002 2023-12-06T18:32:28.416 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:28.416 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:28.416 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:28.417 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:28.417 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:28.417 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:28.418 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:32:28.460 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:28.457+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:28.530 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:28.527+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:28.549 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:28.549 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:28.550 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:28.550 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:28.560 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:32:28.631 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:28.628+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:28.692 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:28.692 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:28.692 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:28.702 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:32:28.732 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:28.729+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:28.734 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:32:28.833 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:28.829+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:28.835 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:28.835 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:28.835 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:28.835 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:28.835 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:28.835 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:32:28.835 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:28.835 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 247.34169901600001, 2023-12-06T18:32:28.835 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:32:28.835 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:28.836 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:32:28.836 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:32:28.836 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:32:28.836 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:28.836 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:32:28.836 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:28.836 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:28.836 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:28.836 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:28.836 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:28.836 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:28.836 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:28.836 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:28.836 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:28.836 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:28.836 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:28.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:28.837 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:28.837 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:28.837 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:32:28.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:32:28.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:28.837 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:28.837 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:28.837 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:32:28.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:32:28.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:28.837 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:28.837 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:28.837 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:32:28.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:32:28.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:28.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:28.838 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:28.838 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:32:28.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:32:28.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:28.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:28.838 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:28.838 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:32:28.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:32:28.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:28.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:28.838 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:28.838 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:32:28.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:32:28.839 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:28.839 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:28.839 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:28.839 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:28.839 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:28.839 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:28.845 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:32:28.934 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:28.930+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:28.980 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:32:28.980 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:32:29.017 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:29.017 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:29.017 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:29.017 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:29.027 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:32:29.069 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:29.065+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:29.072 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:32:29.072 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:32:29.110 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-12-06T18:32:29.155 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:29.152+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:29.163 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:32:29.164 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:32:29.197 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:29.197 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:29.197 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:29.197 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:29.198 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:29.207 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:32:29.251 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:29.248+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:29.264 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:32:29.264 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:32:29.338 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:29.335+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:29.369 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:32:29.369 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:32:29.399 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:29.399 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:29.399 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:29.399 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:29.399 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.399 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:32:29.399 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:29.399 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 332.78006524900002, 2023-12-06T18:32:29.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:32:29.400 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.400 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.400 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:29.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.400 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.400 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:29.400 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.400 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.400 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:32:29.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:32:29.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:32:29.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:32:29.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.401 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.401 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:32:29.401 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:32:29.401 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.401 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.402 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.402 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:32:29.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:32:29.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.402 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.402 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:32:29.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:32:29.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.402 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.402 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:32:29.402 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:32:29.402 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.402 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.403 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.403 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:32:29.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:32:29.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.403 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.403 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:32:29.403 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:32:29.403 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.403 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.403 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.403 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.403 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.403 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:32:29.403 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:29.404 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 331.75723823999999, 2023-12-06T18:32:29.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:32:29.404 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.404 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.404 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.404 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.404 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:29.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.404 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.404 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.404 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.404 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:29.404 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:32:29.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:32:29.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:32:29.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:32:29.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.405 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.405 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.405 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:32:29.405 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:32:29.405 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:32:29.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:32:29.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:32:29.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:32:29.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.406 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.406 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.406 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:32:29.406 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:32:29.406 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.407 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.407 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.407 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:32:29.407 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:32:29.407 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.407 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.407 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.407 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:32:29.407 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:32:29.407 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.407 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.407 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.407 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.408 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.408 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:32:29.408 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:29.408 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 331.74519160199998, 2023-12-06T18:32:29.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:32:29.408 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.408 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.408 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.408 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.408 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.408 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:29.408 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.408 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:29.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:32:29.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:32:29.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.409 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.409 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.409 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:32:29.409 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:32:29.409 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:32:29.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:32:29.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:32:29.410 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:32:29.410 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.410 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.410 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.410 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:32:29.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:32:29.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:32:29.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:32:29.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:32:29.411 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:32:29.411 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.411 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.411 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.411 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:32:29.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:32:29.412 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.412 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.412 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.412 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.412 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:32:29.412 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:29.412 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 331.723790995, 2023-12-06T18:32:29.412 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:32:29.412 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.412 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.412 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.412 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.413 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:29.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.413 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.413 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:29.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.413 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.413 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:32:29.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:32:29.413 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.413 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.413 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:32:29.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:32:29.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:32:29.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:32:29.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.414 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.414 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.414 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:32:29.414 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:32:29.414 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:32:29.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:32:29.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:32:29.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:32:29.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.415 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.415 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:32:29.415 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:32:29.415 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.415 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.416 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.416 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:32:29.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:32:29.416 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.416 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.416 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.416 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.416 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.416 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:32:29.416 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:29.416 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 331.72261167200003, 2023-12-06T18:32:29.416 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:32:29.416 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.416 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.416 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.417 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:29.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.417 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:29.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.417 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.417 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:32:29.417 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:32:29.417 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.417 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:32:29.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:32:29.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:32:29.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:32:29.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.418 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.418 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:32:29.418 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:32:29.418 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.418 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:32:29.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:32:29.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:32:29.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:32:29.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.419 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:32:29.419 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:32:29.419 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.419 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.419 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.420 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:32:29.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:32:29.420 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.420 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.420 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.420 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.420 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:32:29.420 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:29.420 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 331.72203969100002, 2023-12-06T18:32:29.420 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:32:29.420 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.420 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.420 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.420 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:29.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:29.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.421 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:32:29.421 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:32:29.421 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.421 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.421 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.422 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:32:29.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:32:29.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.422 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.422 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:32:29.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:32:29.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.422 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.422 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:32:29.422 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:32:29.422 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.422 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.422 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:32:29.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:32:29.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:32:29.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:32:29.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.423 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:32:29.423 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:32:29.423 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.423 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.423 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.424 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:32:29.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:32:29.424 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.424 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.424 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.424 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.424 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:32:29.424 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:29.424 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 331.276204221, 2023-12-06T18:32:29.424 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:32:29.424 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.424 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.424 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.424 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.424 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:29.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:29.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:32:29.425 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:32:29.425 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.425 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.425 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.425 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:32:29.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:32:29.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.426 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.426 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:32:29.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:32:29.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.426 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.426 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:32:29.426 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:32:29.426 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.426 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.426 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.427 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:32:29.427 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:32:29.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.427 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.427 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:32:29.427 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:32:29.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.427 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.427 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:32:29.427 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:32:29.427 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.427 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.427 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.428 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:32:29.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:32:29.428 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.428 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.428 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.428 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.428 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.428 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:32:29.428 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:29.428 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 331.27460436500002, 2023-12-06T18:32:29.428 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:32:29.428 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.428 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.428 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.428 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.428 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:29.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:29.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:32:29.429 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:32:29.429 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.429 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.429 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.429 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:32:29.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:32:29.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:32:29.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:32:29.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.430 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:32:29.430 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:32:29.430 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.430 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.430 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:32:29.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:32:29.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:32:29.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:32:29.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:32:29.431 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:32:29.431 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.431 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.431 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.431 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:32:29.432 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:32:29.432 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.432 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.432 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.432 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.432 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:32:29.432 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:29.432 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 330.89539874899998, 2023-12-06T18:32:29.432 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:32:29.432 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.432 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.432 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.432 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.432 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:29.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:29.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:32:29.433 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:32:29.433 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.433 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.433 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.433 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:32:29.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:32:29.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.434 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.434 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:32:29.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:32:29.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.434 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.434 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:32:29.434 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:32:29.434 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.434 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.434 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.434 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:32:29.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:32:29.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:32:29.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:32:29.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:32:29.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:32:29.435 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.435 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.435 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.435 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:32:29.435 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:32:29.436 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.436 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.436 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.436 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.436 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.436 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:32:29.436 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:29.436 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 330.65789656700002, 2023-12-06T18:32:29.436 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:32:29.436 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.436 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.436 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:29.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:29.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.437 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.437 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.437 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:32:29.437 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:32:29.437 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:32:29.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:32:29.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:32:29.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:32:29.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.438 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.438 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.438 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:32:29.438 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:32:29.438 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:32:29.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:32:29.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:32:29.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:32:29.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.439 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.439 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:32:29.439 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:32:29.439 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.439 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.440 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.440 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:32:29.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:32:29.440 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.440 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.440 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.440 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.440 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.440 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:32:29.440 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:29.440 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 330.65716856799997, 2023-12-06T18:32:29.440 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:32:29.440 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.440 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.440 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:29.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:29.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.441 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.441 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:32:29.441 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:32:29.441 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.441 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:32:29.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:32:29.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:32:29.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:32:29.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.442 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.442 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:32:29.442 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:32:29.442 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.442 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:32:29.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:32:29.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:32:29.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:32:29.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.443 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:32:29.443 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:32:29.443 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.443 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.443 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.444 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:32:29.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:32:29.444 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.444 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.444 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.444 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.444 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:32:29.444 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:29.444 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 330.65215615800003, 2023-12-06T18:32:29.444 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:32:29.444 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.444 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.444 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.444 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.444 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:29.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:29.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:32:29.445 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:32:29.445 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.445 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.445 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.445 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:32:29.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:32:29.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:32:29.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:32:29.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:32:29.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:32:29.446 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.446 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.446 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.446 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:32:29.446 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:32:29.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:32:29.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:32:29.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:32:29.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:32:29.447 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.447 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.447 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.447 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:32:29.447 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:32:29.447 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.448 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.448 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.448 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:32:29.448 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:29.448 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 330.64662907899998, 2023-12-06T18:32:29.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:32:29.448 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.448 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.448 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.448 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.448 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.448 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:29.448 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.448 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:29.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:32:29.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:32:29.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.449 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:32:29.449 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:32:29.449 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.449 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.449 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:32:29.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:32:29.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:32:29.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:32:29.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.450 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:32:29.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:32:29.450 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.450 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:32:29.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:32:29.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:32:29.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:32:29.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.451 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.451 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.451 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:32:29.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:32:29.451 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.451 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.451 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.451 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.452 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:32:29.452 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:29.452 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 330.646118242, 2023-12-06T18:32:29.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:32:29.452 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.452 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.452 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:29.452 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.452 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:29.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:32:29.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:32:29.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:32:29.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:32:29.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:32:29.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:32:29.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:32:29.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:32:29.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:32:29.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:32:29.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:32:29.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:32:29.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:32:29.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:32:29.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:32:29.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:32:29.455 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.455 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.455 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.455 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:32:29.456 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:29.456 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 330.645658981, 2023-12-06T18:32:29.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:32:29.456 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.456 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.456 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.456 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:29.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:29.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:32:29.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:32:29.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:32:29.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:32:29.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:32:29.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:32:29.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:32:29.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:32:29.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:32:29.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:32:29.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:32:29.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:32:29.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:32:29.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:32:29.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:32:29.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:32:29.459 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.459 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.459 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.460 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:32:29.460 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:29.460 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 324.06574317399998, 2023-12-06T18:32:29.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:32:29.460 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.460 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.460 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.460 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:29.460 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.460 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.460 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.460 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:29.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:32:29.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:32:29.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:32:29.461 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:32:29.461 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.461 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.461 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.461 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:32:29.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:32:29.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:32:29.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:32:29.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.462 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.462 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.462 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:32:29.462 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:32:29.462 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:32:29.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:32:29.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:32:29.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:32:29.463 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.463 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.463 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.463 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:32:29.463 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:32:29.464 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.464 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.464 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.464 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.464 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:32:29.464 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:29.464 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 324.05950536900002, 2023-12-06T18:32:29.464 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:32:29.464 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.464 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.464 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.464 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.464 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.464 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:29.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:29.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:32:29.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:32:29.465 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.465 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.465 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.465 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:32:29.465 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:32:29.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:32:29.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:32:29.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:32:29.466 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:32:29.466 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.466 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.466 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.466 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:32:29.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:32:29.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:32:29.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:32:29.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:32:29.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:32:29.467 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.467 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.467 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.467 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:32:29.467 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:32:29.468 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.468 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.468 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.468 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.468 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:32:29.468 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:29.468 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 324.05344229000002, 2023-12-06T18:32:29.468 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:32:29.468 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.468 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.468 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.468 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.468 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:29.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:29.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.469 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:32:29.469 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:32:29.469 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.469 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.469 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.470 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:32:29.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:32:29.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.470 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.470 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:32:29.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:32:29.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.470 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.470 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:32:29.470 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:32:29.470 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.470 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.470 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.470 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:32:29.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:32:29.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:32:29.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:32:29.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:32:29.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:32:29.471 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.471 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.471 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.471 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:32:29.471 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:32:29.472 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.472 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.472 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.472 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.472 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:32:29.472 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:29.472 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 323.91605520500002, 2023-12-06T18:32:29.472 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:32:29.472 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.472 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.472 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.472 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.472 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.472 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:29.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:29.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:32:29.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:32:29.473 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.473 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.473 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.473 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:32:29.473 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:32:29.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:32:29.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:32:29.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:32:29.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:32:29.474 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.474 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.474 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.474 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:32:29.474 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:32:29.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:32:29.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:32:29.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:32:29.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:32:29.475 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.475 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.475 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.475 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:32:29.475 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:32:29.475 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.476 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.476 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.476 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:32:29.476 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:29.476 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 323.90938447799999, 2023-12-06T18:32:29.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:32:29.476 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:29.476 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:29.476 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:29.476 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.476 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:29.476 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:29.476 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.476 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:29.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:29.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:29.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:29.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:32:29.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:32:29.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.477 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:29.477 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:32:29.477 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:32:29.477 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.477 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:29.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:32:29.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:32:29.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:29.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:32:29.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:32:29.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:29.478 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:32:29.478 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:32:29.478 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.478 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.478 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:29.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:32:29.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:32:29.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:29.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:32:29.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:32:29.479 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:29.479 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:29.479 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:29.479 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:32:29.479 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:32:29.479 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.479 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.479 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.479 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:29.480 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:29.480 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:29.481 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:32:29.481 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:29.478+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:29.487 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:32:29.487 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:32:29.582 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:29.579+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:29.588 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:32:29.588 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:32:29.653 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:29.653 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:29.662 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:32:29.678 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:32:29.678 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:32:29.683 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:29.681+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:29.785 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:29.782+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:29.832 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:29.832 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:29.842 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:32:29.887 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:29.883+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:29.972 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:29.972 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:29.981 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:32:30.023 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:30.019+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:30.087 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:30.084+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:30.151 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:30.151 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:30.156 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:32:30.160 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:32:30.201 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:30.198+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:30.289 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:30.286+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:30.331 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:30.331 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:30.340 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:32:30.390 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:30.387+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:30.491 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:30.488+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:30.511 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:30.511 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:30.521 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:32:30.592 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:30.589+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:30.647 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is set 2023-12-06T18:32:30.693 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:30.689+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:30.716 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:30.717 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:30.726 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:32:30.794 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:30.791+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:30.871 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:30.871 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:30.880 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:32:30.895 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:30.891+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:30.899 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":40,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":39,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:32:30.996 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:30.993+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:31.038 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:31.038 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:31.047 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:32:31.132 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:31.129+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:31.198 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:31.194+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:31.225 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:31.225 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:31.234 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:32:31.298 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:31.295+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:31.399 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:31.396+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:31.411 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:31.411 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:31.421 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:32:31.500 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:31.496+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:31.570 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:31.570 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:31.580 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:32:31.624 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:31.620+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:31.690 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:32:31.702 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:31.699+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:31.723 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:31.723 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:31.732 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:32:31.804 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:31.800+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:31.905 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:31.901+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:31.922 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:31.923 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:31.932 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:32:31.951 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:32:31.951 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:32:32.006 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:32.002+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:32.047 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:32:32.047 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:32:32.107 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:32.103+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:32.149 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:32:32.150 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:32:32.153 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:32.153 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:32.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:32.153 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:32:32.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.153 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:32:32.153 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:32.153 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 310.33719634900001, 2023-12-06T18:32:32.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:32:32.154 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:32.154 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:32:32.154 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:32.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:32.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:32.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:32.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:32.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:32.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:32.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:32.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:32:32.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:32:32.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:32.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:32:32.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:32:32.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:32.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:32:32.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:32:32.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:32.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:32:32.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:32:32.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:32.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:32:32.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:32:32.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:32.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:32:32.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:32:32.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:32.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:32:32.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:32:32.157 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:32.157 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:32.157 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:32.157 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:32.157 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:32.157 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:32.162 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:32:32.208 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:32.205+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:32.246 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:32:32.246 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:32:32.304 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:32.304 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:32.304 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:32.304 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:32.308 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:32.305+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:32.313 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:32:32.346 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:32:32.347 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:32:32.410 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:32.406+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:32.435 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:32:32.435 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:32:32.483 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:32.484 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:32.484 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:32.484 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:32.484 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:32.493 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:32:32.524 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:32:32.525 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:32:32.525 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:32.522+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:32.618 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:32:32.618 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:32:32.620 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:32.616+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:32.665 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-12-06T18:32:32.693 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:32.693 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:32.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:32.693 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:32:32.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.693 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:32:32.694 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:32:32.694 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 191.59114622000001, 2023-12-06T18:32:32.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:32:32.694 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:32.694 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:32:32.694 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:32.694 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:32:32.694 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:32.694 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:32:32.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.694 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:32.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:32.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:32:32.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:32.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:32.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:32:32.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:32.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:32.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:32:32.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:32:32.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:32.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:32:32.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:32:32.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:32.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:32:32.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:32:32.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:32.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:32:32.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:32:32.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:32.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:32:32.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:32:32.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.697 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:32.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:32:32.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:32:32.697 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:32.697 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:32.697 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:32.697 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.697 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.697 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:32:32.697 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:32:32.697 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 131.57288891799999, 2023-12-06T18:32:32.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:32:32.697 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:32.698 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:32:32.698 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:32.698 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:32:32.698 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:32.698 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:32:32.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.698 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:32.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:32.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:32:32.698 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:32.698 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.698 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.698 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:32.698 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:32:32.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:32.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:32.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:32:32.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:32:32.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:32.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:32:32.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:32:32.699 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.699 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.699 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:32.699 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:32:32.699 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:32:32.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:32.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:32:32.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:32:32.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:32.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:32:32.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:32:32.700 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.700 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:32.700 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:32:32.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:32:32.700 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:32.701 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:32.701 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:32.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.701 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:11 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2023-12-06T18:32:32.701 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:32:32.701 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 11.548467800999999, 2023-12-06T18:32:32.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0086075449999999994, 2023-12-06T18:32:32.701 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:32.701 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:32:32.701 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:32.701 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:32:32.701 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:32.701 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 11 2023-12-06T18:32:32.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.702 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:32.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:32.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:32:32.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:32.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:32.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:32:32.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:32.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:32.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135441+0000", 2023-12-06T18:32:32.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.8695999999999999e-05 2023-12-06T18:32:32.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:32.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135650+0000", 2023-12-06T18:32:32.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020883699999999999 2023-12-06T18:32:32.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:32.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135863+0000", 2023-12-06T18:32:32.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000213931 2023-12-06T18:32:32.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:32.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.136054+0000", 2023-12-06T18:32:32.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000190416 2023-12-06T18:32:32.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:32.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.142056+0000", 2023-12-06T18:32:32.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0060019330000000001 2023-12-06T18:32:32.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:32.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:32.704 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:32.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.144030+0000", 2023-12-06T18:32:32.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0019737320000000002 2023-12-06T18:32:32.704 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:32.704 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:32.704 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:32.704 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:32.704 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:32.704 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:32.706 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:32:32.747 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:32.743+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:32.812 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:32.808+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:32.836 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:32.836 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:32.836 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:32.836 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:32.845 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:32:32.897 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 7, 5] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:32:32.897 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:32:32.897 INFO:tasks.thrashosds.thrasher:primary_affinity 2023-12-06T18:32:32.897 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.412497 2023-12-06T18:32:32.898 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0.41249651209995075 2023-12-06T18:32:32.942 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:32.938+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:33.013 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:33.009+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:33.018 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:33.018 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:33.018 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:33.018 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:33.018 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:33.027 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:32:33.114 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:33.110+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:33.159 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:33.159 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:33.159 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:33.160 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:33.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.160 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:32:33.160 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:33.160 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 251.66609692200001, 2023-12-06T18:32:33.160 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:32:33.160 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.160 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:32:33.160 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:32:33.160 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:32:33.160 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:33.160 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:32:33.160 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.160 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.160 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.161 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.161 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:33.161 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.161 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.161 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.161 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:33.161 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.161 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.161 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.161 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:32:33.161 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:32:33.161 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.161 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.162 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.162 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:32:33.162 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:32:33.162 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.162 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.162 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.162 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:32:33.162 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:32:33.162 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.162 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.162 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.162 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:32:33.162 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:32:33.162 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.162 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.163 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.163 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:32:33.163 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:32:33.163 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.163 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.163 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.163 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:32:33.163 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:32:33.163 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.163 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.163 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.163 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.163 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.163 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:33.168 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:32:33.215 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:33.212+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:33.298 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:33.298 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:33.299 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:33.299 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:33.307 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:32:33.316 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:33.313+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:33.417 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:33.414+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:33.480 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:33.480 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:33.480 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:33.480 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:33.480 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:33.489 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:32:33.518 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:33.514+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:33.619 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:33.615+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:33.693 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is set 2023-12-06T18:32:33.702 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:33.702 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:33.702 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:33.703 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:33.703 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.703 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:32:33.703 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:33.703 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 337.08406562599998, 2023-12-06T18:32:33.703 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:32:33.703 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.703 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.703 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.703 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.703 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.703 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:33.703 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.703 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.703 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:33.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:32:33.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:32:33.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.704 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.704 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:32:33.704 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:32:33.704 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.704 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:32:33.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:32:33.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:32:33.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:32:33.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.705 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:32:33.705 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:32:33.705 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.705 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.705 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.706 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:32:33.706 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:32:33.706 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.706 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.706 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.706 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:32:33.706 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:32:33.706 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.706 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.706 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.706 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:32:33.706 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:32:33.706 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.706 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.707 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.707 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:32:33.707 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:33.707 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 336.06123861700001, 2023-12-06T18:32:33.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:32:33.707 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.707 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.707 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.707 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.707 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.707 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:33.707 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.707 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:33.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:32:33.708 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:32:33.708 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.708 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.708 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.708 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:32:33.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:32:33.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:32:33.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:32:33.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:32:33.709 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:32:33.709 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.709 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.709 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.709 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:32:33.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:32:33.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.710 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.710 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:32:33.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:32:33.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.710 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.710 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:32:33.710 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:32:33.710 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.710 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.710 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.711 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:32:33.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:32:33.711 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.711 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.711 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.711 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.711 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:32:33.711 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:33.711 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 336.049191979, 2023-12-06T18:32:33.711 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:32:33.711 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.711 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.711 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.711 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:33.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:33.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.712 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:32:33.712 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:32:33.712 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.712 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.712 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:32:33.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:32:33.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:32:33.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:32:33.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.713 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:32:33.713 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:32:33.713 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.713 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.713 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.714 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:32:33.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:32:33.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.714 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.714 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:32:33.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:32:33.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.714 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.714 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:32:33.714 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:32:33.714 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.714 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.714 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.715 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:32:33.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:32:33.715 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.715 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.715 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.715 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.715 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:32:33.715 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:33.715 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 336.02779137200002, 2023-12-06T18:32:33.715 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:32:33.715 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.715 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.715 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.715 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:33.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:33.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.716 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.716 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:32:33.716 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:32:33.716 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.716 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:32:33.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:32:33.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:32:33.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:32:33.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.717 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.717 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:32:33.717 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:32:33.717 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.717 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.718 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.718 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:32:33.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:32:33.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.718 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.718 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:32:33.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:32:33.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.718 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.718 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.718 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:32:33.718 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:32:33.718 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.719 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.719 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:32:33.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:32:33.719 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.719 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.719 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.719 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.719 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.719 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:32:33.719 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:33.719 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 336.02661204899999, 2023-12-06T18:32:33.719 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:32:33.719 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.719 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.719 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:32:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:32:33.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:32:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:32:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:32:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:32:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:32:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:32:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.721 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:32:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:32:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:32:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:32:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:32:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:32:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.722 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:32:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:32:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:32:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 336.02604006799999, 2023-12-06T18:32:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:32:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.723 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:32:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:32:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.724 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:32:33.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:32:33.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:32:33.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:32:33.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.725 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.725 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.725 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:32:33.725 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:32:33.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:32:33.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:32:33.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:32:33.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:32:33.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.726 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.726 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.726 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:32:33.726 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:32:33.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.727 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.727 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:32:33.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:32:33.727 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.727 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.727 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.727 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.727 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:32:33.727 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:33.727 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 335.58020459800002, 2023-12-06T18:32:33.727 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:32:33.727 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.727 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.728 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:33.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:33.728 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.728 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.728 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.728 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:32:33.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:32:33.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:32:33.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:32:33.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:32:33.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:32:33.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.729 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.729 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:32:33.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:32:33.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:32:33.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:32:33.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:32:33.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:32:33.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:32:33.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:32:33.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:32:33.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:32:33.731 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.731 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.731 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.731 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:32:33.731 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:33.731 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 335.57860474199998, 2023-12-06T18:32:33.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:32:33.731 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.732 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.732 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:33.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:33.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:32:33.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:32:33.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:32:33.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:32:33.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:32:33.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:32:33.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:32:33.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:32:33.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:32:33.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:32:33.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:32:33.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:32:33.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:32:33.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:32:33.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:32:33.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:32:33.735 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.735 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.735 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.735 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:32:33.735 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:33.735 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 335.199399126, 2023-12-06T18:32:33.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:32:33.735 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.736 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.736 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:33.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:33.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:32:33.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:32:33.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:32:33.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:32:33.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:32:33.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:32:33.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:32:33.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:32:33.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:32:33.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:32:33.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:32:33.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:32:33.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:32:33.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:32:33.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:32:33.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:32:33.739 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.739 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.739 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.739 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:32:33.739 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:33.739 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 334.96189694399999, 2023-12-06T18:32:33.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:32:33.740 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.740 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.740 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:33.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:33.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:32:33.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:32:33.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:32:33.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:32:33.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:32:33.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:32:33.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:32:33.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:32:33.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:32:33.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:32:33.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:32:33.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:32:33.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:32:33.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:32:33.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:32:33.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:32:33.743 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.743 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.743 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.743 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:32:33.743 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:33.744 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 334.961168945, 2023-12-06T18:32:33.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:32:33.744 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.744 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.744 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:33.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:33.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:32:33.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:32:33.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:32:33.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:32:33.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:32:33.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:32:33.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:32:33.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:32:33.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:32:33.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:32:33.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:32:33.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:32:33.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:32:33.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:32:33.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:32:33.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:32:33.747 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.747 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.747 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.747 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:32:33.748 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:33.748 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 334.95615653499999, 2023-12-06T18:32:33.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:32:33.748 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.748 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.748 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:33.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:33.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:32:33.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:32:33.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:32:33.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:32:33.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:32:33.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:32:33.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.750 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:32:33.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:32:33.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.750 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:32:33.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:32:33.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.750 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.750 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:32:33.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:32:33.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:32:33.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:32:33.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:32:33.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:32:33.751 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.751 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.751 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.751 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:32:33.752 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:33.752 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 334.950629456, 2023-12-06T18:32:33.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:32:33.752 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.752 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.752 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:33.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:33.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:32:33.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:32:33.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:32:33.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:32:33.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:32:33.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:32:33.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:32:33.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:32:33.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:32:33.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:32:33.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:32:33.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:32:33.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:32:33.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:32:33.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:32:33.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:32:33.755 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.755 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.755 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.756 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:32:33.756 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:33.756 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 334.95011861900002, 2023-12-06T18:32:33.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:32:33.756 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.756 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.756 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:33.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:33.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:32:33.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:32:33.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:32:33.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:32:33.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:32:33.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:32:33.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:32:33.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:32:33.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:32:33.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:32:33.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:32:33.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:32:33.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:32:33.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:32:33.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:32:33.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:32:33.759 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.759 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.759 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.760 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:32:33.760 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:33.760 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 334.94965935800002, 2023-12-06T18:32:33.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:32:33.760 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.760 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.760 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.760 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:33.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:33.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:32:33.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:32:33.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:32:33.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:32:33.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:32:33.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:32:33.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:32:33.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:32:33.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:32:33.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:32:33.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:32:33.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:32:33.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:32:33.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:32:33.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:32:33.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:32:33.763 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.763 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.763 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.764 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:32:33.764 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:33.764 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 328.369743551, 2023-12-06T18:32:33.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:32:33.764 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.764 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.764 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:33.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:33.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:32:33.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:32:33.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:32:33.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:32:33.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:32:33.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:32:33.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:32:33.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:32:33.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:32:33.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:32:33.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:32:33.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:32:33.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:32:33.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:32:33.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:32:33.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:32:33.767 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.767 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.767 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.767 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:32:33.768 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:33.768 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 328.36350574599999, 2023-12-06T18:32:33.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:32:33.768 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.768 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.768 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.768 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.768 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:33.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.768 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.768 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:33.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:32:33.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:32:33.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:32:33.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:32:33.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:32:33.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:32:33.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:32:33.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:32:33.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:32:33.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:32:33.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:32:33.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.771 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:32:33.772 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:33.772 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 328.35744266699999, 2023-12-06T18:32:33.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:32:33.772 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.772 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.772 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:33.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:33.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:32:33.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:32:33.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:32:33.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:32:33.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:32:33.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:32:33.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:32:33.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:32:33.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:32:33.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:32:33.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:32:33.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:32:33.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:32:33.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:32:33.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:32:33.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:32:33.775 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.775 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.775 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.775 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:32:33.775 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:33.775 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 328.22005558199999, 2023-12-06T18:32:33.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:32:33.776 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.776 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.776 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:33.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:33.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:32:33.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:32:33.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:32:33.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:32:33.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:32:33.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:32:33.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:32:33.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:32:33.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:32:33.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:32:33.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:32:33.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:32:33.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:32:33.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:32:33.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:32:33.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:32:33.779 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.779 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.779 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.779 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:32:33.779 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:33.779 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 328.21338485500002, 2023-12-06T18:32:33.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:32:33.779 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:33.779 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:33.779 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:33.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:33.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:33.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:33.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:33.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:33.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:33.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:32:33.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:32:33.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:33.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:32:33.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:32:33.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:33.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:32:33.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:32:33.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:33.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:32:33.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:32:33.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:33.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:32:33.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:32:33.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:33.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:32:33.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:32:33.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:33.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:32:33.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:32:33.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:33.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:33.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:33.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:32:33.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:32:33.783 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.783 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.783 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.783 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:33.783 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:33.783 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:33.785 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:32:33.785 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:33.781+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:33.786 INFO:teuthology.orchestra.run.smithi061.stderr:set osd.0 primary-affinity to 0.412497 (8270332) 2023-12-06T18:32:33.885 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:33.882+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:33.915 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:32:33.964 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:33.965 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:33.965 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:33.965 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:33.974 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:32:34.015 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:34.012+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:34.087 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:34.084+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:34.144 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:34.144 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:34.144 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:34.144 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:34.144 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:34.153 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:32:34.198 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:34.194+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:34.289 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:34.285+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:34.299 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:34.299 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:34.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:34.299 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:32:34.299 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:34.310 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:32:34.346 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:07.937 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:34.390 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:34.386+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:34.475 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:34.475 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:34.476 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:34.476 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:34.485 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:32:34.489 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:34.486+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:34.580 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":40,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":42,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:32:34.585 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:08.177 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:34.590 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:34.586+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:34.633 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:32:34.655 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:34.655 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:34.655 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:34.655 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:34.655 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:34.664 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:32:34.708 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:34.704+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:34.792 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:34.788+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:34.797 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:34.798 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:34.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:34.798 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:34.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:34.798 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:32:34.798 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:32:34.798 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 133.133987596, 2023-12-06T18:32:34.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:32:34.798 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:34.798 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:32:34.798 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:32:34.798 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:32:34.798 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:34.799 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:32:34.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:34.799 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:34.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:34.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:34.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:32:34.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:34.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:34.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:34.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:34.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:32:34.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:34.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:34.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:34.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:34.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:32:34.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:32:34.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:34.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:34.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:34.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:32:34.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:32:34.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:34.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:34.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:34.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:32:34.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:32:34.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:34.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:34.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:34.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:32:34.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:32:34.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:34.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:34.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:34.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:32:34.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:32:34.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:34.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:34.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:34.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:32:34.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:32:34.801 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:34.801 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:34.801 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:34.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:34.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:34.802 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:12 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2023-12-06T18:32:34.802 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:32:34.802 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 12.984040971000001, 2023-12-06T18:32:34.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0032177460000000001, 2023-12-06T18:32:34.802 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:34.802 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:32:34.802 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:32:34.802 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:32:34.802 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:34.802 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 12 2023-12-06T18:32:34.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:34.802 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:34.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:34.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:34.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:32:34.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:34.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:34.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:34.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:34.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:32:34.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:34.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:34.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:34.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:34.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807030+0000", 2023-12-06T18:32:34.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6030000000000001e-05 2023-12-06T18:32:34.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:34.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:34.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:34.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807210+0000", 2023-12-06T18:32:34.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017936800000000001 2023-12-06T18:32:34.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:34.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:34.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:34.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807443+0000", 2023-12-06T18:32:34.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00023339699999999999 2023-12-06T18:32:34.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:34.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:34.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:34.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807614+0000", 2023-12-06T18:32:34.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000171032 2023-12-06T18:32:34.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:34.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:34.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:34.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.808696+0000", 2023-12-06T18:32:34.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010817190000000001 2023-12-06T18:32:34.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:34.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:34.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:34.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.810232+0000", 2023-12-06T18:32:34.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0015361999999999999 2023-12-06T18:32:34.805 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:34.805 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:34.805 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:34.805 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:34.805 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:34.805 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:34.807 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:32:34.886 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:32:34.886 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:32:34.893 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:34.889+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:34.985 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:34.985 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:34.994 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:32:34.995 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:34.991+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:35.014 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:32:35.014 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:32:35.095 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:35.091+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:35.108 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:32:35.108 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:32:35.153 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:35.154 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:35.163 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:32:35.196 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:35.192+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:35.201 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:32:35.201 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:32:35.289 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:32:35.289 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:32:35.297 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:35.293+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:35.302 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:35.302 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:35.311 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:32:35.377 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:32:35.377 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:32:35.398 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:35.394+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:35.489 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:35.489 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:35.492 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:32:35.492 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:32:35.498 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:32:35.499 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:35.495+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:35.581 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:32:35.581 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:32:35.635 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:35.631+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:35.681 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:35.681 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:35.690 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:32:35.732 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:35.728+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:35.785 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-12-06T18:32:35.800 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:35.797+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:35.869 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:35.870 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:35.879 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:32:35.902 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:35.898+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:36.003 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:35.999+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:36.021 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:36.021 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:36.031 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:32:36.104 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:36.101+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:36.168 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:36.169 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:36.179 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:32:36.206 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:36.202+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:36.307 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:36.303+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:36.381 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:36.111 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:36.385 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:36.116 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:36.392 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:36.392 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:36.402 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:32:36.408 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:36.139 112393== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:36.409 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:36.405+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:36.439 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:15.884 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:36.440 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:15.886 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:36.462 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:01:15.907 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:36.508 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:36.505+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:36.544 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:36.544 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:36.554 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:32:36.639 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:36.635+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:36.711 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:36.707+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:36.747 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:36.747 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:36.757 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:32:36.841 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:36.837+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:36.913 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:36.909+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:36.936 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:36.936 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:36.945 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:32:37.014 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:37.011+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:37.080 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:37.080 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:37.090 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:32:37.115 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:37.112+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:37.216 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:37.212+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:37.220 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:37.220 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:37.229 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:32:37.317 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:37.314+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:37.357 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:37.358 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:37.367 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:32:37.452 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:37.448+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:37.495 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is unset 2023-12-06T18:32:37.539 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:37.539 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:37.549 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:32:37.552 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:37.548+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:37.594 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:32:37.594 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:32:37.640 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:37.636+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:37.720 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:37.720 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:37.721 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:37.717+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:37.729 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:32:37.821 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:37.818+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:37.862 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:32:37.862 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:32:37.900 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:37.900 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:37.909 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:32:37.921 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:11.512 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:37.922 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:37.918+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:37.970 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:32:37.970 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:32:38.051 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:38.051 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:38.060 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:32:38.062 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:32:38.062 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:32:38.063 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:38.059+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:38.123 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:38.120+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:38.142 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:11.733 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:38.194 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:38.194 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:38.195 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:32:38.195 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:32:38.203 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:32:38.247 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:38.244+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:38.286 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:32:38.286 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:32:38.325 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:38.321+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:38.342 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:38.342 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:38.352 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:32:38.359 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":40,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"c","quorum_age":46,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:32:38.382 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:32:38.382 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:32:38.425 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:12.016 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:32:38.425 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:38.422+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:38.478 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:32:38.479 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:32:38.526 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:38.523+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:38.578 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:32:38.578 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:32:38.579 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:38.579 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:38.579 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:38.579 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:38.590 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:32:38.631 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:38.628+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:38.728 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:38.725+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:38.767 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:38.767 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:38.767 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:38.767 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:38.767 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:38.776 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:32:38.786 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 7, 5] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:32:38.787 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:32:38.787 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [0, 1, 2, 3, 4, 7, 5] 2023-12-06T18:32:38.787 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2023-12-06T18:32:38.861 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:38.857+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:38.930 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:38.926+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:38.949 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:32:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:32:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 137.285436479, 2023-12-06T18:32:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:32:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:38.949 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:32:38.950 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:32:38.950 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:32:38.950 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:38.950 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:32:38.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:38.950 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:38.950 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:38.950 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:38.950 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:32:38.950 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:38.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:38.950 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:38.950 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:38.951 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:32:38.951 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:38.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:38.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:38.951 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:38.951 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:32:38.951 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:32:38.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:38.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:38.951 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:38.951 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:32:38.951 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:32:38.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:38.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:38.951 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:38.952 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:32:38.952 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:32:38.952 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:38.952 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:38.952 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:38.952 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:32:38.952 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:32:38.952 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:38.952 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:38.952 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:38.952 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:32:38.952 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:32:38.952 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:38.952 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:38.952 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:38.953 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:32:38.953 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:32:38.953 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:38.953 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:38.953 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:38.953 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:38.953 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:38.953 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:12 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2023-12-06T18:32:38.953 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:32:38.953 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 17.135489853999999, 2023-12-06T18:32:38.953 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0032177460000000001, 2023-12-06T18:32:38.953 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:38.953 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:32:38.953 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:32:38.953 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:32:38.954 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:38.954 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 12 2023-12-06T18:32:38.954 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:38.954 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:38.954 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:38.954 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:38.954 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:32:38.954 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:38.954 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:38.954 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:38.954 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:38.954 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:32:38.954 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:38.954 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:38.954 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:38.955 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:38.955 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807030+0000", 2023-12-06T18:32:38.955 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6030000000000001e-05 2023-12-06T18:32:38.955 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:38.955 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:38.955 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:38.955 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807210+0000", 2023-12-06T18:32:38.955 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017936800000000001 2023-12-06T18:32:38.955 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:38.955 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:38.955 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:38.955 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807443+0000", 2023-12-06T18:32:38.955 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00023339699999999999 2023-12-06T18:32:38.955 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:38.955 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:38.955 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:38.956 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807614+0000", 2023-12-06T18:32:38.956 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000171032 2023-12-06T18:32:38.956 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:38.956 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:38.956 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:38.956 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.808696+0000", 2023-12-06T18:32:38.956 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010817190000000001 2023-12-06T18:32:38.956 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:38.956 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:38.956 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:38.956 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.810232+0000", 2023-12-06T18:32:38.956 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0015361999999999999 2023-12-06T18:32:38.956 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:38.956 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:38.956 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:38.957 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:38.957 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:38.957 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:38.958 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:32:39.031 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:39.027+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:39.106 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:39.106 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:39.106 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:39.106 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:39.118 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:32:39.160 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:39.155+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:39.233 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:39.229+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:39.298 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:39.298 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:39.298 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:39.299 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:39.299 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:39.307 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:32:39.334 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:39.331+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:39.435 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:39.432+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:39.447 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:39.447 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:39.447 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:39.447 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:32:39.447 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:39.456 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:32:39.510 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-12-06T18:32:39.537 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:39.534+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:39.629 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:39.629 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:39.630 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:39.630 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:39.637 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:39.634+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:39.640 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:32:39.730 INFO:tasks.ceph.mon.c.smithi061.stderr:2023-12-06T18:32:39.725+0000 ff5f640 -1 mon.c@2(leader).osd e96 definitely_dead 0 2023-12-06T18:32:39.738 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:39.734+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:39.833 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:39.833 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:39.834 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:39.834 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:39.834 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:39.839 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:39.836+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:39.842 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:32:39.941 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:39.937+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:40.042 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:40.038+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:40.097 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:40.097 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:40.097 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:40.097 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:32:40.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.097 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:32:40.097 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:32:40.097 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 344.66725656599999, 2023-12-06T18:32:40.098 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:32:40.098 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.098 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.098 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.098 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.098 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.098 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:32:40.098 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.098 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.098 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.098 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.098 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:32:40.098 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.098 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.098 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.098 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.099 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:32:40.099 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.099 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.099 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.099 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:32:40.099 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:32:40.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.099 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.099 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.099 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:32:40.099 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:32:40.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.099 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.099 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.099 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:32:40.099 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:32:40.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.100 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:32:40.100 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:32:40.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.100 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:32:40.100 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:32:40.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:32:40.100 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:32:40.100 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:32:40.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:32:40.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:32:40.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:32:40.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:32:40.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:32:40.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:32:40.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:32:40.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:32:40.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:32:40.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:32:40.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:32:40.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:32:40.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.103 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.103 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.103 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:32:40.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:32:40.103 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.103 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.103 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.103 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.103 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:40.103 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:32:40.103 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 342.55868332699998, 2023-12-06T18:32:40.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:32:40.103 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.103 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.103 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.104 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.104 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.104 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:32:40.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.104 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.104 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.104 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:32:40.104 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.104 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.104 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:32:40.104 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.105 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.105 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:32:40.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:32:40.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.105 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.105 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:32:40.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:32:40.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.105 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.105 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:32:40.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:32:40.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.105 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.106 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:32:40.106 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:32:40.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.106 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.106 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:32:40.106 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:32:40.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.106 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.106 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:32:40.106 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:32:40.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.106 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:32:40.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:32:40.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.107 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:32:40.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:32:40.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.107 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:32:40.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:32:40.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.107 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:32:40.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:32:40.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.108 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.108 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:32:40.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:32:40.108 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.108 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.108 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.108 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:40.108 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:32:40.108 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 342.557938095, 2023-12-06T18:32:40.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:32:40.108 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.109 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.109 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.109 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.109 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.109 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:32:40.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.109 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.109 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.109 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.109 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:32:40.109 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.109 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.109 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.109 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:32:40.109 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.110 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.110 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.110 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.110 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:32:40.110 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:32:40.110 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.110 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.110 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.110 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:32:40.110 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:32:40.110 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.110 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.110 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.110 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:32:40.110 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:32:40.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:32:40.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:32:40.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:32:40.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:32:40.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:32:40.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:32:40.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:32:40.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:32:40.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:32:40.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:32:40.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:32:40.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:32:40.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:32:40.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:32:40.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:32:40.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:32:40.113 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.113 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.113 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.113 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:40.113 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:32:40.113 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 342.55725433999999, 2023-12-06T18:32:40.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:32:40.114 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.114 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.114 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.114 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.114 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.114 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:32:40.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.114 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.114 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:32:40.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:32:40.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:32:40.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:32:40.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:32:40.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:32:40.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:32:40.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:32:40.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:32:40.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:32:40.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:32:40.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:32:40.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:32:40.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:32:40.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:32:40.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:32:40.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:32:40.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:32:40.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:32:40.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:32:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:32:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:32:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:32:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:32:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.118 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:32:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 342.55168108499998, 2023-12-06T18:32:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:32:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:32:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:32:40.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:32:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:32:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:32:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:32:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:32:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:32:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:32:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:32:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:32:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:32:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:32:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:32:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:32:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:32:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:32:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:32:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:32:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:32:40.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:32:40.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:32:40.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:32:40.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:32:40.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:32:40.123 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.123 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.123 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.124 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.124 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:40.124 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:32:40.124 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 342.55100744499998, 2023-12-06T18:32:40.124 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:32:40.124 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.124 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.124 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.124 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.124 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.124 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:32:40.124 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.124 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.124 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.124 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.124 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:32:40.124 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.125 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.125 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:32:40.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.125 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.125 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:32:40.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:32:40.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.125 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.125 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:32:40.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:32:40.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.126 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.126 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:32:40.126 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:32:40.126 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.126 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.126 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.126 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:32:40.126 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:32:40.126 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.126 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.126 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.126 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:32:40.126 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:32:40.126 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.127 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.127 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:32:40.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:32:40.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.127 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.127 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:32:40.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:32:40.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.127 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.127 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:32:40.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:32:40.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:32:40.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:32:40.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.128 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:32:40.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:32:40.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.128 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:32:40.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:32:40.128 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.128 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.128 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.129 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.129 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:40.129 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:32:40.129 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 342.55034524600001, 2023-12-06T18:32:40.129 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:32:40.129 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.129 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.129 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.129 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.129 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.129 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:32:40.129 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.129 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.129 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.129 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.129 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:32:40.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.130 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.130 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.130 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:32:40.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.130 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.130 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.130 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:32:40.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:32:40.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.130 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.130 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.130 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:32:40.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:32:40.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:32:40.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:32:40.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:32:40.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:32:40.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:32:40.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:32:40.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:32:40.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:32:40.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:32:40.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:32:40.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:32:40.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:32:40.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:32:40.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:32:40.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:32:40.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:32:40.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:32:40.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:32:40.134 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.134 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.134 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.134 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:40.134 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:32:40.134 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 342.47763581100003, 2023-12-06T18:32:40.134 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:32:40.134 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.134 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.134 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.134 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.134 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.134 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:32:40.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.135 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.135 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.135 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:32:40.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.135 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.135 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:32:40.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.135 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.135 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:32:40.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:32:40.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:32:40.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:32:40.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:32:40.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:32:40.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:32:40.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:32:40.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.137 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:32:40.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:32:40.137 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.137 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:32:40.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:32:40.137 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.137 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:32:40.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:32:40.137 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.138 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:32:40.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:32:40.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.138 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:32:40.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:32:40.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.138 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:32:40.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:32:40.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:32:40.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:32:40.139 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.139 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.139 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.139 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:40.139 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:32:40.139 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 342.47695410199998, 2023-12-06T18:32:40.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:32:40.139 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.139 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.139 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.139 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.139 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.139 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:32:40.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.140 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:32:40.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:32:40.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:32:40.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:32:40.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:32:40.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:32:40.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:32:40.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:32:40.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:32:40.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:32:40.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:32:40.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:32:40.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:32:40.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:32:40.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:32:40.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:32:40.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:32:40.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:32:40.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.143 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:32:40.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:32:40.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.143 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:32:40.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:32:40.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.143 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:32:40.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:32:40.143 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.143 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.143 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.144 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:40.144 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:32:40.144 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 341.39815001400001, 2023-12-06T18:32:40.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:32:40.144 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.144 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.144 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.144 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.144 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.144 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:32:40.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.144 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:32:40.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:32:40.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:32:40.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:32:40.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:32:40.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:32:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:32:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:32:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:32:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:32:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:32:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:32:40.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:32:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:32:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:32:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:32:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:32:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:32:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:32:40.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:32:40.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:32:40.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:32:40.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:32:40.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:32:40.148 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.148 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.148 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.149 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:40.149 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:32:40.149 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 341.39673343999999, 2023-12-06T18:32:40.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:32:40.149 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.149 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.149 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.149 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.149 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.149 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:32:40.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.149 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:32:40.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:32:40.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:32:40.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:32:40.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:32:40.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:32:40.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.151 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:32:40.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:32:40.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.151 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:32:40.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:32:40.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.151 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:32:40.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:32:40.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.151 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:32:40.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:32:40.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:32:40.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:32:40.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:32:40.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:32:40.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:32:40.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:32:40.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.153 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.153 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:32:40.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:32:40.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.153 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.153 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:32:40.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:32:40.153 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.153 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.153 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.153 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:40.153 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:32:40.154 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 341.39564831500002, 2023-12-06T18:32:40.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:32:40.154 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.154 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.154 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.154 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.154 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.154 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:32:40.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.154 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:32:40.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:32:40.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:32:40.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:32:40.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:32:40.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:32:40.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:32:40.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:32:40.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:32:40.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:32:40.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:32:40.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:32:40.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:32:40.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:32:40.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:32:40.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:32:40.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:32:40.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:32:40.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:32:40.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:32:40.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.158 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:32:40.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:32:40.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.158 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:32:40.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:32:40.158 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.158 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.158 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.159 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:40.159 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:32:40.159 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 341.36049921300003, 2023-12-06T18:32:40.159 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:32:40.159 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.159 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.159 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.159 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.159 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.159 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:32:40.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.159 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.159 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:32:40.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:32:40.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:32:40.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:32:40.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:32:40.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:32:40.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:32:40.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:32:40.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:32:40.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:32:40.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:32:40.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:32:40.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.162 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.162 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:32:40.162 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:32:40.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.162 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.162 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:32:40.162 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:32:40.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.162 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.162 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:32:40.162 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:32:40.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.163 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.163 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.163 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:32:40.163 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:32:40.163 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.163 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.163 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.163 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:32:40.163 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:32:40.163 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.163 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.163 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.163 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:32:40.163 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:32:40.163 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.164 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.164 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.164 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.164 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:40.164 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:32:40.164 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 334.810768688, 2023-12-06T18:32:40.164 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:32:40.164 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.164 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.164 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.164 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.164 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.164 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:32:40.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.164 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:32:40.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:32:40.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:32:40.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:32:40.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.166 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.166 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.166 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:32:40.166 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:32:40.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.166 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.166 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.166 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:32:40.166 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:32:40.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.166 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.166 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.166 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:32:40.166 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:32:40.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.167 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.167 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.167 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:32:40.167 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:32:40.167 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.167 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.167 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.167 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:32:40.167 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:32:40.167 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.167 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.167 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.167 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:32:40.167 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:32:40.167 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.167 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.168 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.168 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:32:40.168 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:32:40.168 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.168 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.168 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.168 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:32:40.168 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:32:40.168 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.168 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.168 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.168 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:32:40.168 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:32:40.168 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.168 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.168 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.169 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:32:40.169 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:32:40.169 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.169 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.169 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.169 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.169 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.169 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:40.169 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:32:40.169 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 334.80996307999999, 2023-12-06T18:32:40.169 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:32:40.169 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.169 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.169 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.169 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.170 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.170 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:32:40.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.170 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.170 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.170 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.170 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:32:40.170 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.170 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.170 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.170 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:32:40.170 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.170 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.170 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.171 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.171 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:32:40.171 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:32:40.171 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.171 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.171 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.171 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:32:40.171 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:32:40.171 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.171 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.171 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.171 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:32:40.171 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:32:40.171 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.171 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.172 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.172 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:32:40.172 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:32:40.172 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.172 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.172 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.172 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:32:40.172 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:32:40.172 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.172 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.172 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.172 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:32:40.172 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:32:40.172 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.172 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.173 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.173 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:32:40.173 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:32:40.173 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.173 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.173 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.173 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:32:40.173 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:32:40.173 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.173 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.173 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.173 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:32:40.173 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:32:40.173 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.173 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.174 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.174 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:32:40.174 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:32:40.174 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.174 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.174 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.174 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:32:40.174 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:32:40.174 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.174 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.174 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.174 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.174 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.174 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:40.174 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:32:40.175 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 334.809272271, 2023-12-06T18:32:40.175 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:32:40.175 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.175 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.175 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.175 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.175 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.175 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:32:40.175 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.175 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.175 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.175 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.175 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:32:40.175 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.175 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.175 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.175 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.176 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:32:40.176 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.176 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.176 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.176 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.176 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:32:40.176 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:32:40.176 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.176 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.176 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.176 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:32:40.176 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:32:40.176 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.176 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.176 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.176 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:32:40.176 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:32:40.176 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.177 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.177 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.177 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:32:40.177 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:32:40.177 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.177 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.177 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.177 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:32:40.177 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:32:40.177 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.177 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.177 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.177 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:32:40.177 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:32:40.177 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.177 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.178 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.178 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:32:40.178 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:32:40.178 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.178 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.178 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.178 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:32:40.178 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:32:40.178 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.178 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.178 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.178 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:32:40.178 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:32:40.178 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.178 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.178 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.179 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:32:40.179 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:32:40.179 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.179 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.179 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.179 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:32:40.179 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:32:40.179 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.179 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.179 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.179 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.179 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.179 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:40.179 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:32:40.179 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 334.80497476300002, 2023-12-06T18:32:40.180 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:32:40.180 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.180 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.180 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.180 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.180 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.180 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:32:40.180 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.180 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.180 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.180 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.180 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:32:40.180 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.180 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.180 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.180 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.180 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:32:40.181 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.181 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.181 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.181 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.181 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:32:40.181 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:32:40.181 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.181 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.181 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.181 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:32:40.181 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:32:40.181 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.181 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.181 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.181 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:32:40.181 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:32:40.182 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.182 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.182 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.182 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:32:40.182 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:32:40.182 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.182 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.182 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.182 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:32:40.182 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:32:40.182 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.182 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.182 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.182 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:32:40.183 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:32:40.183 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.183 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.183 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.183 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:32:40.183 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:32:40.183 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.183 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.183 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.183 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:32:40.183 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:32:40.183 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.183 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.183 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.183 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:32:40.183 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:32:40.183 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.184 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.184 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.184 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:32:40.184 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:32:40.184 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.184 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.184 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.184 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:32:40.184 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:32:40.184 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.184 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.184 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.184 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.184 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.184 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:40.185 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:32:40.185 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 334.80430708799997, 2023-12-06T18:32:40.185 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:32:40.185 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.185 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.185 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.185 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.185 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.185 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:32:40.185 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.185 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.185 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.185 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.185 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:32:40.185 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.186 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.186 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.186 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.186 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:32:40.186 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.186 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.186 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.186 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.186 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:32:40.186 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:32:40.186 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.186 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.186 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.186 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:32:40.186 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:32:40.186 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.187 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.187 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.187 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:32:40.187 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:32:40.187 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.187 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.187 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.187 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:32:40.187 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:32:40.187 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.187 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.187 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.187 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:32:40.187 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:32:40.187 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.187 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.187 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.187 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:32:40.188 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:32:40.188 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.188 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.188 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.188 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:32:40.188 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:32:40.188 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.188 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.188 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.188 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:32:40.188 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:32:40.188 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.188 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.188 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.188 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:32:40.188 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:32:40.188 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.189 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.189 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.189 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:32:40.189 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:32:40.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.189 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.189 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.189 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:32:40.189 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:32:40.189 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.189 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.189 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.189 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.189 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.189 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:40.190 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:32:40.190 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 334.80365166000001, 2023-12-06T18:32:40.190 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:32:40.190 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.190 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.190 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.190 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.190 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.190 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:32:40.190 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.190 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.190 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.190 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.190 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:32:40.190 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.190 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.191 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.191 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.191 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:32:40.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.191 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.191 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.191 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.191 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:32:40.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:32:40.191 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.191 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.191 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.191 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:32:40.191 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:32:40.191 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.191 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.191 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.192 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:32:40.192 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:32:40.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.192 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.192 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.192 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:32:40.192 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:32:40.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.192 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.192 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.192 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:32:40.192 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:32:40.192 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.192 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.192 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.192 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:32:40.192 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:32:40.193 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.193 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.193 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.193 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:32:40.193 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:32:40.193 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.193 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.193 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.193 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:32:40.193 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:32:40.193 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.193 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.193 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.193 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:32:40.193 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:32:40.194 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.194 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.194 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.194 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:32:40.194 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:32:40.194 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.194 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.194 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.194 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:32:40.194 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:32:40.194 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.194 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.194 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.194 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.194 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.194 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:40.195 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:32:40.195 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 334.76299711299998, 2023-12-06T18:32:40.195 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:32:40.195 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.195 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:40.195 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:40.195 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:40.195 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:40.195 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:32:40.195 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.195 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.195 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.195 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.195 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:32:40.195 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.195 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.195 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.196 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.196 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:32:40.196 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.196 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.196 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.196 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.196 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:32:40.196 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:32:40.196 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.196 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.196 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.196 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:32:40.196 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:32:40.196 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.196 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.196 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.197 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:32:40.197 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:32:40.197 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.197 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.197 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.197 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:32:40.197 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:32:40.197 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.197 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.197 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.197 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:32:40.197 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:32:40.197 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.197 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.197 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.197 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:32:40.198 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:32:40.198 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.198 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.198 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:40.198 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:32:40.198 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:32:40.198 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.198 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.198 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:40.198 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:32:40.198 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:32:40.198 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.198 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.198 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:40.198 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:32:40.199 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:32:40.199 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.199 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.199 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:40.199 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:32:40.199 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:32:40.199 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.199 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.199 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.199 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:32:40.199 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:32:40.199 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.199 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.199 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.199 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.199 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.200 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:40.201 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:32:40.201 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:40.198+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:40.302 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:40.299+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:40.384 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:40.384 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:40.397 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:32:40.403 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:40.400+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:40.504 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:40.501+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:40.533 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:40.534 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:40.534 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:40.534 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:40.534 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:40.543 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:32:40.591 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:32:40.636 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:40.632+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:40.693 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:40.693 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:40.693 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:40.693 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:32:40.693 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.693 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:32:40.693 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:40.694 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 318.87696955899997, 2023-12-06T18:32:40.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:32:40.694 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:40.694 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:32:40.694 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:40.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:40.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:40.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.694 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.694 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.694 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:40.694 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:40.694 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:40.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:40.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:32:40.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:32:40.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:40.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:32:40.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:32:40.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.695 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:40.695 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:32:40.695 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:32:40.695 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.695 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:40.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:32:40.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:32:40.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:40.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:32:40.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:32:40.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:40.696 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:32:40.696 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:32:40.696 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:40.696 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:40.696 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:40.697 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:32:40.697 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:32:40.697 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.697 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.697 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.697 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:40.697 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:40.697 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:40.705 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:40.702+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:40.706 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:32:40.807 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:40.803+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:40.832 INFO:teuthology.orchestra.run.smithi061.stderr:marked out osd.4. 2023-12-06T18:32:40.896 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is unset 2023-12-06T18:32:40.908 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:40.904+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:40.932 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:40.932 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:40.933 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:40.933 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:40.941 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:32:40.949 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:32:40.949 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:32:41.009 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:41.005+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:41.062 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:32:41.062 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:32:41.099 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:41.100 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:41.100 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:41.100 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:41.100 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:41.110 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:41.106+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:41.112 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:32:41.166 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:32:41.166 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:32:41.211 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:41.207+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:41.258 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:32:41.258 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:32:41.311 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:41.307+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:41.311 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:41.311 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:41.324 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:32:41.347 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:32:41.347 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:32:41.374 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:32:41.419 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:41.415+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:41.447 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:32:41.448 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:32:41.466 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:41.466 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:41.478 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:32:41.513 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:41.509+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:41.551 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:32:41.551 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:32:41.613 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:41.610+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:41.614 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:41.614 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:41.624 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:32:41.648 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:32:41.648 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:32:41.714 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:41.710+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:41.753 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:41.753 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:41.764 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:32:41.815 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:41.812+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:41.916 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:41.913+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:41.917 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:41.917 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:41.927 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:32:42.017 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:42.014+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:42.092 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":42,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"c","quorum_age":3,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:32:42.096 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:42.096 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:42.105 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:32:42.106 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2023-12-06T18:32:42.107 INFO:tasks.mon_recovery:removing mon c 2023-12-06T18:32:42.107 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-12-06T18:32:42.107 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:32:42.146 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:42.143+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:42.163 INFO:tasks.ceph.mon.c.smithi061.stderr:2023-12-06T18:32:42.148+0000 9f54640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 112384) UID: 0 2023-12-06T18:32:42.163 INFO:tasks.ceph.mon.c.smithi061.stderr:2023-12-06T18:32:42.149+0000 9f54640 -1 mon.c@2(leader) e1 *** Got Signal Terminated *** 2023-12-06T18:32:42.219 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:42.215+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:42.299 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:42.300 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:42.309 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:32:42.320 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:42.317+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:42.422 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:42.418+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:42.448 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:42.449 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:42.461 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:32:42.545 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:42.542+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:42.623 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:42.620+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:42.646 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:42.647 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:42.655 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:32:42.699 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:42.429 112393== 2023-12-06T18:32:42.699 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:42.429 112393== Exit program on first error (--exit-on-first-error=yes) 2023-12-06T18:32:42.711 INFO:tasks.ceph.mon.c.smithi061.stderr:daemon-helper: command failed with exit status 42 2023-12-06T18:32:42.760 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:42.756+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:42.823 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:42.824 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:42.824 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:42.820+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:42.833 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:32:42.915 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-12-06T18:32:42.925 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:42.921+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:43.003 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:43.003 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:43.012 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:32:43.025 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:43.022+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:43.126 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:43.123+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:43.179 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:43.179 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:43.187 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:32:43.227 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:43.224+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:43.315 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:43.315 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:43.324 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:32:43.328 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:43.324+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:43.429 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:43.425+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:43.462 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:43.463 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:43.472 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:32:43.530 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:43.526+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:43.608 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:43.608 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:43.617 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:32:43.631 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:43.628+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:43.660 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:32:43.733 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:43.729+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:43.801 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:43.802 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:43.811 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:32:43.834 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:43.830+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:43.916 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:32:43.916 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:32:43.934 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:43.931+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:44.035 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:44.032+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:44.045 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:32:44.045 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:32:44.045 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:44.045 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:44.046 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:44.046 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:44.056 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:32:44.136 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:44.132+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:44.168 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:32:44.168 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:32:44.229 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:44.230 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:44.230 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:44.230 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:44.230 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:44.236 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:44.233+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:44.239 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:32:44.276 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:32:44.276 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:32:44.337 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:44.334+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:44.364 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:32:44.365 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:32:44.439 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:44.435+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:44.451 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:32:44.451 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:32:44.471 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:44.471 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:44.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:44.471 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:32:44.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.472 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:32:44.472 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:32:44.472 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 349.061013575, 2023-12-06T18:32:44.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:32:44.472 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.472 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.472 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.472 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.472 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.472 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:32:44.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.472 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:32:44.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:32:44.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:32:44.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:32:44.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:32:44.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:32:44.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:32:44.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:32:44.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:32:44.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:32:44.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:32:44.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:32:44.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:32:44.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:32:44.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:32:44.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:32:44.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:32:44.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:32:44.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:32:44.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:32:44.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:32:44.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:32:44.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:32:44.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:32:44.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:32:44.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:32:44.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:32:44.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.478 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:32:44.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:32:44.478 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.478 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.478 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.478 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:44.478 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:32:44.478 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 346.952440336, 2023-12-06T18:32:44.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:32:44.478 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.478 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.478 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.479 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.479 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.479 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:32:44.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.479 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:32:44.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:32:44.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:32:44.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:32:44.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:32:44.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:32:44.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:32:44.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:32:44.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:32:44.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:32:44.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:32:44.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:32:44.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:32:44.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:32:44.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:32:44.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:32:44.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:32:44.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:32:44.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:32:44.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:32:44.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:32:44.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:32:44.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:32:44.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:32:44.483 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.484 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.484 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.484 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.484 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.484 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:44.484 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:32:44.484 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 346.95169510400001, 2023-12-06T18:32:44.484 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:32:44.484 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.484 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.484 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.484 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.484 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.484 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:32:44.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.485 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.485 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.485 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:32:44.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:32:44.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:32:44.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:32:44.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:32:44.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:32:44.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:32:44.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:32:44.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:32:44.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:32:44.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:32:44.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:32:44.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:32:44.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:32:44.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.488 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:32:44.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:32:44.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:32:44.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:32:44.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:32:44.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:32:44.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:32:44.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:32:44.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.490 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.490 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:32:44.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:32:44.490 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.490 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.490 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.490 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:44.490 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:32:44.490 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 346.951011349, 2023-12-06T18:32:44.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:32:44.491 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.491 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.491 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.491 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.491 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.491 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:32:44.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.491 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:32:44.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:32:44.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:32:44.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:32:44.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:32:44.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:32:44.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:32:44.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:32:44.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:32:44.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:32:44.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.496 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:32:44.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:32:44.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.496 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:32:44.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:32:44.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:32:44.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:32:44.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:32:44.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:32:44.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:32:44.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:32:44.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:32:44.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:32:44.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:32:44.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:32:44.498 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.498 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.498 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.498 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:44.498 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:32:44.498 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 346.945438094, 2023-12-06T18:32:44.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:32:44.499 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.499 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.499 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.499 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.499 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:32:44.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.499 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:32:44.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:32:44.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:32:44.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:32:44.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:32:44.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:32:44.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.501 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:32:44.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:32:44.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.501 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.501 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:32:44.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:32:44.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.501 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.501 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:32:44.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:32:44.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:32:44.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:32:44.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:32:44.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:32:44.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:32:44.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:32:44.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:32:44.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:32:44.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:32:44.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:32:44.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:32:44.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:32:44.503 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.503 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.503 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.504 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:44.504 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:32:44.504 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 346.94476445399999, 2023-12-06T18:32:44.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:32:44.504 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.504 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.504 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.504 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.504 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.504 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:32:44.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.504 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.504 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:32:44.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:32:44.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:32:44.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:32:44.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:32:44.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:32:44.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:32:44.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:32:44.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:32:44.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:32:44.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:32:44.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:32:44.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:32:44.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:32:44.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:32:44.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:32:44.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:32:44.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:32:44.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:32:44.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:32:44.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:32:44.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:32:44.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:32:44.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:32:44.509 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.509 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.509 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.509 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:44.509 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:32:44.509 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 346.94410225500002, 2023-12-06T18:32:44.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:32:44.509 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.509 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.509 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.509 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.510 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.510 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:32:44.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.510 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.510 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.510 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:32:44.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.510 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.510 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:32:44.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:32:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:32:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:32:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:32:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:32:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:32:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:32:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:32:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:32:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:32:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:32:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:32:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:32:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:32:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:32:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:32:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:32:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:32:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:32:44.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:32:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:32:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:32:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:32:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 346.87139281999998, 2023-12-06T18:32:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:32:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.514 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:32:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:32:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:32:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:32:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:32:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:32:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:32:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:32:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:32:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.516 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:32:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:32:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:32:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:32:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:32:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:32:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:32:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:32:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:32:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:32:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:32:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:32:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:32:44.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:32:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:32:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:32:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:32:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:32:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 346.87071111099999, 2023-12-06T18:32:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:32:44.519 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.520 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.520 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.520 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.520 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.520 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:32:44.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.520 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:32:44.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:32:44.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:32:44.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:32:44.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:32:44.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:32:44.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:32:44.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:32:44.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.522 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:32:44.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:32:44.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.522 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.522 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:32:44.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:32:44.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.522 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.522 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:32:44.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:32:44.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.522 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:32:44.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:32:44.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:32:44.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:32:44.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:32:44.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:32:44.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:32:44.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:32:44.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:32:44.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:32:44.524 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.524 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.524 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.524 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:44.524 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:32:44.524 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 345.79190702300002, 2023-12-06T18:32:44.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:32:44.524 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.524 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.524 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.525 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.525 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.525 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:32:44.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.525 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:32:44.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:32:44.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:32:44.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:32:44.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:32:44.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:32:44.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:32:44.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:32:44.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:32:44.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:32:44.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:32:44.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:32:44.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:32:44.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:32:44.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:32:44.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:32:44.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:32:44.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:32:44.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:32:44.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:32:44.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:32:44.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:32:44.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:32:44.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:32:44.529 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.529 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.529 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.529 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:44.529 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:32:44.529 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 345.790490449, 2023-12-06T18:32:44.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:32:44.529 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.529 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.529 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.530 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.530 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.530 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:32:44.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.530 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:32:44.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:32:44.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:32:44.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:32:44.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:32:44.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:32:44.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:32:44.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:32:44.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:32:44.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:32:44.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:32:44.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:32:44.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:32:44.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:32:44.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:32:44.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:32:44.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:32:44.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:32:44.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:32:44.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:32:44.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:32:44.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:32:44.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:32:44.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:32:44.537 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.537 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.537 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.538 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:44.538 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:32:44.538 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 345.78940532399997, 2023-12-06T18:32:44.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:32:44.538 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.538 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.539 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.539 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.539 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.539 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:32:44.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.539 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:32:44.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:32:44.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:32:44.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:32:44.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:32:44.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:32:44.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:32:44.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:32:44.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:32:44.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:32:44.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:32:44.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:32:44.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:32:44.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:32:44.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:32:44.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:32:44.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:32:44.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:32:44.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:32:44.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:32:44.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:32:44.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:32:44.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:32:44.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:32:44.544 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.544 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.545 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.545 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:32:44.545 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:32:44.545 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 345.75425622199998, 2023-12-06T18:32:44.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:32:44.545 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.545 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.545 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.545 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.545 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.545 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:32:44.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.546 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:32:44.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:32:44.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:32:44.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:32:44.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:32:44.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:32:44.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:32:44.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:32:44.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:32:44.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:32:44.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:32:44.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:32:44.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:32:44.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:32:44.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:32:44.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:32:44.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:32:44.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:32:44.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:32:44.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:32:44.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:32:44.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:32:44.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:32:44.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:32:44.550 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.550 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.550 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.550 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:44.551 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:32:44.551 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 339.20452569700001, 2023-12-06T18:32:44.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:32:44.551 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.551 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.551 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.551 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.551 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.551 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:32:44.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.551 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:32:44.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:32:44.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:32:44.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:32:44.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:32:44.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:32:44.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:32:44.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:32:44.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:32:44.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:32:44.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:32:44.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:32:44.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:32:44.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:32:44.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:32:44.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:32:44.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:32:44.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:32:44.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:32:44.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:32:44.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:32:44.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:32:44.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:32:44.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:32:44.555 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.555 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.556 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.556 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:44.556 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:32:44.556 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 339.203720089, 2023-12-06T18:32:44.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:32:44.556 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.556 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.556 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.556 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.556 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.556 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:32:44.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.556 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.556 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:32:44.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:32:44.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:32:44.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:32:44.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:32:44.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:32:44.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:32:44.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:32:44.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:32:44.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:32:44.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:32:44.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:32:44.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:32:44.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:32:44.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:32:44.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:32:44.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:32:44.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:32:44.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:32:44.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:32:44.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:32:44.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:32:44.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:32:44.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:32:44.560 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.560 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.561 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.561 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.561 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:44.561 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:32:44.561 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 339.20302928000001, 2023-12-06T18:32:44.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:32:44.561 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.561 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.561 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.561 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.561 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.561 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:32:44.561 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.561 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:32:44.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.562 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.562 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.562 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.562 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:32:44.562 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.563 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:32:44.563 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:32:44.563 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.563 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.563 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:32:44.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:32:44.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:32:44.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:32:44.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:32:44.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:32:44.564 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.564 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.564 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.564 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:32:44.564 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:32:44.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:32:44.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:32:44.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:32:44.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:32:44.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.565 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.565 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:32:44.565 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:32:44.565 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.565 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:32:44.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:32:44.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:32:44.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:32:44.566 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.566 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.566 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.566 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:32:44.566 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:32:44.566 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.566 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.567 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.567 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.567 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:44.567 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:32:44.567 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 339.19873177199997, 2023-12-06T18:32:44.567 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:32:44.567 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.567 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.567 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.567 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.567 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.567 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:32:44.567 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.567 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.568 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.568 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:32:44.568 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.568 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.568 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:32:44.568 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.568 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.568 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:32:44.568 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:32:44.568 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.568 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:32:44.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:32:44.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:32:44.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:32:44.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.569 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.569 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:32:44.569 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:32:44.569 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.569 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.570 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.570 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:32:44.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:32:44.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.570 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.570 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:32:44.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:32:44.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.570 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.570 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.570 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:32:44.570 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:32:44.570 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:32:44.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:32:44.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:32:44.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:32:44.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.571 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.571 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:32:44.571 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:32:44.571 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.571 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.572 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.572 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:32:44.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:32:44.572 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.572 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.572 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.572 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.572 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:44.572 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:32:44.572 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 339.19806409699999, 2023-12-06T18:32:44.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:32:44.572 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.572 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.572 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.572 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.572 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.573 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:32:44.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.573 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.573 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.573 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:32:44.573 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.573 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.573 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:32:44.573 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.573 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.573 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:32:44.573 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:32:44.574 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:32:44.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:32:44.574 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:32:44.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:32:44.574 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:32:44.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:32:44.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.575 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.575 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:32:44.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:32:44.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.575 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.575 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:32:44.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:32:44.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.575 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.575 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:32:44.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:32:44.576 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.576 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.576 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.576 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:32:44.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:32:44.576 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.576 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.576 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.576 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:32:44.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:32:44.576 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.576 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.576 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.576 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:32:44.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:32:44.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.577 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.577 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:32:44.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:32:44.577 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.577 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.577 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.577 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:44.577 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:32:44.577 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 339.19740866900003, 2023-12-06T18:32:44.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:32:44.577 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.577 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.578 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.578 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.578 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.578 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:32:44.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.578 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:32:44.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:32:44.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.579 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.579 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.579 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:32:44.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:32:44.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.579 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.579 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.579 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:32:44.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:32:44.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.579 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.579 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.579 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:32:44.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:32:44.580 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.580 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.580 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:32:44.580 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:32:44.580 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.580 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.580 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:32:44.580 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:32:44.580 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.580 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.580 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:32:44.580 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:32:44.580 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.581 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:32:44.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:32:44.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.581 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.581 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:32:44.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:32:44.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.581 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.581 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:32:44.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:32:44.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.581 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.581 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.582 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:32:44.582 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:32:44.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.582 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.582 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:32:44.582 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:32:44.582 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.582 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.582 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.582 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:32:44.583 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:32:44.583 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 339.156754122, 2023-12-06T18:32:44.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:32:44.583 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:44.583 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:44.583 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:44.583 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:32:44.583 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:32:44.583 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:32:44.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.583 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:44.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:44.583 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:32:44.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:44.584 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:32:44.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:44.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:44.584 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:32:44.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:32:44.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:44.584 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:32:44.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:32:44.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.585 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:44.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:32:44.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:32:44.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.585 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.585 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:44.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:32:44.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:32:44.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.585 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.585 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:44.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:32:44.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:32:44.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.585 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.585 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:44.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:32:44.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:32:44.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:32:44.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:32:44.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:32:44.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:32:44.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:32:44.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:32:44.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:32:44.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:32:44.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:32:44.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:32:44.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:32:44.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:32:44.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:44.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:44.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:44.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:32:44.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:32:44.587 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.587 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.587 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.588 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:44.588 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:44.588 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:44.589 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:32:44.590 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:32:44.590 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:32:44.590 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:44.586+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:44.622 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:32:44.623 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:32:44.691 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:44.687+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:44.760 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:44.760 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:44.760 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:44.760 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:44.770 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:32:44.791 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:44.788+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:44.892 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:44.889+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:44.947 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:44.948 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:44.948 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:44.948 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:44.948 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:44.957 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:32:45.000 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:44.995+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:45.097 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:45.091+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:45.136 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:45.137 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:45.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:45.137 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:32:45.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.137 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:32:45.137 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:45.137 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 323.32087985999999, 2023-12-06T18:32:45.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:32:45.137 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:45.137 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:32:45.137 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:45.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:45.138 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:45.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:45.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:45.138 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:32:45.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:45.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:45.138 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:32:45.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:32:45.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:45.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:32:45.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:32:45.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:45.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:32:45.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:32:45.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:45.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:32:45.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:32:45.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:45.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:32:45.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:32:45.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:32:45.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:32:45.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:32:45.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:45.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:32:45.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:32:45.140 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:45.140 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:45.140 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:45.140 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:45.140 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:45.141 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:45.147 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:32:45.196 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:45.192+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:45.311 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:45.307+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:45.325 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:45.326 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:45.326 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:32:45.326 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:45.336 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:32:45.397 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:45.394+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:45.498 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:45.495+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:45.515 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:45.515 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:32:45.515 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:32:45.515 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:32:45.515 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:45.526 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:32:45.599 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:45.596+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:45.672 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:32:45.672 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:32:45.672 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:32:45.672 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:32:45.672 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.672 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:32:45.673 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:32:45.673 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 204.57106193499999, 2023-12-06T18:32:45.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:32:45.673 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:45.673 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:32:45.673 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:45.673 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:32:45.673 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:45.673 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:32:45.673 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.673 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:45.673 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.673 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:45.673 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:32:45.673 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:45.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:45.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:32:45.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:45.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:45.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:32:45.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:32:45.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.674 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.674 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:45.674 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:32:45.674 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:32:45.674 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:45.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:32:45.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:32:45.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:45.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:32:45.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:32:45.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.675 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.675 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:45.675 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:32:45.675 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:32:45.675 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.676 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.676 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:45.676 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:32:45.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:32:45.676 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:45.676 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:45.676 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:45.676 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.676 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.676 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:32:45.676 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:32:45.676 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 144.55280463299999, 2023-12-06T18:32:45.676 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:32:45.676 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:45.676 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:32:45.676 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:45.677 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:32:45.677 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:45.677 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:32:45.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.677 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:45.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:45.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:32:45.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:45.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:45.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:32:45.677 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:45.677 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.677 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.677 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:45.677 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:32:45.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:32:45.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:45.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:32:45.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:32:45.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:45.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:32:45.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:32:45.678 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.678 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.678 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:45.678 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:32:45.678 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:32:45.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:45.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:32:45.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:32:45.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.679 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:45.679 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:32:45.679 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:32:45.679 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:45.679 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:45.679 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:45.679 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.679 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.679 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:11 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2023-12-06T18:32:45.679 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:32:45.680 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 24.528383516000002, 2023-12-06T18:32:45.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0086075449999999994, 2023-12-06T18:32:45.680 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:32:45.680 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:32:45.680 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:32:45.680 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:32:45.680 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:45.680 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 11 2023-12-06T18:32:45.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.680 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:32:45.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:32:45.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:32:45.680 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:45.680 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.680 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.680 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:32:45.680 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:32:45.681 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:32:45.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:32:45.681 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135441+0000", 2023-12-06T18:32:45.681 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.8695999999999999e-05 2023-12-06T18:32:45.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:32:45.681 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135650+0000", 2023-12-06T18:32:45.681 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020883699999999999 2023-12-06T18:32:45.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.681 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:32:45.681 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135863+0000", 2023-12-06T18:32:45.681 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000213931 2023-12-06T18:32:45.681 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.681 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.682 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:32:45.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.136054+0000", 2023-12-06T18:32:45.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000190416 2023-12-06T18:32:45.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.682 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.682 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:32:45.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.142056+0000", 2023-12-06T18:32:45.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0060019330000000001 2023-12-06T18:32:45.682 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:32:45.682 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:32:45.682 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:32:45.682 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.144030+0000", 2023-12-06T18:32:45.682 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0019737320000000002 2023-12-06T18:32:45.682 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:45.682 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:45.682 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:45.683 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:32:45.683 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:32:45.683 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:32:45.684 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:32:45.700 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:45.696+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:45.801 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:45.797+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:45.823 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:45.823 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:45.823 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:45.823 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:45.833 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:32:45.851 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 7, 5] out_osds: [6, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:32:45.852 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:32:45.918 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:45.914+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:46.001 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:46.001 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:46.001 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:46.001 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:46.002 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:46.003 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:45.999+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:46.010 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:32:46.110 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:46.104+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:46.184 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:32:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 264.69159570900001, 2023-12-06T18:32:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:32:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:32:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:32:46.185 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:32:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:32:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.186 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:32:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:32:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:32:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:32:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:32:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:32:46.187 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:32:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:32:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:32:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:32:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:32:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:32:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.188 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.189 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.189 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.189 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.189 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:46.195 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:32:46.237 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:46.233+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:46.306 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:46.303+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:46.365 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:46.365 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:46.365 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:46.365 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:46.374 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:32:46.414 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:46.411+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:46.508 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:46.504+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:46.542 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:46.542 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:46.542 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:46.542 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:46.542 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:46.551 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:32:46.609 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:46.605+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:46.635 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:32:46.709 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:46.705+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:46.768 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 350.151598992, 2023-12-06T18:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:32:46.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 349.12877198299998, 2023-12-06T18:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.772 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.775 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 349.11672534500002, 2023-12-06T18:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:32:46.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:32:46.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.779 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 349.09532473799999, 2023-12-06T18:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:32:46.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:32:46.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 349.09414541500001, 2023-12-06T18:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:46.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:32:46.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:32:46.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.787 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 349.09357343400001, 2023-12-06T18:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 348.64773796399999, 2023-12-06T18:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.791 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 348.646138108, 2023-12-06T18:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:46.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:32:46.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:32:46.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:32:46.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 348.26693249200002, 2023-12-06T18:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:46.802 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 348.02943031000001, 2023-12-06T18:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:32:46.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:32:46.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 348.02870231100002, 2023-12-06T18:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:32:46.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 348.02368990100001, 2023-12-06T18:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:46.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.812 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.812 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:32:46.812 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:32:46.812 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.812 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.812 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.812 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:32:46.812 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:32:46.812 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.812 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.812 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.812 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:32:46.812 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:32:46.812 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.812 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.812 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.813 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:32:46.813 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:32:46.813 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.813 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.813 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.813 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:32:46.813 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:32:46.813 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.813 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.813 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.813 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.813 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.813 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:32:46.813 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:46.813 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 348.01816282200002, 2023-12-06T18:32:46.813 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:32:46.813 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.813 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.814 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.814 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.814 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.814 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:46.814 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.814 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.814 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.814 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.814 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:46.814 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.814 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.814 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.814 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.814 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:32:46.814 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:32:46.814 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.815 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.815 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.815 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:32:46.815 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:32:46.815 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.815 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.815 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.815 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:32:46.815 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:32:46.815 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.815 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.815 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.815 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:32:46.815 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:32:46.815 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.815 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.815 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.816 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:32:46.816 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:32:46.816 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.816 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.816 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.816 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:32:46.816 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:32:46.816 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.816 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.816 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.816 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:32:46.816 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:32:46.816 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.816 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.816 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.816 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:32:46.816 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:32:46.817 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.817 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.817 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.817 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.817 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.817 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:32:46.817 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:46.817 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 348.01765198499999, 2023-12-06T18:32:46.817 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:32:46.817 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.817 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.817 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.817 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.817 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.817 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:46.817 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.817 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.818 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.818 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.818 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:46.818 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.818 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.818 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.818 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.818 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:32:46.818 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:32:46.818 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.818 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.818 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.818 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:32:46.818 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:32:46.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.819 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:32:46.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:32:46.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.819 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:32:46.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:32:46.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.819 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:32:46.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:32:46.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:32:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:32:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:32:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:32:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:32:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:32:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.821 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:32:46.821 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:46.821 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 348.01719272399998, 2023-12-06T18:32:46.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:32:46.821 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.821 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.821 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.821 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.821 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:46.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.821 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.821 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:46.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.822 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.822 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:32:46.822 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:32:46.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.822 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.822 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.822 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:32:46.822 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:32:46.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.822 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.822 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.822 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:32:46.822 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:32:46.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.822 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.823 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.823 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:32:46.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:32:46.823 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.823 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.823 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:32:46.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:32:46.823 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.823 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.823 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:32:46.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:32:46.823 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.823 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.824 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:32:46.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:32:46.824 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.824 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.824 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.824 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:32:46.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:32:46.824 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.824 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.824 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.824 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.824 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.824 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:32:46.824 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:46.824 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 341.43727691700002, 2023-12-06T18:32:46.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:32:46.824 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.824 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.825 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.825 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:46.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.825 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:46.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.825 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:32:46.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:32:46.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:32:46.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:32:46.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:32:46.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:32:46.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:32:46.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:32:46.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:32:46.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:32:46.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:32:46.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:32:46.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:32:46.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:32:46.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.828 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.828 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:32:46.828 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:32:46.828 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.828 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.828 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.828 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.828 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.828 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:32:46.828 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:46.828 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 341.43103911200001, 2023-12-06T18:32:46.828 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:32:46.828 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.828 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.828 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.828 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.829 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.829 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:46.829 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.829 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.829 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.829 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.829 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:46.829 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.829 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.829 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.829 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.829 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:32:46.829 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:32:46.829 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.829 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.829 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.829 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:32:46.829 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:32:46.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.830 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.830 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.830 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:32:46.830 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:32:46.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.830 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.830 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.830 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:32:46.830 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:32:46.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.830 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.830 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.831 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:32:46.831 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:32:46.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.831 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.831 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.831 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:32:46.831 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:32:46.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.831 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.831 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.831 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:32:46.831 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:32:46.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.831 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.832 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.832 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:32:46.832 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:32:46.832 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.832 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.832 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.832 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.832 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.832 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:32:46.832 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:46.832 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 341.42497603300001, 2023-12-06T18:32:46.832 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:32:46.832 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.832 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.832 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.832 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.832 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.832 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:46.833 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.833 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.833 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.833 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:46.833 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.833 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.833 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.833 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:32:46.833 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:32:46.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.833 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.833 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.833 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:32:46.833 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:32:46.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.834 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.834 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.834 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:32:46.834 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:32:46.834 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.834 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.834 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.834 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:32:46.834 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:32:46.834 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.834 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.834 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.834 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:32:46.834 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:32:46.834 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.835 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.835 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.835 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:32:46.835 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:32:46.835 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.835 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.835 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.835 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:32:46.835 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:32:46.835 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.835 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.835 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.835 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:32:46.835 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:32:46.835 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.835 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.835 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:32:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 341.28758894800001, 2023-12-06T18:32:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:32:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.836 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:46.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.837 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.837 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.837 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:32:46.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:32:46.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.837 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.837 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.837 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:32:46.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:32:46.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.837 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.837 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.837 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:32:46.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:32:46.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.838 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.838 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:32:46.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:32:46.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.838 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.838 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:32:46.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:32:46.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.838 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.838 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:32:46.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:32:46.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.839 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.839 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:32:46.839 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:32:46.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.839 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.839 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.839 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:32:46.839 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:32:46.839 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.839 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.839 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.839 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.839 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:32:46.839 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:46.839 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 341.28091822099998, 2023-12-06T18:32:46.839 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:32:46.840 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:46.840 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:46.840 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:46.840 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.840 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:46.840 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:46.840 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.840 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.840 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:46.840 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:46.840 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:46.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.840 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.840 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:46.840 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:32:46.840 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:32:46.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.841 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:46.841 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:32:46.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:32:46.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.841 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:46.841 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:32:46.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:32:46.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.841 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:46.841 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:32:46.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:32:46.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:46.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:32:46.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:32:46.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:46.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:32:46.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:32:46.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:46.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:32:46.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:32:46.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:46.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:46.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:46.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:32:46.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:32:46.843 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.843 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.843 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.843 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:46.843 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:46.843 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:46.844 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:32:46.844 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:46.841+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:46.894 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:32:46.894 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:32:46.945 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:46.942+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:46.987 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:32:46.987 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:32:47.012 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:47.012 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:47.012 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:47.012 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:47.022 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:32:47.064 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:47.060+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:47.080 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:32:47.081 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:32:47.147 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:47.143+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:47.171 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:32:47.171 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:32:47.191 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:47.192 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:47.192 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:47.192 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:47.192 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:47.201 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:32:47.260 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:32:47.261 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:32:47.288 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:47.285+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:47.349 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:32:47.349 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:32:47.349 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:47.346+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:47.385 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:47.385 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:47.385 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:47.385 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:32:47.385 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:47.394 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:32:47.450 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:32:47.450 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:32:47.450 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:47.446+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:47.537 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:32:47.538 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:32:47.574 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:47.574 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:47.583 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:32:47.591 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:47.587+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:47.652 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:47.649+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:47.709 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:47.709 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:47.718 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:32:47.763 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:47.759+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:47.845 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:47.846 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:47.854 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:47.851+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:47.855 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:32:47.958 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:47.954+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:47.993 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:47.993 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:48.004 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:32:48.056 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:48.052+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:48.140 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:48.141 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:48.150 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:32:48.157 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:48.153+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:48.208 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-12-06T18:32:48.209 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi061 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c" 2023-12-06T18:32:48.209 INFO:tasks.ceph.mon.c:Stopped 2023-12-06T18:32:48.209 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2023-12-06T18:32:48.209 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:32:48.257 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:48.254+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:48.304 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-12-06T18:32:48.328 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:48.328 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:48.337 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:32:48.358 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:48.355+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:48.459 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:48.455+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:48.480 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:48.480 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:48.489 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:32:48.560 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:48.556+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:48.661 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:48.657+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:48.667 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:48.668 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:48.677 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:32:48.761 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:48.758+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:48.854 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:48.855 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:48.862 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:48.859+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:48.865 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:32:48.902 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":42,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"c","quorum_age":9,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:32:48.963 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:48.960+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:49.004 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:49.004 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:49.013 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:32:49.064 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:49.061+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:49.150 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:49.150 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:49.159 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:32:49.200 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:49.197+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:49.266 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:49.263+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:49.336 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:49.336 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:49.346 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:32:49.388 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:49.383+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:49.469 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:49.465+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:49.528 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:49.528 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:49.537 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:32:49.550 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:32:49.570 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:49.567+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:49.671 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:49.668+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:49.679 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:49.679 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:49.689 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:32:49.772 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:49.768+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:49.827 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:49.827 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:49.837 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:32:49.879 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:49.875+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:49.964 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:49.964 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:49.973 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:32:49.974 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:49.970+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:50.075 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:50.071+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:50.102 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:50.102 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:50.111 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:32:50.176 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:50.172+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:50.244 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:50.244 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:50.254 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:32:50.295 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:50.292+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:50.378 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:50.375+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:50.423 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:50.423 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:50.433 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:32:50.517 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:50.513+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:50.580 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:50.577+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:50.601 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:50.602 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:50.611 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:32:50.681 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:50.678+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:50.740 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:50.741 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:50.750 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:32:50.791 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:50.788+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:50.853 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 7, 5] out_osds: [6, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:32:50.853 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:32:50.883 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:50.879+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:50.918 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:50.918 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:50.927 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:32:51.011 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:51.008+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:51.084 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:51.081+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:51.096 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:51.096 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:51.105 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:32:51.186 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:51.182+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:51.275 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:51.275 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:51.285 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:32:51.287 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:51.283+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:51.387 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:51.384+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:51.414 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:51.414 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:51.423 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:32:51.488 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:51.484+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:51.551 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:51.551 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:51.560 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:32:51.603 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:51.598+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:51.690 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:51.687+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:51.728 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:51.728 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:51.737 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:32:51.791 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:51.788+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:51.887 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:51.887 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:51.892 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:51.888+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:51.898 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:32:51.917 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:32:51.992 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:51.989+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:52.040 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:52.040 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:52.050 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:32:52.094 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:52.090+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:52.189 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:52.189 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:52.195 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:52.191+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:52.198 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:32:52.295 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:52.292+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:52.379 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:52.380 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:52.389 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:32:52.396 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:52.392+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:52.497 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:52.493+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:52.573 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:52.574 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:52.584 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:32:52.625 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:52.621+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:52.698 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:52.695+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:52.761 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:52.761 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:52.770 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:32:52.799 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:52.796+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:52.901 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:52.897+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:52.909 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:52.909 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:52.920 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:32:53.002 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:52.998+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:53.056 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:53.056 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:53.065 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:32:53.102 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:53.099+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:53.203 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:53.203 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:53.203 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:53.200+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:53.213 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:32:53.305 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:53.301+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:53.350 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:53.350 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:53.359 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:32:53.443 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:53.440+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:53.506 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:53.503+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:53.536 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:53.536 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:53.545 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:32:53.630 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:53.626+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:53.708 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:53.704+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:53.723 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:53.723 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:53.733 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:32:53.809 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:53.805+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:53.873 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:53.873 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:53.882 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:32:53.926 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:53.923+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:54.010 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:54.007+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:54.011 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:54.011 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:54.021 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:32:54.111 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:54.108+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:54.192 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:54.192 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:54.201 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:32:54.212 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:54.209+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:54.313 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:54.310+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:54.370 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:54.370 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:54.379 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:32:54.414 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:54.410+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:54.513 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-12-06T18:32:54.514 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:54.511+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:54.550 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:54.550 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:54.559 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:32:54.615 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:54.611+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:54.687 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:54.687 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:54.696 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:32:54.739 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:54.734+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:54.817 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:54.813+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:54.864 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:54.864 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:54.873 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:32:54.917 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:54.914+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:55.009 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:55.009 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:55.018 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:55.015+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:55.019 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:32:55.119 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:55.116+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:55.145 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:55.145 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:55.154 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:32:55.220 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:55.216+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:55.282 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:55.282 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:55.291 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:32:55.321 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:55.318+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:55.419 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:55.419 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:55.421 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:55.418+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:55.429 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:32:55.523 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:55.519+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:55.597 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:55.598 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:55.607 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:32:55.652 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:55.648+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:55.724 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:55.720+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:55.758 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:55.758 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:55.767 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:32:55.852 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:55.848+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:55.854 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 7, 5] out_osds: [6, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:32:55.854 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:32:55.854 INFO:tasks.thrashosds.thrasher:Adding osd 4 2023-12-06T18:32:55.854 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2023-12-06T18:32:55.925 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:55.922+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:55.948 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:55.948 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:55.958 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:32:56.026 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:56.023+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:56.093 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:56.093 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:56.103 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:32:56.127 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:56.124+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:56.228 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:56.224+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:56.240 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:56.240 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:56.250 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:32:56.329 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:56.326+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:56.387 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:56.387 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:56.396 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:32:56.430 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:56.427+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:56.532 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:56.528+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:56.576 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:56.576 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:56.587 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:32:56.633 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:56.629+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:56.727 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:56.728 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:56.733 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:56.730+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:56.737 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:32:56.834 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:56.831+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:56.914 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:56.914 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:56.923 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:32:56.935 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:56.932+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:57.035 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:57.032+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:57.126 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:57.126 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:57.137 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:32:57.138 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:57.135+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:57.237 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:57.233+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:57.296 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:57.296 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:57.306 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:32:57.377 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:32:57.378 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:32:57.378 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:57.374+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:57.442 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:57.438+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:57.456 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:57.456 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:57.466 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:32:57.499 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:32:57.500 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:32:57.544 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:57.540+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:57.640 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:32:57.640 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:32:57.640 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:57.637+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:57.718 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:32:57.718 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:32:57.731 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:32:57.776 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:57.772+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:57.780 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:32:57.780 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:32:57.834 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is set 2023-12-06T18:32:57.842 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:32:57.839+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:32:57.888 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:32:57.889 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:32:57.926 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:57.927 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:57.927 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:57.927 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:57.936 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:32:57.975 INFO:tasks.ceph.mon.a.smithi061.stderr:2023-12-06T18:32:57.959+0000 ff5f640 -1 mon.a@0(leader).osd e99 definitely_dead 0 2023-12-06T18:32:57.981 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:57.978+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:58.000 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:32:58.000 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:32:58.043 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:58.040+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:58.075 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:58.075 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:58.075 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:58.075 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:58.075 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:58.084 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:32:58.095 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:32:58.095 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:32:58.169 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:58.165+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:58.183 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:32:58.183 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:32:58.247 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:32:58.243+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:32:58.257 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:58.257 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:58.257 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:58.257 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:58.257 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.257 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:32:58.257 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:58.257 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 276.76364554100002, 2023-12-06T18:32:58.257 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:32:58.257 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.257 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:32:58.257 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:32:58.257 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:32:58.258 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:58.258 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:32:58.258 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.258 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.258 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.258 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.258 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:58.258 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.258 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.258 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.258 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.258 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:32:58.258 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.258 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.259 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.259 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.259 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:32:58.259 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:32:58.259 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.259 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.259 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.259 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:32:58.259 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:32:58.259 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.259 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.259 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.259 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:32:58.259 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:32:58.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.260 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.260 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.260 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:32:58.260 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:32:58.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.260 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.260 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.260 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:32:58.260 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:32:58.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.260 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.260 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.260 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:32:58.260 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:32:58.260 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.261 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.261 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.261 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.261 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.261 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:58.266 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:32:58.351 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:58.347+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:58.434 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:58.434 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:58.434 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:58.434 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:58.443 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:32:58.447 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:58.444+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:58.548 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:32:58.545+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:32:58.614 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:58.614 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:58.614 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:58.614 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:58.615 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:58.622 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":44,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:32:58.623 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:32:58.636 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2023-12-06T18:32:58.636 INFO:tasks.mon_recovery:causing some monitor log activity 2023-12-06T18:32:58.636 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '1 of 30' 2023-12-06T18:32:58.666 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:58.661+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:58.750 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:58.746+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:58.839 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:58.839 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:58.839 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:58.839 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:58.839 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.840 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:32:58.840 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:58.840 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 362.22162261900002, 2023-12-06T18:32:58.840 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:32:58.840 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.840 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.840 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.840 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.840 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.840 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:58.840 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.840 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.840 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.840 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:32:58.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.841 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.841 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:32:58.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:32:58.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.841 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.841 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:32:58.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:32:58.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.841 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:32:58.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:32:58.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:32:58.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:32:58.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:32:58.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:32:58.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:32:58.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:32:58.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:32:58.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:32:58.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:32:58.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:32:58.843 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.843 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.844 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.844 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:32:58.844 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:58.844 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 361.19879560999999, 2023-12-06T18:32:58.844 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:32:58.844 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.844 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.844 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:58.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.845 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.845 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.845 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.845 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:32:58.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.845 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.845 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.845 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.845 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:32:58.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:32:58.845 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.845 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.845 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.846 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:32:58.846 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:32:58.846 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.846 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.846 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.846 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:32:58.846 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:32:58.846 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.846 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.846 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.846 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:32:58.846 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:32:58.846 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.846 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.846 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:32:58.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:32:58.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:32:58.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:32:58.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:32:58.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:32:58.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:32:58.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:32:58.848 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.848 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.848 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.848 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:32:58.848 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:58.848 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 361.18674897199998, 2023-12-06T18:32:58.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:32:58.848 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.848 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.848 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.849 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:58.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.849 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.849 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:32:58.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.849 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.849 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:32:58.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:32:58.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.849 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.849 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:32:58.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:32:58.850 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.850 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.850 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:32:58.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:32:58.850 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.850 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.850 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:32:58.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:32:58.850 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.850 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.851 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:32:58.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:32:58.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.851 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.851 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:32:58.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:32:58.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.851 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.851 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:32:58.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:32:58.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.851 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:32:58.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:32:58.852 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.852 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.852 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.852 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:32:58.852 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:58.852 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 361.165348365, 2023-12-06T18:32:58.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:32:58.852 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.852 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.852 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.853 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:58.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.853 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.853 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.853 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.853 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:32:58.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.853 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.853 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.853 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.853 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:32:58.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:32:58.853 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.853 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.854 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.854 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:32:58.854 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:32:58.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.854 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.854 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:32:58.854 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:32:58.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.854 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.854 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:32:58.854 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:32:58.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:32:58.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:32:58.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:32:58.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:32:58.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:32:58.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:32:58.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:32:58.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:32:58.856 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.856 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.856 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.856 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:32:58.856 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:58.856 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 361.16416904200003, 2023-12-06T18:32:58.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:32:58.856 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.857 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.857 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:58.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:32:58.857 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.857 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.857 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.857 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.857 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:32:58.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:32:58.858 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:32:58.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:32:58.858 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:32:58.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:32:58.858 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.858 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.858 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.858 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:32:58.858 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:32:58.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:32:58.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:32:58.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:32:58.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:32:58.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.859 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.859 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:32:58.859 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:32:58.859 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.859 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.860 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.860 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:32:58.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:32:58.860 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.860 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.860 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.860 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.860 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:32:58.860 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:58.860 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 361.16359706100002, 2023-12-06T18:32:58.860 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:32:58.860 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.860 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.860 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.860 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:58.861 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:32:58.861 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:32:58.861 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:32:58.861 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.861 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.861 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.861 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:32:58.862 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:32:58.862 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.862 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.862 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.862 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:32:58.862 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:32:58.862 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.862 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.862 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.862 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:32:58.862 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:32:58.862 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.862 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.862 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.862 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:32:58.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:32:58.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:32:58.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:32:58.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:32:58.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:32:58.863 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.863 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.863 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.863 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:32:58.863 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:32:58.863 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.864 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.864 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.864 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:32:58.864 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:58.864 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 360.717761591, 2023-12-06T18:32:58.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:32:58.864 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.864 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.864 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.864 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.864 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.864 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:58.864 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.864 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:32:58.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:32:58.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:32:58.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.865 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.865 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.865 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:32:58.865 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:32:58.865 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.866 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:32:58.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:32:58.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.866 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:32:58.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:32:58.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.866 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.866 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:32:58.866 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:32:58.866 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.866 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.867 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:32:58.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:32:58.867 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.867 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:32:58.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:32:58.867 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.867 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.867 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.867 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:32:58.867 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:32:58.867 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.867 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.867 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.867 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.868 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:32:58.868 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:58.868 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 360.71616173500001, 2023-12-06T18:32:58.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:32:58.868 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.868 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.868 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:58.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.868 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.868 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.868 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.868 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:32:58.868 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:32:58.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:32:58.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:32:58.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:32:58.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.869 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:32:58.869 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:32:58.869 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.869 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.869 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:32:58.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:32:58.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:32:58.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:32:58.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.870 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:32:58.870 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:32:58.870 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.870 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.870 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.871 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:32:58.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:32:58.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.871 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.871 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:32:58.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:32:58.871 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.871 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.871 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.871 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.871 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.871 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:32:58.871 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:58.871 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 360.33695611899998, 2023-12-06T18:32:58.871 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:32:58.872 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.872 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.872 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:58.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.872 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.872 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:32:58.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.872 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:32:58.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:32:58.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:32:58.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:32:58.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:32:58.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:32:58.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:32:58.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:32:58.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:32:58.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:32:58.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:32:58.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:32:58.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:32:58.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:32:58.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:32:58.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:32:58.875 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.875 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.875 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.875 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:32:58.875 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:58.875 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 360.09945393700002, 2023-12-06T18:32:58.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:32:58.875 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.875 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.875 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:58.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:32:58.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:32:58.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:32:58.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:32:58.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:32:58.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:32:58.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:32:58.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:32:58.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:32:58.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:32:58.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:32:58.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:32:58.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:32:58.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:32:58.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:32:58.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:32:58.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:32:58.879 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.879 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.879 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.879 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:32:58.879 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:58.879 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 360.09872593799997, 2023-12-06T18:32:58.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:32:58.879 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.879 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.879 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:58.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:32:58.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.880 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.880 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.880 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:32:58.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:32:58.880 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:32:58.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:32:58.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:32:58.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:32:58.881 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.881 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.881 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.881 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:32:58.881 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:32:58.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:32:58.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:32:58.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.882 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:32:58.882 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:32:58.882 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.882 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.882 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:32:58.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:32:58.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.883 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.883 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:32:58.883 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:32:58.883 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.883 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.883 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.883 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.883 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.883 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:32:58.883 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:58.883 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 360.09371352800002, 2023-12-06T18:32:58.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:32:58.884 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.884 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.884 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.884 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.884 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.884 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:58.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.884 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.884 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.884 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.884 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:32:58.884 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.884 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.884 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.884 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:32:58.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:32:58.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:32:58.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:32:58.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:32:58.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:32:58.885 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.885 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.885 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.885 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:32:58.885 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:32:58.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:32:58.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:32:58.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:32:58.886 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:32:58.886 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.886 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.886 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.886 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:32:58.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:32:58.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.887 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.887 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:32:58.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:32:58.887 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.887 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.887 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.887 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.887 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.887 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:32:58.887 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:58.887 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 360.08818644899998, 2023-12-06T18:32:58.887 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:32:58.887 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:32:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:32:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:32:58.888 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.889 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.889 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:32:58.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:32:58.889 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.889 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.889 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:32:58.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:32:58.889 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.889 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.889 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:32:58.889 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:32:58.889 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.889 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:32:58.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:32:58.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:32:58.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:32:58.890 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.890 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.890 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.890 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:32:58.890 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:32:58.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.891 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.891 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:32:58.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:32:58.891 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.891 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.891 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.891 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.891 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.891 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:32:58.891 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:58.891 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 360.087675612, 2023-12-06T18:32:58.891 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:32:58.891 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.892 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.892 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:58.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:32:58.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.892 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.892 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.892 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.892 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:32:58.892 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:32:58.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.893 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.893 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:32:58.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:32:58.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.893 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.893 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:32:58.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:32:58.893 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.893 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.893 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.893 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:32:58.893 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:32:58.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:32:58.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:32:58.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:32:58.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:32:58.894 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.894 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.894 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.894 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:32:58.894 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:32:58.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.895 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.895 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:32:58.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:32:58.895 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.895 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.895 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.895 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.895 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.895 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:32:58.895 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:58.895 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 360.087216351, 2023-12-06T18:32:58.895 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:32:58.895 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.895 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.896 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:58.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:32:58.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.896 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.896 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.896 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.896 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:32:58.896 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:32:58.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.897 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.897 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:32:58.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:32:58.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.897 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.897 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:32:58.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:32:58.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.897 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.897 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.897 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:32:58.897 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:32:58.897 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:32:58.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:32:58.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:32:58.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:32:58.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.898 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.898 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.898 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:32:58.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:32:58.898 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:32:58.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:32:58.899 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.899 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.899 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.899 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:32:58.899 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:58.899 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 353.50730054399997, 2023-12-06T18:32:58.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:32:58.899 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.899 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.899 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:58.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:32:58.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:32:58.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:32:58.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:32:58.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:32:58.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:32:58.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:32:58.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:32:58.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:32:58.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:32:58.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:32:58.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:32:58.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:32:58.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:32:58.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:32:58.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:32:58.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:32:58.903 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.903 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.903 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.903 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:32:58.903 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:58.903 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 353.50106273900002, 2023-12-06T18:32:58.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:32:58.903 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.903 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.903 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:58.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:32:58.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:32:58.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:32:58.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:32:58.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:32:58.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:32:58.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:32:58.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:32:58.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:32:58.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:32:58.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:32:58.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:32:58.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:32:58.906 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.906 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.906 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.906 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:32:58.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:32:58.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.907 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.907 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:32:58.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:32:58.907 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.907 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.907 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.907 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.907 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.907 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:32:58.907 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:58.907 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 353.49499966000002, 2023-12-06T18:32:58.907 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:32:58.908 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.908 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.908 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:58.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:32:58.908 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.908 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.908 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.908 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.908 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:32:58.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:32:58.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:32:58.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:32:58.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:32:58.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:32:58.909 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.909 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.909 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.909 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:32:58.909 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:32:58.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:32:58.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:32:58.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.910 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:32:58.910 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:32:58.910 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.910 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.910 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:32:58.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:32:58.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.911 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.911 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:32:58.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:32:58.911 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.911 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.911 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.911 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.911 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.911 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:32:58.911 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:58.911 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 353.35761257500002, 2023-12-06T18:32:58.911 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:32:58.912 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.912 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.912 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:58.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.912 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:32:58.912 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.912 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.912 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.912 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:32:58.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:32:58.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:32:58.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:32:58.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.913 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:32:58.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:32:58.913 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.913 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:32:58.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:32:58.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:32:58.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:32:58.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:32:58.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:32:58.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:32:58.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:32:58.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:32:58.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:32:58.915 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.915 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.915 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.915 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:32:58.915 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:58.915 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 353.35094184799999, 2023-12-06T18:32:58.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:32:58.916 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:58.916 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:32:58.916 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:58.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:58.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:58.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:58.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:32:58.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:58.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:58.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:32:58.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:32:58.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:58.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:32:58.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:32:58.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:58.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:32:58.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:32:58.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.917 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:58.917 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:32:58.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:32:58.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:58.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:32:58.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:32:58.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:32:58.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:32:58.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:32:58.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:32:58.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:32:58.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:32:58.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:58.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:58.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:58.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:32:58.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:32:58.919 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.919 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.919 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.919 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:58.919 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:58.919 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:58.920 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:32:58.921 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:58.917+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:59.021 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:59.018+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:59.092 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:59.093 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:59.093 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:59.093 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:59.102 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:32:59.145 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:59.142+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:59.223 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:32:59.219+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:32:59.223 INFO:teuthology.orchestra.run.smithi061.stderr:marked in osd.4. 2023-12-06T18:32:59.234 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:59.234 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:59.234 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:59.234 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:59.234 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:59.238 INFO:tasks.thrashosds.thrasher:Added osd 4 2023-12-06T18:32:59.244 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:32:59.324 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:59.320+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:59.413 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:59.413 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:59.413 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:59.414 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:32:59.414 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:59.422 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:32:59.425 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:32:59.421+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:32:59.534 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:59.531+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:59.549 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:59.549 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:59.549 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:32:59.549 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:59.558 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:32:59.629 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:32:59.623+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:32:59.727 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:59.724+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:32:59.731 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:59.732 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:32:59.732 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:32:59.732 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:32:59.732 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:59.740 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:32:59.828 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:32:59.825+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:32:59.849 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-12-06T18:32:59.913 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:32:59.913 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:32:59.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:32:59.913 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:32:59.913 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:59.913 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:32:59.913 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:32:59.913 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 158.249154885, 2023-12-06T18:32:59.913 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:32:59.913 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:59.913 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:32:59.913 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:32:59.914 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:32:59.914 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:59.914 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:32:59.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:59.914 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:59.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:59.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:59.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:32:59.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:59.914 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:59.914 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:59.914 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:59.914 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:32:59.914 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:59.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:59.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:59.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:59.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:32:59.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:32:59.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:59.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:59.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:59.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:32:59.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:32:59.915 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:59.915 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:59.915 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:59.915 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:32:59.915 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:32:59.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:59.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:59.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:59.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:32:59.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:32:59.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:59.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:59.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:59.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:32:59.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:32:59.916 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:59.916 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:59.916 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:59.916 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:32:59.916 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:32:59.917 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:59.917 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:59.917 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:59.917 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:59.917 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:59.917 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:12 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2023-12-06T18:32:59.917 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:32:59.917 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 38.099208259999997, 2023-12-06T18:32:59.917 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0032177460000000001, 2023-12-06T18:32:59.917 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:32:59.917 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:32:59.917 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:32:59.917 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:32:59.917 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:32:59.918 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 12 2023-12-06T18:32:59.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:59.918 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:32:59.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:59.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:32:59.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:32:59.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:59.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:59.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:59.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:32:59.918 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:32:59.918 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:32:59.918 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:59.918 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:59.918 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:32:59.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807030+0000", 2023-12-06T18:32:59.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6030000000000001e-05 2023-12-06T18:32:59.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:59.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:59.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:32:59.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807210+0000", 2023-12-06T18:32:59.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017936800000000001 2023-12-06T18:32:59.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:59.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:59.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:32:59.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807443+0000", 2023-12-06T18:32:59.919 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00023339699999999999 2023-12-06T18:32:59.919 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:59.919 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:59.919 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:32:59.919 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807614+0000", 2023-12-06T18:32:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000171032 2023-12-06T18:32:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:32:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.808696+0000", 2023-12-06T18:32:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010817190000000001 2023-12-06T18:32:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:32:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:32:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:32:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.810232+0000", 2023-12-06T18:32:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0015361999999999999 2023-12-06T18:32:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:59.920 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:32:59.921 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:32:59.921 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:32:59.922 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:32:59.963 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:32:59.960+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:00.031 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:00.027+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:00.124 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:00.125 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:00.125 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:00.125 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:00.131 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:00.128+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:00.137 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:33:00.195 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:33:00.233 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:00.229+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:00.320 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:00.320 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:00.320 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:00.320 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:00.320 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:00.331 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:33:00.334 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:00.330+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:00.435 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:00.431+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:00.436 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '2 of 30' 2023-12-06T18:33:00.508 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:00.508 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:00.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:00.508 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:33:00.508 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:00.518 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:33:00.565 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:00.561+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:00.597 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:33:00.597 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:33:00.638 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:00.634+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:00.711 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:33:00.711 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:33:00.714 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:00.714 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:00.724 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:33:00.808 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:33:00.809 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:33:00.810 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:00.806+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:00.837 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:00.834+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:00.912 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:33:00.913 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:33:00.917 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:00.917 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:00.926 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:33:00.938 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:00.935+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:01.026 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:33:01.027 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:33:01.039 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:01.036+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:01.108 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:01.108 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:01.115 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:33:01.116 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:33:01.117 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:33:01.144 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:01.141+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:01.203 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:33:01.203 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:33:01.243 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:01.240+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:01.259 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:01.259 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:01.269 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:33:01.289 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:33:01.290 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:33:01.342 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:01.339+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:01.426 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:01.426 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:01.436 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:33:01.442 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:01.439+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:01.451 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '3 of 30' 2023-12-06T18:33:01.485 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is set 2023-12-06T18:33:01.543 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:01.540+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:01.597 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:01.597 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:01.607 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:33:01.644 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:01.641+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:01.746 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:01.742+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:01.786 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:01.786 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:01.795 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:33:01.846 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:01.842+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:01.934 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:01.934 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:01.944 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:33:01.985 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:01.982+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:02.048 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:02.045+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:02.127 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:02.127 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:02.137 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:33:02.179 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:02.175+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:02.251 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:02.247+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:02.265 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:02.265 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:02.274 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:33:02.352 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:02.348+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:02.420 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '4 of 30' 2023-12-06T18:33:02.463 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:02.458+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:02.469 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:02.470 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:02.479 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:33:02.554 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:02.550+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:02.614 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:02.614 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:02.624 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:33:02.655 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:02.651+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:02.755 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:02.752+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:02.793 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:02.793 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:02.802 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:33:02.856 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:02.852+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:02.933 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:02.933 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:02.942 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:33:02.957 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:02.953+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:03.058 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:03.054+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:03.069 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:03.069 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:03.078 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:33:03.162 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:03.159+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:03.247 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:03.247 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:03.255 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:33:03.298 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:03.293+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:03.302 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:33:03.361 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:03.358+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:03.428 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:03.428 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:03.437 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:33:03.462 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:03.458+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:03.490 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '5 of 30' 2023-12-06T18:33:03.502 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-12-06T18:33:03.575 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:03.571+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:03.607 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:03.607 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:03.616 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:33:03.616 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:33:03.616 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:33:03.664 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:03.660+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:03.734 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:33:03.735 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:33:03.748 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:03.748 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:03.757 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:33:03.765 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:03.761+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:03.825 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:33:03.825 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:33:03.866 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:03.862+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:03.927 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:03.927 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:03.937 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:33:03.939 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:33:03.939 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:33:03.967 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:03.963+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:04.033 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:33:04.034 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:33:04.068 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:04.064+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:04.069 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:04.069 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:04.078 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:33:04.124 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:33:04.125 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:33:04.169 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:04.165+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:04.228 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:33:04.228 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:33:04.235 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:04.235 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:04.235 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:04.235 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:04.239 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 7, 5, 4] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:33:04.239 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:33:04.240 INFO:tasks.thrashosds.thrasher:primary_affinity 2023-12-06T18:33:04.240 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 1.000000 2023-12-06T18:33:04.240 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2023-12-06T18:33:04.245 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:33:04.287 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:04.282+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:04.312 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:33:04.312 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:33:04.370 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:04.366+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:04.429 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:04.429 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:04.429 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:04.429 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:04.430 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:04.439 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:33:04.470 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:04.467+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:04.571 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:04.568+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:04.575 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:04.576 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:04.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:04.576 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:33:04.576 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:04.585 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:33:04.669 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '6 of 30' 2023-12-06T18:33:04.672 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:04.669+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:04.779 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:04.774+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:04.798 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:04.798 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:04.799 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:04.799 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:04.802 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is unset 2023-12-06T18:33:04.811 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:33:04.874 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:04.870+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:04.975 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:04.971+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:05.006 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:05.007 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:05.007 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:05.007 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:05.007 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:05.016 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:33:05.075 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:05.072+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:05.176 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:05.173+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:05.253 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:05.254 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:05.254 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:05.254 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:05.254 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.254 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:33:05.254 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:05.254 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 369.84246142900002, 2023-12-06T18:33:05.254 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:33:05.254 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.254 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.254 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.255 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.255 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.255 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:33:05.255 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.255 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.255 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.255 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.255 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:05.255 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.255 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.255 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.255 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.255 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:05.255 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.255 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.255 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.256 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.256 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:33:05.256 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:33:05.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.256 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.256 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.256 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:33:05.256 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:33:05.256 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.256 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.256 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.256 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:33:05.256 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:33:05.257 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.257 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.257 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.257 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:33:05.257 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:33:05.257 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.257 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.257 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.257 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:33:05.257 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:33:05.257 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.257 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.257 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:33:05.257 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:33:05.258 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:33:05.258 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.258 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.258 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.258 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:33:05.258 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:33:05.258 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.258 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.258 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.258 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:33:05.258 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:33:05.258 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.258 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.258 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.259 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:33:05.259 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:33:05.259 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.259 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.259 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.259 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:33:05.259 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:33:05.259 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.259 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.259 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.259 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:33:05.259 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:33:05.259 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.259 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.259 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.260 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:33:05.260 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:33:05.260 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.260 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.260 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.260 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:33:05.260 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:33:05.260 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.260 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.260 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.260 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.260 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.260 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:05.260 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:05.261 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 367.73388819000002, 2023-12-06T18:33:05.261 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:33:05.261 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.261 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.261 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.261 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.261 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.261 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:33:05.261 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.261 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.261 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.261 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.261 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:05.261 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.262 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.262 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.262 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.262 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:05.262 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.262 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.262 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.262 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.262 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:33:05.262 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:33:05.262 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.262 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.262 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.262 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:33:05.262 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:33:05.263 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.263 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.263 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.263 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:33:05.263 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:33:05.263 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.263 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.263 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.263 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:33:05.263 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:33:05.263 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.263 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.263 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.263 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:33:05.263 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:33:05.264 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.264 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.264 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.264 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:33:05.264 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:33:05.264 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.264 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.264 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.264 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:33:05.264 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:33:05.264 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.264 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.264 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.264 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:33:05.265 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:33:05.265 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.265 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.265 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.265 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:33:05.265 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:33:05.265 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.265 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.265 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.265 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:33:05.265 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:33:05.265 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.265 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.266 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.266 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:33:05.266 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:33:05.266 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.266 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.266 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.266 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.266 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.266 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:05.266 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:05.266 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 367.73314295799997, 2023-12-06T18:33:05.266 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:33:05.266 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.266 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.266 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.267 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.267 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.267 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:33:05.267 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.267 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.267 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.267 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.267 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:05.267 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.267 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.267 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.267 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.267 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:05.268 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.268 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.268 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.268 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.268 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:33:05.268 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:33:05.268 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.268 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.268 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.268 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:33:05.268 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:33:05.268 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.268 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.268 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.269 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:33:05.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:33:05.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.269 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.269 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:33:05.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:33:05.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.269 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.269 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:33:05.269 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:33:05.269 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.269 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.269 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.270 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:33:05.270 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:33:05.270 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.270 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.270 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.270 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:33:05.270 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:33:05.270 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.270 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.270 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.270 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:33:05.270 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:33:05.270 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.270 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.271 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.271 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:33:05.271 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:33:05.271 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.271 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.271 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.271 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:33:05.271 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:33:05.271 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.271 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.271 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.271 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:33:05.271 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:33:05.271 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.271 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.272 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.272 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.272 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.272 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:05.272 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:05.272 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 367.73245920300002, 2023-12-06T18:33:05.272 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:33:05.272 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.272 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.272 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.272 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.272 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.272 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:33:05.272 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.272 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.273 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.273 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.273 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:05.273 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.273 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.273 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.273 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.273 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:05.273 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.273 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.273 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.273 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.273 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:33:05.273 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:33:05.273 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.274 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.274 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.274 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:33:05.274 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:33:05.274 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.274 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.274 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.274 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:33:05.274 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:33:05.274 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.274 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.274 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.274 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:33:05.274 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:33:05.275 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.275 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.275 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:33:05.275 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:33:05.275 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.275 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.275 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:33:05.275 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:33:05.275 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.275 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.275 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.275 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:33:05.275 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:33:05.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.276 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:33:05.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:33:05.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.276 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:33:05.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:33:05.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.276 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.276 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.276 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:33:05.276 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:33:05.276 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.277 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.277 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.277 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:33:05.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:33:05.277 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.277 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.277 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.277 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.277 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.277 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:05.277 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:05.277 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 367.72688594800002, 2023-12-06T18:33:05.277 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:33:05.277 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.277 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.278 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.278 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.278 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.278 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:33:05.278 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.278 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.278 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.278 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.278 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:05.278 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.278 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.278 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.278 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.278 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:05.278 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.279 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:33:05.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:33:05.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.279 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:33:05.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:33:05.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.279 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.279 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.279 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:33:05.279 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:33:05.279 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:33:05.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:33:05.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:33:05.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:33:05.280 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.280 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.280 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.280 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:33:05.280 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:33:05.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.281 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:33:05.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:33:05.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.281 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:33:05.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:33:05.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.281 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.281 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.281 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:33:05.281 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:33:05.281 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:33:05.282 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:33:05.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.282 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.282 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:33:05.282 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:33:05.282 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.282 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.282 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.282 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.282 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.283 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:05.283 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:05.283 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 367.72621230800002, 2023-12-06T18:33:05.283 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:33:05.283 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.283 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.283 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.283 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.283 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.283 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:33:05.283 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.283 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.283 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.283 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.283 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:05.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.284 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.284 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:05.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.284 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.284 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.284 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:33:05.284 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:33:05.284 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.285 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.285 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:33:05.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:33:05.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.285 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.285 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:33:05.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:33:05.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.285 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.285 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.285 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:33:05.285 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:33:05.285 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.286 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:33:05.286 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:33:05.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.286 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:33:05.286 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:33:05.286 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.286 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.286 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.286 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:33:05.286 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:33:05.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:33:05.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:33:05.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:33:05.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:33:05.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.287 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.287 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.287 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:33:05.287 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:33:05.287 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.288 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.288 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.288 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:33:05.288 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:33:05.288 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.288 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.288 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.288 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.288 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.288 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:05.288 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:05.288 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 367.72555010899998, 2023-12-06T18:33:05.288 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:33:05.288 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.288 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.289 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.289 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.289 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.289 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:33:05.289 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.289 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.289 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.289 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.289 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:05.289 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.289 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.289 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.289 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.289 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:05.289 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.289 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.290 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.290 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:33:05.290 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:33:05.290 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.290 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.290 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:33:05.290 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:33:05.290 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.290 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.290 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.290 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:33:05.290 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:33:05.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.291 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.291 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.291 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:33:05.291 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:33:05.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.291 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.291 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.291 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:33:05.291 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:33:05.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.291 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.291 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.291 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:33:05.291 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:33:05.291 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.292 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.292 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:33:05.292 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:33:05.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.292 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.292 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:33:05.292 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:33:05.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.292 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.292 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.292 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:33:05.292 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:33:05.292 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.293 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.293 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.293 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:33:05.293 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:33:05.293 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.293 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.293 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.293 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:33:05.293 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:33:05.293 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.293 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.293 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.293 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.293 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.293 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:05.294 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:05.294 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 367.652840674, 2023-12-06T18:33:05.294 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:33:05.294 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.294 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.294 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.294 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.294 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.294 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:33:05.294 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.294 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.294 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.294 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.295 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:05.295 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.295 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.295 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.295 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:05.295 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.295 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.295 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.295 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:33:05.295 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:33:05.295 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.295 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.295 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.296 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:33:05.296 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:33:05.296 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.296 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.296 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.296 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:33:05.296 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:33:05.296 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.296 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.296 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.296 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:33:05.296 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:33:05.296 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.296 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.297 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.297 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:33:05.297 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:33:05.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.297 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.297 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.297 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:33:05.297 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:33:05.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.297 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.297 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.297 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:33:05.297 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:33:05.297 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.298 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.298 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.298 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:33:05.298 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:33:05.298 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.298 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.298 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.298 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:33:05.298 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:33:05.298 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.298 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.298 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.298 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:33:05.299 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:33:05.299 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.299 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.299 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.299 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:33:05.299 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:33:05.299 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.299 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.299 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.299 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.299 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.299 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:05.299 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:05.299 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 367.65215896500001, 2023-12-06T18:33:05.300 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:33:05.300 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.300 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.300 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.300 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.300 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.300 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:33:05.300 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.300 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.300 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.300 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.300 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:05.300 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.300 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.301 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.301 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.301 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:05.301 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.301 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.301 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.301 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.301 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:33:05.301 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:33:05.301 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.301 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.301 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.301 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:33:05.301 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:33:05.301 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.302 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.302 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.302 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:33:05.302 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:33:05.302 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.302 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.302 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.302 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:33:05.302 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:33:05.302 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.302 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.302 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.302 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:33:05.302 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:33:05.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.303 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:33:05.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:33:05.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.303 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:33:05.303 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:33:05.303 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.303 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.303 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.303 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:33:05.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:33:05.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:33:05.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:33:05.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:33:05.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:33:05.304 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.304 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.304 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.304 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:33:05.304 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:33:05.305 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.305 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.305 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.305 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.305 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:05.305 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:05.305 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 366.57335487699999, 2023-12-06T18:33:05.305 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:33:05.305 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.305 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.305 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.305 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.305 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.305 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:33:05.305 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.306 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.306 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:05.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.306 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.306 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:05.306 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.306 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.306 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.306 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.306 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:33:05.307 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:33:05.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.307 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.307 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.307 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:33:05.307 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:33:05.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.307 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.307 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.307 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:33:05.307 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:33:05.307 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.307 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.307 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.307 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:33:05.308 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:33:05.308 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.308 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.308 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.308 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:33:05.308 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:33:05.308 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.308 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.308 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.308 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:33:05.308 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:33:05.308 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.308 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.308 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.308 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:33:05.308 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:33:05.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.309 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.309 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:33:05.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:33:05.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.309 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.309 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:33:05.309 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:33:05.309 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.309 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.309 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.309 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:33:05.310 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:33:05.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.310 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.310 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.310 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:33:05.310 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:33:05.310 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.310 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.310 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.310 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.310 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.310 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:05.310 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:05.311 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 366.57193830300002, 2023-12-06T18:33:05.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:33:05.311 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.311 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.311 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.311 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.311 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.311 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:33:05.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.311 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.311 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.311 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.311 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:05.311 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.311 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:05.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:33:05.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:33:05.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.312 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.312 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:33:05.312 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:33:05.312 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.312 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.313 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:33:05.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:33:05.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.313 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:33:05.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:33:05.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.313 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.313 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:33:05.313 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:33:05.313 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.313 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:33:05.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:33:05.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:33:05.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:33:05.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.314 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.314 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:33:05.314 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:33:05.314 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.314 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.315 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.315 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:33:05.315 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:33:05.315 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.315 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.315 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.315 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:33:05.315 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:33:05.315 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.315 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.315 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.315 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:33:05.315 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:33:05.315 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.315 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.315 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.316 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.316 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.316 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:05.316 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:05.316 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 366.57085317799999, 2023-12-06T18:33:05.316 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:33:05.316 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.316 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.316 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.316 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.316 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.316 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:33:05.316 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.316 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.316 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.317 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.317 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:05.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.317 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.317 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:05.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.317 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.317 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:33:05.317 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:33:05.317 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.317 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:33:05.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:33:05.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:33:05.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:33:05.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.318 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.318 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:33:05.318 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:33:05.318 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.318 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:33:05.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:33:05.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:33:05.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:33:05.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.319 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:33:05.319 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:33:05.319 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.319 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.319 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:33:05.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:33:05.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:33:05.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:33:05.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.320 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:33:05.320 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:33:05.320 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.320 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.320 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.321 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:33:05.321 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:33:05.321 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.321 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.321 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.321 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.321 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.321 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:05.321 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:05.321 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 366.535704076, 2023-12-06T18:33:05.321 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:33:05.321 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.321 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.321 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.322 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.322 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.322 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:33:05.322 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.322 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.322 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.322 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.322 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:05.322 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.322 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.322 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.322 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.322 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:05.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.323 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.323 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.323 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:33:05.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:33:05.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.323 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.323 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.323 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:33:05.323 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:33:05.323 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.323 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.323 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.323 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:33:05.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:33:05.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.324 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:33:05.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:33:05.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.324 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:33:05.324 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:33:05.324 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.324 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.324 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:33:05.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:33:05.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:33:05.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:33:05.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.325 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:33:05.325 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:33:05.325 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.325 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.325 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.326 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:33:05.326 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:33:05.326 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.326 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.326 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.326 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:33:05.326 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:33:05.326 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.326 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.326 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.326 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:33:05.326 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:33:05.326 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.326 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.327 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.327 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.327 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.327 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:05.327 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:05.327 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 359.98597355099997, 2023-12-06T18:33:05.327 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:33:05.327 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.327 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.327 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.327 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.327 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.327 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:33:05.327 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.328 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.328 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.328 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.328 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:05.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.328 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.328 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.328 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:05.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.328 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.328 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.328 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:33:05.328 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:33:05.328 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.329 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:33:05.329 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:33:05.329 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.329 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:33:05.329 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:33:05.329 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.329 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.329 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.329 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:33:05.329 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:33:05.329 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:33:05.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:33:05.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:33:05.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:33:05.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.330 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.330 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.330 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:33:05.330 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:33:05.330 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.331 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.331 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.331 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:33:05.331 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:33:05.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.331 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.331 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.331 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:33:05.331 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:33:05.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.331 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.331 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.331 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:33:05.331 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:33:05.331 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.332 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.332 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.332 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:33:05.332 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:33:05.332 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.332 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.332 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.332 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.332 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.332 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:05.332 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:05.332 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 359.98516794300002, 2023-12-06T18:33:05.332 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:33:05.332 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.333 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.333 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.333 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.333 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.333 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:33:05.333 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.333 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.333 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.333 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.333 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:05.333 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.333 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.333 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.333 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.333 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:05.334 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.334 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.334 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.334 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.334 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:33:05.334 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:33:05.334 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.334 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.334 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.334 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:33:05.334 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:33:05.334 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.334 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.334 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.334 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:33:05.335 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:33:05.335 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.335 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.335 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.335 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:33:05.335 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:33:05.335 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.335 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.335 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.335 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:33:05.335 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:33:05.335 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.335 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.335 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.335 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:33:05.335 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:33:05.335 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.336 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.336 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.336 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:33:05.336 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:33:05.336 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.336 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.336 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.336 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:33:05.336 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:33:05.336 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.336 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.336 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.336 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:33:05.336 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:33:05.337 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.337 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.337 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.337 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:33:05.337 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:33:05.337 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.337 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.337 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.337 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:33:05.337 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:33:05.337 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.337 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.337 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.337 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.337 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.337 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:05.338 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:05.338 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 359.98447713399997, 2023-12-06T18:33:05.338 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:33:05.338 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.338 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.338 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.338 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.338 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.338 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:33:05.338 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.338 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.338 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.338 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.338 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:05.338 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.339 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.339 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.339 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.339 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:05.339 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.339 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.339 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.339 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.339 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:33:05.339 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:33:05.339 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.339 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.339 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.339 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:33:05.339 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:33:05.340 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.340 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.340 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.340 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:33:05.340 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:33:05.340 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.340 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.340 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.340 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:33:05.340 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:33:05.340 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.340 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.340 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.340 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:33:05.340 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:33:05.341 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.341 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.341 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.341 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:33:05.341 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:33:05.341 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.341 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.341 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.341 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:33:05.341 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:33:05.341 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.341 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.341 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.341 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:33:05.341 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:33:05.342 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.342 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.342 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.342 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:33:05.342 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:33:05.342 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.342 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.342 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.342 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:33:05.342 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:33:05.342 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.342 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.342 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.342 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:33:05.343 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:33:05.343 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.343 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.343 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.343 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.343 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.343 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:05.343 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:05.343 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 359.98017962599999, 2023-12-06T18:33:05.343 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:33:05.343 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.343 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.343 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.343 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.343 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.344 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:33:05.344 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.344 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.344 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.344 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.344 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:05.344 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.344 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.344 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.344 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.344 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:05.344 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.345 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.345 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.345 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.345 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:33:05.345 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:33:05.345 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.345 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.345 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.345 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:33:05.345 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:33:05.345 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.345 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.345 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.345 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:33:05.345 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:33:05.346 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.346 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.346 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.346 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:33:05.346 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:33:05.346 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.346 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.346 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.346 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:33:05.346 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:33:05.346 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.346 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.346 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.347 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:33:05.347 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:33:05.347 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.347 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.347 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.347 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:33:05.347 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:33:05.347 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.347 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.347 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.347 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:33:05.347 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:33:05.347 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.347 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.348 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.348 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:33:05.348 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:33:05.348 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.348 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.348 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.348 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:33:05.348 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:33:05.348 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.348 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.348 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.348 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:33:05.348 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:33:05.348 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.349 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.349 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.349 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.349 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.349 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:05.349 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:05.349 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 359.97951195100001, 2023-12-06T18:33:05.349 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:33:05.349 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.349 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.349 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.349 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.349 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.349 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:33:05.350 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.350 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.350 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.350 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.350 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:05.350 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.350 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.350 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.350 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.350 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:05.350 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.350 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.350 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.350 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.350 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:33:05.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:33:05.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.351 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:33:05.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:33:05.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.351 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:33:05.351 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:33:05.351 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.351 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.351 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.351 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:33:05.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:33:05.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:33:05.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:33:05.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:33:05.352 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:33:05.352 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.352 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.352 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.352 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:33:05.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:33:05.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:33:05.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:33:05.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.353 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:33:05.353 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:33:05.353 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.353 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.353 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.354 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:33:05.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:33:05.354 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.354 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.354 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.354 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:33:05.354 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:33:05.354 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.354 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.354 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.354 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.354 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.354 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:05.354 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:05.355 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 359.97885652299999, 2023-12-06T18:33:05.355 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:33:05.355 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.355 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.355 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.355 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.355 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.355 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:33:05.355 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.355 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.355 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.355 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.355 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:05.355 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.355 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.355 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.356 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.356 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:05.356 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.356 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.356 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.356 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.356 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:33:05.356 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:33:05.356 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.356 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.356 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.356 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:33:05.356 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:33:05.356 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.357 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:33:05.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:33:05.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.357 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:33:05.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:33:05.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.357 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.357 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.357 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:33:05.357 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:33:05.357 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.358 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.358 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.358 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:33:05.358 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:33:05.358 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.358 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.358 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.358 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:33:05.358 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:33:05.358 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.358 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.358 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.358 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:33:05.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:33:05.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.359 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.359 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.359 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:33:05.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:33:05.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.359 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.359 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.359 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:33:05.359 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:33:05.359 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.359 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.359 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.360 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:33:05.360 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:33:05.360 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.360 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.360 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.360 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.360 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.360 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:05.360 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:05.360 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 359.93820197600002, 2023-12-06T18:33:05.360 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:33:05.360 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.360 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:05.360 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:05.360 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:33:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.361 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:33:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:33:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:33:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:33:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.362 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:33:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:33:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:33:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:33:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:33:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:33:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.363 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:33:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:33:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:33:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:33:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:33:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:33:05.364 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:33:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:33:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:33:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:33:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:33:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:33:05.365 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.366 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.366 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.366 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.366 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.366 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:05.367 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:33:05.368 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:05.364+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:05.469 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:05.465+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:05.506 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:05.506 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:05.507 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:05.507 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:05.516 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:33:05.569 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:05.566+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:05.670 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:05.666+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:05.695 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:05.695 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:05.695 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:05.695 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:05.695 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:05.704 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:33:05.734 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '7 of 30' 2023-12-06T18:33:05.771 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:05.768+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:05.818 INFO:teuthology.orchestra.run.smithi061.stderr:set osd.2 primary-affinity to 1 (8655362) 2023-12-06T18:33:05.872 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:05.869+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:05.925 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:05.926 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:05.926 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:05.926 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:05.926 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.926 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:33:05.926 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:05.926 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 344.10974127700001, 2023-12-06T18:33:05.926 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:33:05.926 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:05.926 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:33:05.926 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:05.927 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.927 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:05.927 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:05.927 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.927 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.927 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.927 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:05.927 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:05.927 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:05.927 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.927 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.927 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:05.927 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:33:05.927 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:33:05.927 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.928 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.928 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:05.928 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:33:05.928 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:33:05.928 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.928 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.928 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:05.928 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:33:05.928 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:33:05.928 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.928 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.928 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:05.928 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:33:05.928 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:33:05.928 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:33:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:33:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:33:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:33:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:33:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:33:05.929 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.930 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.930 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.930 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:05.930 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:05.930 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:05.935 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:33:05.979 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:05.975+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:06.074 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:06.074 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:06.074 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:06.074 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:06.075 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:06.071+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:06.084 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:33:06.175 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:06.172+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:06.267 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:06.267 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:06.267 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:06.267 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:06.267 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:06.276 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:06.272+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:06.277 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:33:06.326 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:33:06.377 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:06.374+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:06.464 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:06.464 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:06.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:06.465 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:06.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.465 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:33:06.465 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:06.465 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 225.361093399, 2023-12-06T18:33:06.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:33:06.465 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:06.465 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:06.465 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:06.465 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:06.465 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:06.465 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:33:06.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.466 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:06.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:06.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:06.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:06.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:06.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:06.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:06.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:06.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:33:06.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:33:06.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:06.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:33:06.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:33:06.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:06.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:33:06.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:33:06.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:06.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:33:06.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:33:06.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:06.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:33:06.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:33:06.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:06.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:33:06.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:33:06.468 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:06.468 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:06.468 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:06.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.469 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:33:06.469 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:06.469 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 165.342836097, 2023-12-06T18:33:06.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:33:06.469 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:06.469 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:06.469 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:06.469 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:06.469 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:06.469 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:33:06.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.469 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:06.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:06.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:06.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:06.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:06.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:06.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:06.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:06.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:33:06.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:33:06.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:06.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:33:06.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:33:06.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:06.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:33:06.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:33:06.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:06.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:33:06.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:33:06.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:06.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:33:06.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:33:06.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:06.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:33:06.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:33:06.472 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:06.472 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:06.472 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:06.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.472 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:11 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2023-12-06T18:33:06.472 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:06.472 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 45.31841498, 2023-12-06T18:33:06.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0086075449999999994, 2023-12-06T18:33:06.472 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:06.472 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:06.472 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:06.473 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:06.473 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:06.473 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 11 2023-12-06T18:33:06.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.473 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:06.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:06.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:06.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:06.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:06.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:06.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:06.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:06.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135441+0000", 2023-12-06T18:33:06.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.8695999999999999e-05 2023-12-06T18:33:06.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:06.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135650+0000", 2023-12-06T18:33:06.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020883699999999999 2023-12-06T18:33:06.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:06.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135863+0000", 2023-12-06T18:33:06.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000213931 2023-12-06T18:33:06.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:06.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.136054+0000", 2023-12-06T18:33:06.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000190416 2023-12-06T18:33:06.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:06.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.142056+0000", 2023-12-06T18:33:06.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0060019330000000001 2023-12-06T18:33:06.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:06.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:06.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:06.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.144030+0000", 2023-12-06T18:33:06.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0019737320000000002 2023-12-06T18:33:06.475 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:06.475 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:06.475 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:06.475 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:06.475 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:06.475 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:06.476 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:33:06.478 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:06.474+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:06.578 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:06.575+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:06.607 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:06.608 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:06.608 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:06.608 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:06.614 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:33:06.614 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:33:06.616 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:33:06.700 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:06.697+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:06.709 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:33:06.709 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:33:06.755 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '8 of 30' 2023-12-06T18:33:06.789 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:06.789 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:06.789 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:06.789 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:06.789 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:06.796 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:33:06.799 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:06.795+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:06.804 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:33:06.804 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:33:06.822 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-12-06T18:33:06.881 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:06.878+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:06.899 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:33:06.900 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:33:06.983 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:06.979+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:07.001 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:33:07.001 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:33:07.002 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:07.002 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:07.012 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:33:07.093 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:33:07.093 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:33:07.097 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:07.093+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:07.182 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:33:07.182 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:33:07.184 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:07.180+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:07.185 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:07.185 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:07.195 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:33:07.271 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:33:07.271 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:33:07.285 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:07.281+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:07.382 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:07.382 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:07.386 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:07.382+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:07.391 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:33:07.487 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:07.483+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:07.517 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:07.518 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:07.526 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:33:07.588 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:07.585+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:07.654 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:07.654 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:07.663 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:33:07.689 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:07.685+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:07.790 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:07.786+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:07.797 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '9 of 30' 2023-12-06T18:33:07.815 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:07.815 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:07.825 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:33:07.909 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:07.905+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:07.992 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:07.988+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:07.993 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:07.993 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:08.002 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:33:08.092 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:08.089+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:08.171 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:08.171 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:08.181 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:33:08.225 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:08.221+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:08.291 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is unset 2023-12-06T18:33:08.294 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:08.290+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:08.308 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:08.308 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:08.317 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:33:08.394 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:08.391+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:08.486 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:08.486 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:08.495 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:33:08.496 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:08.492+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:08.597 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:08.593+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:08.635 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:08.635 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:08.641 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:33:08.726 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:08.722+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:08.737 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '10 of 30' 2023-12-06T18:33:08.822 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:08.818+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:08.834 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:08.834 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:08.844 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:33:08.899 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:08.896+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:09.001 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:08.997+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:09.021 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:09.022 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:09.031 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:33:09.102 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:09.098+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:09.203 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:09.199+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:09.210 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:09.210 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:09.221 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:33:09.288 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:33:09.305 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:09.301+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:09.401 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:09.401 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:09.405 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:09.401+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:09.410 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:33:09.505 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:09.502+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:09.566 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:09.567 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:09.577 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:33:09.589 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:33:09.589 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:33:09.618 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:09.615+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:09.682 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:33:09.683 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:33:09.707 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:09.704+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:09.731 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:09.731 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:09.740 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:33:09.794 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '11 of 30' 2023-12-06T18:33:09.794 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:33:09.795 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:33:09.808 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:09.805+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:09.909 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:09.905+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:09.915 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:33:09.916 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:33:09.959 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:09.959 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:09.959 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:09.959 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:09.959 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:09.971 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:33:10.010 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:10.007+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:10.022 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:33:10.023 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:33:10.111 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:10.108+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:10.115 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:33:10.115 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:33:10.116 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:10.116 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:10.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:10.116 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:10.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.116 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:33:10.117 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:10.117 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 348.300633839, 2023-12-06T18:33:10.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:33:10.117 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:10.117 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:33:10.117 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:10.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:10.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:10.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:10.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:10.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:10.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:10.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:10.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:33:10.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:33:10.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:10.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:33:10.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:33:10.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:10.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:33:10.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:33:10.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:10.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:33:10.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:33:10.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:10.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:33:10.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:33:10.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:10.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:33:10.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:33:10.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:10.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:33:10.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:33:10.120 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:10.120 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:10.120 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:10.120 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:10.120 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:10.120 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:10.126 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:33:10.207 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:33:10.208 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:33:10.212 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:10.208+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:10.292 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:33:10.292 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:33:10.306 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-12-06T18:33:10.312 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:10.312 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:10.313 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:10.313 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:10.313 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:10.310+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:10.323 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:33:10.415 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:10.411+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:10.505 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:10.505 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:10.505 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:10.505 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:10.505 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:10.514 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:33:10.556 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:10.552+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:10.616 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:10.613+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:10.700 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:10.700 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:10.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:10.700 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:10.700 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.700 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:33:10.700 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:10.700 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 229.59828981499999, 2023-12-06T18:33:10.700 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:33:10.700 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:10.700 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:10.701 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:10.701 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:10.701 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:10.701 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:33:10.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.701 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:10.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:10.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:10.701 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:10.701 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.701 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.701 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:10.701 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:10.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:10.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:10.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:33:10.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:33:10.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:10.702 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:33:10.702 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:33:10.702 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.702 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.702 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:10.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:33:10.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:33:10.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:10.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:33:10.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:33:10.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:10.703 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:33:10.703 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:33:10.703 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.703 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.703 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:10.704 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:33:10.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:33:10.704 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:10.704 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:10.704 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:10.704 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.704 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.704 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:33:10.704 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:10.704 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 169.58003251299999, 2023-12-06T18:33:10.704 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:33:10.704 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:10.704 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:10.704 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:10.704 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:10.705 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:10.705 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:33:10.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.705 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:10.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.705 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:10.705 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:10.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:10.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.705 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.705 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:10.705 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:10.705 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:10.705 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:10.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:33:10.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:33:10.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:10.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:33:10.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:33:10.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.706 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.706 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:10.706 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:33:10.706 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:33:10.706 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:10.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:33:10.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:33:10.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:10.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:33:10.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:33:10.707 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.707 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.707 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:10.707 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:33:10.707 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:33:10.708 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:10.708 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:10.708 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:10.708 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.708 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.708 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:11 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2023-12-06T18:33:10.708 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:10.708 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 49.555611396000003, 2023-12-06T18:33:10.708 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0086075449999999994, 2023-12-06T18:33:10.708 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:10.708 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:10.708 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:10.708 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:10.708 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:10.708 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 11 2023-12-06T18:33:10.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.709 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:10.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:10.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:10.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:10.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:10.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:10.709 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:10.709 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.709 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.709 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:10.709 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135441+0000", 2023-12-06T18:33:10.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.8695999999999999e-05 2023-12-06T18:33:10.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.710 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:10.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135650+0000", 2023-12-06T18:33:10.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020883699999999999 2023-12-06T18:33:10.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.710 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:10.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135863+0000", 2023-12-06T18:33:10.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000213931 2023-12-06T18:33:10.710 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.710 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.710 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:10.710 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.136054+0000", 2023-12-06T18:33:10.710 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000190416 2023-12-06T18:33:10.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:10.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.142056+0000", 2023-12-06T18:33:10.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0060019330000000001 2023-12-06T18:33:10.711 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:10.711 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:10.711 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:10.711 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.144030+0000", 2023-12-06T18:33:10.711 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0019737320000000002 2023-12-06T18:33:10.711 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:10.711 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:10.711 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:10.711 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:10.711 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:10.711 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:10.712 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:33:10.717 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:10.714+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:10.818 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:10.815+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:10.837 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 7, 5, 4] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:33:10.838 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:33:10.838 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2023-12-06T18:33:10.838 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2023-12-06T18:33:10.838 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2023-12-06T18:33:10.838 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2023-12-06T18:33:10.846 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:10.846 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:10.846 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:10.846 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:10.856 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:33:10.941 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:10.937+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:10.979 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:10.979 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "" 2023-12-06T18:33:10.979 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:11.020 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:11.017+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:11.064 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:11.065 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:11.065 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:11.065 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:11.065 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:11.065 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:33:11.120 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:11.117+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:11.222 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:11.218+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:11.238 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:33:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:33:11.238 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:33:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 289.74484618000002, 2023-12-06T18:33:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:33:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:33:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.239 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:33:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:33:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:33:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:33:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.240 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:33:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:33:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:33:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:33:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:33:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:33:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:33:11.241 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:33:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:33:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:33:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.242 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.242 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:11.248 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:33:11.322 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:11.319+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:11.376 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:11.376 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:11.376 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:11.376 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:11.386 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:33:11.427 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:11.424+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:11.524 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:11.520+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:11.554 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:11.554 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:11.554 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:11.554 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:11.555 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:11.564 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:33:11.625 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:11.621+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:11.726 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:11.723+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:11.777 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:33:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.777 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:33:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:33:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 375.160547769, 2023-12-06T18:33:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:33:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:33:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:33:11.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:33:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:33:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:33:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:33:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:33:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:33:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:33:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:33:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:33:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:33:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:33:11.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:33:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:33:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:33:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:33:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:33:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:33:11.781 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:33:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 374.13772075999998, 2023-12-06T18:33:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:33:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:33:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:33:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:33:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:33:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:33:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:33:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:33:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:33:11.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:33:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:33:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:33:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:33:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:33:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:33:11.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:33:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:33:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:33:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:33:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:33:11.785 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:33:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 374.12567412200002, 2023-12-06T18:33:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:33:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:33:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:33:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:33:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:33:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:33:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:33:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:33:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:33:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:33:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:33:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:33:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:33:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:33:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:33:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:33:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:33:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:33:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:33:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:33:11.789 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:33:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 374.10427351499999, 2023-12-06T18:33:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:33:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:33:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:33:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:33:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:33:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:33:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:33:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:33:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:33:11.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:33:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:33:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:33:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:33:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:33:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:33:11.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:33:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:33:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:33:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:33:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.793 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:33:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:33:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 374.10309419200001, 2023-12-06T18:33:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:33:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:33:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:33:11.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:33:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:33:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:33:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:33:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:33:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:33:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:33:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:33:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:33:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:33:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:33:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:33:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:33:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:33:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:33:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:33:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:33:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:33:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 374.10252221100001, 2023-12-06T18:33:11.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:33:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:33:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:33:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:33:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:33:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:33:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:33:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:33:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:33:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:33:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:33:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:33:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:33:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:33:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:33:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:33:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:33:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:33:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:33:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:33:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:33:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 373.65668674099999, 2023-12-06T18:33:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:33:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.801 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:33:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:33:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:33:11.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:33:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:33:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:33:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:33:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:33:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:33:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:33:11.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:33:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:33:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:33:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:33:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:33:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:33:11.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:33:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:33:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:33:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:33:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 373.655086885, 2023-12-06T18:33:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:33:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.805 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:33:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:33:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:33:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:33:11.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:33:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:33:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:33:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:33:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:33:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:33:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:33:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:33:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:33:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:33:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:33:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:33:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:33:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:33:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:33:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:33:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 373.27588126900002, 2023-12-06T18:33:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:33:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:33:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:33:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:33:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:33:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.810 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:33:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:33:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:33:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:33:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:33:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:33:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.811 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:33:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:33:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:33:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:33:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:33:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:33:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.812 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:33:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:33:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:33:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:33:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 373.03837908700001, 2023-12-06T18:33:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:33:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.813 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:33:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:33:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:33:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:33:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.814 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:33:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:33:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:33:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:33:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:33:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:33:11.815 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:33:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:33:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:33:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:33:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:33:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:33:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.816 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:33:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:33:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:33:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:33:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 373.03765108800002, 2023-12-06T18:33:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:33:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.817 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:33:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:33:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:33:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:33:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.818 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:33:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:33:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:33:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:33:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:33:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:33:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.819 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:33:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:33:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:33:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:33:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:33:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:33:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.820 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:33:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:33:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:33:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:33:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 373.03263867800001, 2023-12-06T18:33:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:33:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.821 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:33:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:33:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.822 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:33:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:33:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:33:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:33:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:33:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:33:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.823 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:33:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:33:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:33:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:33:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:33:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:33:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:33:11.824 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:33:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:33:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:33:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:33:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:33:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 373.02711159900002, 2023-12-06T18:33:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:33:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.825 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:33:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:33:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:33:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:33:11.826 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:33:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:33:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:33:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:33:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:33:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:33:11.827 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:33:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:33:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:33:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:33:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:33:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:33:11.828 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:33:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:33:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:33:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:33:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 373.02660076199999, 2023-12-06T18:33:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:33:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.829 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:33:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:33:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:33:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:33:11.830 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:33:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:33:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:33:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:33:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:33:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:33:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.831 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:33:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:33:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:33:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:33:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:33:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:33:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.832 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:33:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:33:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:33:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:33:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 373.02614150099998, 2023-12-06T18:33:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:33:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.833 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:33:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:33:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:33:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:33:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.834 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:33:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:33:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:33:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:33:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:33:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:33:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.835 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:33:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:33:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:33:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:33:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:33:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:33:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.836 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:33:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:33:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:33:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:33:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 366.44622569400002, 2023-12-06T18:33:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:33:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:33:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:33:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:33:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:33:11.838 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:33:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:33:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:33:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:33:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:33:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:33:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.839 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:33:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:33:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:33:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:33:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:33:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:33:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.840 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:33:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:33:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:33:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:33:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 366.43998788900001, 2023-12-06T18:33:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:33:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.841 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:33:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:33:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:33:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:33:11.842 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:33:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:33:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:33:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:33:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:33:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:33:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.843 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:33:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:33:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:33:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:33:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:33:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:33:11.844 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:33:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:33:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:33:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:33:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 366.43392481000001, 2023-12-06T18:33:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:33:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.845 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:33:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:33:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:33:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:33:11.846 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:33:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:33:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:33:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:33:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:33:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:33:11.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:33:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:33:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:33:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:33:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:33:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:33:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:33:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:33:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:33:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:33:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 366.29653772500001, 2023-12-06T18:33:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:33:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:33:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:33:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:33:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:33:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:33:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:33:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:33:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:33:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:33:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:33:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.851 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:33:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:33:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:33:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:33:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:33:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:33:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:33:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:33:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:33:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:33:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 366.28986699799998, 2023-12-06T18:33:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:33:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:11.853 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:33:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:33:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:33:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:33:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.854 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:33:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:33:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:33:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:33:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:33:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:33:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:11.855 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:33:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:33:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:33:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:33:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:33:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:33:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:33:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:33:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.856 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.857 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.857 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:11.857 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:11.857 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:11.859 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '12 of 30' 2023-12-06T18:33:11.859 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:33:11.860 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:11.855+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:11.957 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is set 2023-12-06T18:33:11.959 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:11.955+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:12.031 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:12.031 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:12.031 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:12.031 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:12.041 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:33:12.059 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:12.056+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:12.160 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:12.157+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:12.168 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:12.168 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:12.169 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:12.169 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:12.177 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:33:12.261 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:12.258+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:12.306 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:12.306 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:12.306 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:12.306 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:33:12.306 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:12.309 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:33:12.316 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:33:12.399 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:12.396+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:12.462 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:12.459+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:12.498 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:12.498 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:12.498 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:12.498 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:12.507 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:33:12.564 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:12.560+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:12.570 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:33:12.570 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:33:12.663 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:33:12.663 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:33:12.665 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:12.661+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:12.680 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:12.680 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:12.680 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:12.681 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:12.681 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:12.690 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:33:12.755 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:33:12.755 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:33:12.775 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:12.771+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:12.801 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '13 of 30' 2023-12-06T18:33:12.867 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:12.864+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:12.872 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:12.872 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:12.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:12.873 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:33:12.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:12.873 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:33:12.873 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:12.873 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 171.20910298699999, 2023-12-06T18:33:12.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:33:12.873 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:12.873 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:12.873 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:12.873 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:12.873 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:12.873 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:33:12.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:12.873 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:12.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:12.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:12.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:12.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:12.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:12.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:12.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:12.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:12.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:12.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:12.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:12.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:12.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:33:12.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:33:12.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:12.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:12.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:12.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:33:12.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:33:12.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:12.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:12.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:12.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:33:12.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:33:12.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:12.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:12.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:12.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:33:12.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:33:12.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:12.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:12.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:12.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:33:12.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:33:12.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:12.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:12.876 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:12.876 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:33:12.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:33:12.876 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:12.876 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:12.876 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:12.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:12.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:12.876 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:12 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2023-12-06T18:33:12.876 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:12.876 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 51.059156362000003, 2023-12-06T18:33:12.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0032177460000000001, 2023-12-06T18:33:12.877 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:12.877 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:12.877 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:12.877 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:12.877 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:12.877 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 12 2023-12-06T18:33:12.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:12.877 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:12.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:12.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:12.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:12.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:12.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:12.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:12.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:12.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:12.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:12.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:12.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:12.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:12.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807030+0000", 2023-12-06T18:33:12.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6030000000000001e-05 2023-12-06T18:33:12.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:12.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:12.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:12.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807210+0000", 2023-12-06T18:33:12.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017936800000000001 2023-12-06T18:33:12.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:12.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:12.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:12.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807443+0000", 2023-12-06T18:33:12.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00023339699999999999 2023-12-06T18:33:12.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:12.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:12.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:12.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807614+0000", 2023-12-06T18:33:12.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000171032 2023-12-06T18:33:12.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:12.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:12.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:12.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.808696+0000", 2023-12-06T18:33:12.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010817190000000001 2023-12-06T18:33:12.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:12.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:12.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:12.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.810232+0000", 2023-12-06T18:33:12.880 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0015361999999999999 2023-12-06T18:33:12.880 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:12.880 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:12.880 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:12.880 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:12.880 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:12.880 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:12.880 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:33:12.880 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:33:12.883 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:33:12.968 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:12.964+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:12.981 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:33:12.981 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:33:13.024 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:13.024 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:13.033 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:33:13.068 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:33:13.068 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:33:13.069 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:13.066+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:13.158 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:33:13.159 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:33:13.169 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:13.166+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:13.205 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:13.205 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:13.215 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:33:13.244 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:33:13.245 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:33:13.302 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:13.298+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:13.371 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:13.367+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:13.383 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:13.383 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:13.393 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:33:13.472 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:13.468+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:13.570 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:13.570 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:13.572 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:13.568+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:13.579 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:33:13.673 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:13.670+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:13.756 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:13.756 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:13.765 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:33:13.774 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:13.771+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:13.857 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '14 of 30' 2023-12-06T18:33:13.875 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:13.872+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:13.906 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:13.906 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:13.918 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:33:13.973 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-12-06T18:33:13.976 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:13.972+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:14.060 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:14.060 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:14.069 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:33:14.076 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:14.073+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:14.177 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:14.173+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:14.209 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:14.209 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:14.220 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:33:14.278 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:14.275+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:14.359 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:14.359 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:14.369 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:33:14.379 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:14.376+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:14.479 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:14.476+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:14.551 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:14.551 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:14.561 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:33:14.580 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:14.576+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:14.684 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:14.681+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:14.703 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:14.703 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:14.712 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:33:14.782 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:14.778+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:14.872 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '15 of 30' 2023-12-06T18:33:14.878 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is set 2023-12-06T18:33:14.882 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:14.879+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:14.900 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:14.900 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:14.910 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:33:14.983 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:14.979+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:15.041 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:15.041 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:15.050 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:33:15.091 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:15.088+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:15.184 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:15.181+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:15.219 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:15.219 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:15.228 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:33:15.260 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:33:15.305 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:15.301+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:15.355 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:15.355 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:15.364 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:33:15.387 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:15.383+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:15.493 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:15.493 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:15.501 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:33:15.513 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:33:15.513 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:33:15.513 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:15.509+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:15.588 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:15.584+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:15.607 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:33:15.607 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:33:15.628 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:15.629 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:15.638 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:33:15.689 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:15.686+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:15.697 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:33:15.697 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:33:15.766 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:15.766 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:15.775 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:33:15.790 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:33:15.790 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:33:15.818 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:15.815+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:15.832 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '16 of 30' 2023-12-06T18:33:15.881 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:33:15.882 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:33:15.890 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:15.886+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:15.949 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:15.949 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:15.958 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:33:15.973 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:33:15.974 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:33:15.989 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 7, 5, 4] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:33:15.989 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:33:15.989 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2023-12-06T18:33:15.989 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2023-12-06T18:33:16.005 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:16.001+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:16.072 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:33:16.073 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:33:16.092 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:16.089+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:16.156 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:16.156 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:16.156 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:16.156 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:16.156 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:16.169 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:33:16.190 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:33:16.190 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:33:16.193 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:16.189+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:16.294 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:16.290+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:16.352 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:16.352 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:16.352 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:16.352 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:33:16.352 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:16.362 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:33:16.395 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:16.391+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:16.495 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:16.495 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:16.495 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:16.495 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:16.496 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:16.492+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:16.504 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:33:16.596 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:16.593+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:16.657 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:16.657 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:16.657 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:16.657 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:16.657 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:16.666 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:33:16.707 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:16.704+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:16.748 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:33:16.748 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:16.749 INFO:teuthology.orchestra.run.smithi061.stdout: "epoch": 107, 2023-12-06T18:33:16.749 INFO:teuthology.orchestra.run.smithi061.stdout: "fsid": "497e3283-cc16-40ff-a72c-a7487474fb4b", 2023-12-06T18:33:16.749 INFO:teuthology.orchestra.run.smithi061.stdout: "created": "2023-12-06T18:26:07.122862+0000", 2023-12-06T18:33:16.749 INFO:teuthology.orchestra.run.smithi061.stdout: "modified": "2023-12-06T18:33:14.786065+0000", 2023-12-06T18:33:16.749 INFO:teuthology.orchestra.run.smithi061.stdout: "last_up_change": "2023-12-06T18:30:04.956811+0000", 2023-12-06T18:33:16.749 INFO:teuthology.orchestra.run.smithi061.stdout: "last_in_change": "2023-12-06T18:32:58.826891+0000", 2023-12-06T18:33:16.749 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-06T18:33:16.749 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_num": 5806080, 2023-12-06T18:33:16.749 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_set": [ 2023-12-06T18:33:16.749 INFO:teuthology.orchestra.run.smithi061.stdout: "nodeep-scrub", 2023-12-06T18:33:16.749 INFO:teuthology.orchestra.run.smithi061.stdout: "noscrub", 2023-12-06T18:33:16.749 INFO:teuthology.orchestra.run.smithi061.stdout: "pglog_hardlimit", 2023-12-06T18:33:16.749 INFO:teuthology.orchestra.run.smithi061.stdout: "purged_snapdirs", 2023-12-06T18:33:16.749 INFO:teuthology.orchestra.run.smithi061.stdout: "recovery_deletes", 2023-12-06T18:33:16.749 INFO:teuthology.orchestra.run.smithi061.stdout: "sortbitwise" 2023-12-06T18:33:16.750 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:33:16.750 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_version": 8, 2023-12-06T18:33:16.750 INFO:teuthology.orchestra.run.smithi061.stdout: "full_ratio": 0.94999998807907104, 2023-12-06T18:33:16.750 INFO:teuthology.orchestra.run.smithi061.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-12-06T18:33:16.750 INFO:teuthology.orchestra.run.smithi061.stdout: "nearfull_ratio": 0.85000002384185791, 2023-12-06T18:33:16.750 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_snapshot": "", 2023-12-06T18:33:16.750 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_max": 2, 2023-12-06T18:33:16.750 INFO:teuthology.orchestra.run.smithi061.stdout: "max_osd": 8, 2023-12-06T18:33:16.750 INFO:teuthology.orchestra.run.smithi061.stdout: "require_min_compat_client": "luminous", 2023-12-06T18:33:16.750 INFO:teuthology.orchestra.run.smithi061.stdout: "min_compat_client": "luminous", 2023-12-06T18:33:16.750 INFO:teuthology.orchestra.run.smithi061.stdout: "require_osd_release": "reef", 2023-12-06T18:33:16.750 INFO:teuthology.orchestra.run.smithi061.stdout: "allow_crimson": false, 2023-12-06T18:33:16.750 INFO:teuthology.orchestra.run.smithi061.stdout: "pools": [ 2023-12-06T18:33:16.751 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.751 INFO:teuthology.orchestra.run.smithi061.stdout: "pool": 1, 2023-12-06T18:33:16.751 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_name": ".mgr", 2023-12-06T18:33:16.751 INFO:teuthology.orchestra.run.smithi061.stdout: "create_time": "2023-12-06T18:26:52.648377+0000", 2023-12-06T18:33:16.751 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-12-06T18:33:16.751 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_names": "hashpspool", 2023-12-06T18:33:16.751 INFO:teuthology.orchestra.run.smithi061.stdout: "type": 1, 2023-12-06T18:33:16.751 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 2, 2023-12-06T18:33:16.751 INFO:teuthology.orchestra.run.smithi061.stdout: "min_size": 1, 2023-12-06T18:33:16.751 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_rule": 0, 2023-12-06T18:33:16.751 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_count": 0, 2023-12-06T18:33:16.751 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_target": 0, 2023-12-06T18:33:16.751 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_barrier": 0, 2023-12-06T18:33:16.751 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-06T18:33:16.751 INFO:teuthology.orchestra.run.smithi061.stdout: "object_hash": 2, 2023-12-06T18:33:16.752 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscale_mode": "off", 2023-12-06T18:33:16.752 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num": 1, 2023-12-06T18:33:16.752 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num": 1, 2023-12-06T18:33:16.752 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num_target": 1, 2023-12-06T18:33:16.752 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_target": 1, 2023-12-06T18:33:16.752 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_pending": 1, 2023-12-06T18:33:16.752 INFO:teuthology.orchestra.run.smithi061.stdout: "last_pg_merge_meta": { 2023-12-06T18:33:16.752 INFO:teuthology.orchestra.run.smithi061.stdout: "source_pgid": "0.0", 2023-12-06T18:33:16.752 INFO:teuthology.orchestra.run.smithi061.stdout: "ready_epoch": 0, 2023-12-06T18:33:16.752 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_started": 0, 2023-12-06T18:33:16.752 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_clean": 0, 2023-12-06T18:33:16.752 INFO:teuthology.orchestra.run.smithi061.stdout: "source_version": "0'0", 2023-12-06T18:33:16.752 INFO:teuthology.orchestra.run.smithi061.stdout: "target_version": "0'0" 2023-12-06T18:33:16.752 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.753 INFO:teuthology.orchestra.run.smithi061.stdout: "last_change": "19", 2023-12-06T18:33:16.753 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend": "0", 2023-12-06T18:33:16.753 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-06T18:33:16.753 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_preluminous": "0", 2023-12-06T18:33:16.753 INFO:teuthology.orchestra.run.smithi061.stdout: "auid": 0, 2023-12-06T18:33:16.753 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_mode": "selfmanaged", 2023-12-06T18:33:16.753 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_seq": 0, 2023-12-06T18:33:16.753 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_epoch": 0, 2023-12-06T18:33:16.753 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_snaps": [], 2023-12-06T18:33:16.753 INFO:teuthology.orchestra.run.smithi061.stdout: "removed_snaps": "[]", 2023-12-06T18:33:16.753 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_bytes": 0, 2023-12-06T18:33:16.753 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_objects": 0, 2023-12-06T18:33:16.753 INFO:teuthology.orchestra.run.smithi061.stdout: "tiers": [], 2023-12-06T18:33:16.753 INFO:teuthology.orchestra.run.smithi061.stdout: "tier_of": -1, 2023-12-06T18:33:16.753 INFO:teuthology.orchestra.run.smithi061.stdout: "read_tier": -1, 2023-12-06T18:33:16.754 INFO:teuthology.orchestra.run.smithi061.stdout: "write_tier": -1, 2023-12-06T18:33:16.754 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_mode": "none", 2023-12-06T18:33:16.754 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_bytes": 0, 2023-12-06T18:33:16.754 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_objects": 0, 2023-12-06T18:33:16.754 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-06T18:33:16.754 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-06T18:33:16.754 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-06T18:33:16.754 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_flush_age": 0, 2023-12-06T18:33:16.754 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_evict_age": 0, 2023-12-06T18:33:16.754 INFO:teuthology.orchestra.run.smithi061.stdout: "erasure_code_profile": "", 2023-12-06T18:33:16.754 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_params": { 2023-12-06T18:33:16.754 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "none" 2023-12-06T18:33:16.754 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.754 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_period": 0, 2023-12-06T18:33:16.754 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_count": 0, 2023-12-06T18:33:16.754 INFO:teuthology.orchestra.run.smithi061.stdout: "use_gmt_hitset": true, 2023-12-06T18:33:16.755 INFO:teuthology.orchestra.run.smithi061.stdout: "min_read_recency_for_promote": 0, 2023-12-06T18:33:16.755 INFO:teuthology.orchestra.run.smithi061.stdout: "min_write_recency_for_promote": 0, 2023-12-06T18:33:16.755 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_grade_decay_rate": 0, 2023-12-06T18:33:16.755 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_search_last_n": 0, 2023-12-06T18:33:16.755 INFO:teuthology.orchestra.run.smithi061.stdout: "grade_table": [], 2023-12-06T18:33:16.755 INFO:teuthology.orchestra.run.smithi061.stdout: "stripe_width": 0, 2023-12-06T18:33:16.755 INFO:teuthology.orchestra.run.smithi061.stdout: "expected_num_objects": 0, 2023-12-06T18:33:16.755 INFO:teuthology.orchestra.run.smithi061.stdout: "fast_read": false, 2023-12-06T18:33:16.755 INFO:teuthology.orchestra.run.smithi061.stdout: "options": { 2023-12-06T18:33:16.755 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_max": 32, 2023-12-06T18:33:16.755 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_min": 1 2023-12-06T18:33:16.755 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.755 INFO:teuthology.orchestra.run.smithi061.stdout: "application_metadata": { 2023-12-06T18:33:16.755 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr": {} 2023-12-06T18:33:16.755 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.755 INFO:teuthology.orchestra.run.smithi061.stdout: "read_balance": { 2023-12-06T18:33:16.756 INFO:teuthology.orchestra.run.smithi061.stdout: "score_acting": 6.9000000953674316, 2023-12-06T18:33:16.756 INFO:teuthology.orchestra.run.smithi061.stdout: "score_stable": 6.9000000953674316, 2023-12-06T18:33:16.756 INFO:teuthology.orchestra.run.smithi061.stdout: "optimal_score": 0.28999999165534973, 2023-12-06T18:33:16.756 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_acting": 2, 2023-12-06T18:33:16.756 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_stable": 2, 2023-12-06T18:33:16.756 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity_weighted": 0.76666069030761719, 2023-12-06T18:33:16.756 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity": 0.87999999523162842, 2023-12-06T18:33:16.756 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity_weighted": 0.76999998092651367 2023-12-06T18:33:16.756 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.756 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.756 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.756 INFO:teuthology.orchestra.run.smithi061.stdout: "pool": 2, 2023-12-06T18:33:16.756 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_name": "rbd", 2023-12-06T18:33:16.756 INFO:teuthology.orchestra.run.smithi061.stdout: "create_time": "2023-12-06T18:27:01.712581+0000", 2023-12-06T18:33:16.756 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-12-06T18:33:16.756 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_names": "hashpspool", 2023-12-06T18:33:16.756 INFO:teuthology.orchestra.run.smithi061.stdout: "type": 1, 2023-12-06T18:33:16.757 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 2, 2023-12-06T18:33:16.757 INFO:teuthology.orchestra.run.smithi061.stdout: "min_size": 1, 2023-12-06T18:33:16.757 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_rule": 0, 2023-12-06T18:33:16.757 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_count": 0, 2023-12-06T18:33:16.757 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_target": 0, 2023-12-06T18:33:16.757 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_barrier": 0, 2023-12-06T18:33:16.757 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-06T18:33:16.757 INFO:teuthology.orchestra.run.smithi061.stdout: "object_hash": 2, 2023-12-06T18:33:16.757 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscale_mode": "off", 2023-12-06T18:33:16.757 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num": 8, 2023-12-06T18:33:16.757 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num": 8, 2023-12-06T18:33:16.757 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num_target": 8, 2023-12-06T18:33:16.757 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_target": 8, 2023-12-06T18:33:16.757 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_pending": 8, 2023-12-06T18:33:16.757 INFO:teuthology.orchestra.run.smithi061.stdout: "last_pg_merge_meta": { 2023-12-06T18:33:16.757 INFO:teuthology.orchestra.run.smithi061.stdout: "source_pgid": "0.0", 2023-12-06T18:33:16.758 INFO:teuthology.orchestra.run.smithi061.stdout: "ready_epoch": 0, 2023-12-06T18:33:16.758 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_started": 0, 2023-12-06T18:33:16.758 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_clean": 0, 2023-12-06T18:33:16.758 INFO:teuthology.orchestra.run.smithi061.stdout: "source_version": "0'0", 2023-12-06T18:33:16.758 INFO:teuthology.orchestra.run.smithi061.stdout: "target_version": "0'0" 2023-12-06T18:33:16.758 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.758 INFO:teuthology.orchestra.run.smithi061.stdout: "last_change": "24", 2023-12-06T18:33:16.758 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend": "0", 2023-12-06T18:33:16.758 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-06T18:33:16.758 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_preluminous": "0", 2023-12-06T18:33:16.758 INFO:teuthology.orchestra.run.smithi061.stdout: "auid": 0, 2023-12-06T18:33:16.758 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_mode": "selfmanaged", 2023-12-06T18:33:16.758 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_seq": 0, 2023-12-06T18:33:16.758 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_epoch": 0, 2023-12-06T18:33:16.758 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_snaps": [], 2023-12-06T18:33:16.758 INFO:teuthology.orchestra.run.smithi061.stdout: "removed_snaps": "[]", 2023-12-06T18:33:16.759 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_bytes": 0, 2023-12-06T18:33:16.759 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_objects": 0, 2023-12-06T18:33:16.759 INFO:teuthology.orchestra.run.smithi061.stdout: "tiers": [], 2023-12-06T18:33:16.759 INFO:teuthology.orchestra.run.smithi061.stdout: "tier_of": -1, 2023-12-06T18:33:16.759 INFO:teuthology.orchestra.run.smithi061.stdout: "read_tier": -1, 2023-12-06T18:33:16.759 INFO:teuthology.orchestra.run.smithi061.stdout: "write_tier": -1, 2023-12-06T18:33:16.759 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_mode": "none", 2023-12-06T18:33:16.759 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_bytes": 0, 2023-12-06T18:33:16.759 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_objects": 0, 2023-12-06T18:33:16.759 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-06T18:33:16.759 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-06T18:33:16.759 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-06T18:33:16.759 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_flush_age": 0, 2023-12-06T18:33:16.759 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_evict_age": 0, 2023-12-06T18:33:16.759 INFO:teuthology.orchestra.run.smithi061.stdout: "erasure_code_profile": "", 2023-12-06T18:33:16.760 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_params": { 2023-12-06T18:33:16.760 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "none" 2023-12-06T18:33:16.760 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.760 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_period": 0, 2023-12-06T18:33:16.760 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_count": 0, 2023-12-06T18:33:16.760 INFO:teuthology.orchestra.run.smithi061.stdout: "use_gmt_hitset": true, 2023-12-06T18:33:16.760 INFO:teuthology.orchestra.run.smithi061.stdout: "min_read_recency_for_promote": 0, 2023-12-06T18:33:16.760 INFO:teuthology.orchestra.run.smithi061.stdout: "min_write_recency_for_promote": 0, 2023-12-06T18:33:16.760 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_grade_decay_rate": 0, 2023-12-06T18:33:16.760 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_search_last_n": 0, 2023-12-06T18:33:16.760 INFO:teuthology.orchestra.run.smithi061.stdout: "grade_table": [], 2023-12-06T18:33:16.760 INFO:teuthology.orchestra.run.smithi061.stdout: "stripe_width": 0, 2023-12-06T18:33:16.760 INFO:teuthology.orchestra.run.smithi061.stdout: "expected_num_objects": 0, 2023-12-06T18:33:16.760 INFO:teuthology.orchestra.run.smithi061.stdout: "fast_read": false, 2023-12-06T18:33:16.761 INFO:teuthology.orchestra.run.smithi061.stdout: "options": {}, 2023-12-06T18:33:16.761 INFO:teuthology.orchestra.run.smithi061.stdout: "application_metadata": { 2023-12-06T18:33:16.761 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd": {} 2023-12-06T18:33:16.761 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.761 INFO:teuthology.orchestra.run.smithi061.stdout: "read_balance": { 2023-12-06T18:33:16.761 INFO:teuthology.orchestra.run.smithi061.stdout: "score_acting": 2.630000114440918, 2023-12-06T18:33:16.761 INFO:teuthology.orchestra.run.smithi061.stdout: "score_stable": 2.630000114440918, 2023-12-06T18:33:16.761 INFO:teuthology.orchestra.run.smithi061.stdout: "optimal_score": 1, 2023-12-06T18:33:16.761 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_acting": 2.630000114440918, 2023-12-06T18:33:16.761 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_stable": 2.630000114440918, 2023-12-06T18:33:16.761 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity_weighted": 0.76666069030761719, 2023-12-06T18:33:16.761 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity": 0.87999999523162842, 2023-12-06T18:33:16.761 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity_weighted": 0.76999998092651367 2023-12-06T18:33:16.761 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.761 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.761 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:33:16.762 INFO:teuthology.orchestra.run.smithi061.stdout: "osds": [ 2023-12-06T18:33:16.762 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.762 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 0, 2023-12-06T18:33:16.762 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "5e6e17b9-e72e-4e34-a531-0830d659bfe0", 2023-12-06T18:33:16.762 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:33:16.762 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:33:16.762 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:33:16.762 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 0.4124908447265625, 2023-12-06T18:33:16.762 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:33:16.762 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:33:16.762 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 16, 2023-12-06T18:33:16.762 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 0, 2023-12-06T18:33:16.762 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:33:16.762 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:33:16.762 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:33:16.763 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.763 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.763 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6805", 2023-12-06T18:33:16.763 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:33:16.763 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.763 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.763 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:33:16.763 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.763 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.763 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.763 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6806", 2023-12-06T18:33:16.763 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:33:16.763 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.763 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.763 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.764 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:33:16.764 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.764 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.764 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6808", 2023-12-06T18:33:16.764 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:33:16.764 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.764 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.764 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.764 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:33:16.764 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.764 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.764 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.764 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6807", 2023-12-06T18:33:16.764 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:33:16.765 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.765 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.765 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6805/281956997", 2023-12-06T18:33:16.765 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6806/281956997", 2023-12-06T18:33:16.765 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6808/281956997", 2023-12-06T18:33:16.765 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6807/281956997", 2023-12-06T18:33:16.765 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:33:16.765 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:33:16.765 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:33:16.765 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.765 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.765 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.765 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 1, 2023-12-06T18:33:16.765 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "b78c49b0-360d-4351-b44e-65ccaaeb768f", 2023-12-06T18:33:16.765 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:33:16.765 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:33:16.766 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:33:16.766 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:33:16.766 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:33:16.766 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:33:16.766 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 16, 2023-12-06T18:33:16.766 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 79, 2023-12-06T18:33:16.766 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:33:16.766 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:33:16.766 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:33:16.766 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.766 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.766 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.766 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6809", 2023-12-06T18:33:16.766 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:33:16.766 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.766 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.766 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.767 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:33:16.767 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.767 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.767 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6810", 2023-12-06T18:33:16.767 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:33:16.767 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.767 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.767 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.767 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:33:16.767 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.767 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.767 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.767 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6812", 2023-12-06T18:33:16.767 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:33:16.767 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.768 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.768 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:33:16.768 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.768 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.768 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.768 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6811", 2023-12-06T18:33:16.768 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:33:16.768 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.768 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.768 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.768 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6809/3306625622", 2023-12-06T18:33:16.768 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6810/3306625622", 2023-12-06T18:33:16.768 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6812/3306625622", 2023-12-06T18:33:16.768 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6811/3306625622", 2023-12-06T18:33:16.768 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:33:16.768 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:33:16.769 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:33:16.769 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.769 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 2, 2023-12-06T18:33:16.769 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "a2af3151-a4da-4c34-8fb6-78abf035785b", 2023-12-06T18:33:16.769 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:33:16.769 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:33:16.769 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:33:16.769 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:33:16.769 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:33:16.769 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:33:16.769 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 16, 2023-12-06T18:33:16.769 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 35, 2023-12-06T18:33:16.769 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:33:16.769 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:33:16.769 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:33:16.770 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.770 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.770 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6801", 2023-12-06T18:33:16.770 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:33:16.770 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.770 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.770 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:33:16.770 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.770 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.770 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6802", 2023-12-06T18:33:16.770 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:33:16.770 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.771 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.771 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:33:16.771 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.771 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.771 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6804", 2023-12-06T18:33:16.771 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:33:16.771 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.771 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.771 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:33:16.771 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.771 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.771 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6803", 2023-12-06T18:33:16.772 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:33:16.772 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.772 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.772 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6801/3589270120", 2023-12-06T18:33:16.772 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6802/3589270120", 2023-12-06T18:33:16.772 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6804/3589270120", 2023-12-06T18:33:16.772 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6803/3589270120", 2023-12-06T18:33:16.772 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:33:16.772 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:33:16.772 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:33:16.772 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.772 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 3, 2023-12-06T18:33:16.772 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "1c754ed6-c535-454f-b35b-00f62cd6ce5a", 2023-12-06T18:33:16.772 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:33:16.773 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:33:16.773 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:33:16.773 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:33:16.773 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:33:16.773 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:33:16.773 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 18, 2023-12-06T18:33:16.773 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 100, 2023-12-06T18:33:16.773 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:33:16.773 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:33:16.773 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:33:16.773 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.773 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.773 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6813", 2023-12-06T18:33:16.773 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:33:16.773 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.773 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.774 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:33:16.774 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.774 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.774 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6814", 2023-12-06T18:33:16.774 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:33:16.774 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.774 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.774 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:33:16.774 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.774 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.774 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6816", 2023-12-06T18:33:16.775 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:33:16.775 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.775 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.775 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:33:16.775 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.775 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.775 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6815", 2023-12-06T18:33:16.775 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:33:16.775 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.775 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.775 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6813/2849920842", 2023-12-06T18:33:16.775 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6814/2849920842", 2023-12-06T18:33:16.775 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6816/2849920842", 2023-12-06T18:33:16.775 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6815/2849920842", 2023-12-06T18:33:16.776 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:33:16.776 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:33:16.776 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:33:16.776 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.776 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 4, 2023-12-06T18:33:16.776 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "766572af-b9ca-4f38-927d-36f26101ba51", 2023-12-06T18:33:16.776 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:33:16.776 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:33:16.776 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:33:16.776 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 0.720794677734375, 2023-12-06T18:33:16.776 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:33:16.776 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:33:16.776 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 15, 2023-12-06T18:33:16.776 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 100, 2023-12-06T18:33:16.776 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:33:16.777 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:33:16.777 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:33:16.777 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.777 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6800", 2023-12-06T18:33:16.777 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:33:16.777 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.777 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.777 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.777 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:33:16.777 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.777 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.777 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.777 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6801", 2023-12-06T18:33:16.777 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:33:16.777 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.778 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.778 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:33:16.778 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.778 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.778 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6803", 2023-12-06T18:33:16.778 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:33:16.778 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.778 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.778 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.778 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:33:16.778 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.778 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.778 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.778 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6802", 2023-12-06T18:33:16.778 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:33:16.779 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.779 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.779 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6800/2778770069", 2023-12-06T18:33:16.779 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6801/2778770069", 2023-12-06T18:33:16.779 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6803/2778770069", 2023-12-06T18:33:16.779 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6802/2778770069", 2023-12-06T18:33:16.779 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:33:16.779 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:33:16.779 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:33:16.779 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.779 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.779 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.779 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 5, 2023-12-06T18:33:16.779 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "7a566b60-1543-41d7-894c-e2fbb63e48dc", 2023-12-06T18:33:16.780 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:33:16.780 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:33:16.780 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:33:16.780 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:33:16.780 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:33:16.780 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:33:16.780 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 15, 2023-12-06T18:33:16.780 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 97, 2023-12-06T18:33:16.780 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:33:16.780 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:33:16.780 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:33:16.780 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.780 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.780 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.780 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6809", 2023-12-06T18:33:16.780 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:33:16.781 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.781 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.781 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:33:16.781 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.781 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.781 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6810", 2023-12-06T18:33:16.781 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:33:16.781 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.781 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.781 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.781 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:33:16.781 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.781 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.781 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.782 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6814", 2023-12-06T18:33:16.782 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:33:16.782 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.782 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.782 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:33:16.782 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.782 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.782 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.782 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6812", 2023-12-06T18:33:16.782 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:33:16.782 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.782 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.782 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.782 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6809/263671335", 2023-12-06T18:33:16.782 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6810/263671335", 2023-12-06T18:33:16.783 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6814/263671335", 2023-12-06T18:33:16.783 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6812/263671335", 2023-12-06T18:33:16.783 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:33:16.783 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:33:16.783 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:33:16.783 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.783 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.783 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.783 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 6, 2023-12-06T18:33:16.783 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "28cc2358-4ac4-40b2-91a4-20edfe625a73", 2023-12-06T18:33:16.783 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:33:16.783 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 0, 2023-12-06T18:33:16.783 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 0, 2023-12-06T18:33:16.783 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 0, 2023-12-06T18:33:16.783 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:33:16.783 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:33:16.784 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 14, 2023-12-06T18:33:16.784 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 25, 2023-12-06T18:33:16.784 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:33:16.784 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:33:16.784 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:33:16.784 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.784 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.784 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.784 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6808", 2023-12-06T18:33:16.784 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:33:16.784 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.784 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.784 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.784 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:33:16.784 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.785 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.785 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6811", 2023-12-06T18:33:16.785 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:33:16.785 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.785 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.785 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.785 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:33:16.785 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.785 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.785 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.785 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6815", 2023-12-06T18:33:16.785 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:33:16.785 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.785 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.786 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:33:16.786 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.786 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.786 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.786 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6813", 2023-12-06T18:33:16.786 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:33:16.786 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.786 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.786 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.786 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6808/4245639079", 2023-12-06T18:33:16.786 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6811/4245639079", 2023-12-06T18:33:16.786 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6815/4245639079", 2023-12-06T18:33:16.786 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6813/4245639079", 2023-12-06T18:33:16.786 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:33:16.787 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:33:16.787 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:33:16.787 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.787 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.787 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.787 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 7, 2023-12-06T18:33:16.787 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "8f79279b-d2d0-438e-b3c2-88df1241ff30", 2023-12-06T18:33:16.787 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:33:16.787 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:33:16.787 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 0.747711181640625, 2023-12-06T18:33:16.787 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:33:16.787 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 14, 2023-12-06T18:33:16.787 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 37, 2023-12-06T18:33:16.787 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 61, 2023-12-06T18:33:16.787 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 62, 2023-12-06T18:33:16.788 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 38, 2023-12-06T18:33:16.788 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:33:16.788 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:33:16.788 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.788 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.788 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6804", 2023-12-06T18:33:16.788 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3075390540 2023-12-06T18:33:16.788 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.788 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.788 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.788 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:33:16.788 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.788 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.788 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.788 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6805", 2023-12-06T18:33:16.788 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3075390540 2023-12-06T18:33:16.789 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.789 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.789 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:33:16.789 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.789 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.789 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6807", 2023-12-06T18:33:16.789 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3075390540 2023-12-06T18:33:16.789 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.789 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.789 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.789 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:33:16.789 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:16.789 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.789 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:16.790 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6806", 2023-12-06T18:33:16.790 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3075390540 2023-12-06T18:33:16.790 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.790 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.790 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.790 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6804/3075390540", 2023-12-06T18:33:16.790 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6805/3075390540", 2023-12-06T18:33:16.790 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6807/3075390540", 2023-12-06T18:33:16.790 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6806/3075390540", 2023-12-06T18:33:16.790 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:33:16.790 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:33:16.790 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:33:16.790 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.791 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.791 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:33:16.791 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_xinfo": [ 2023-12-06T18:33:16.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.791 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 0, 2023-12-06T18:33:16.791 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:33:16.791 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:33:16.791 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:33:16.791 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:33:16.791 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:33:16.791 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:52.171904+0000", 2023-12-06T18:33:16.791 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:33:16.791 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.791 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.791 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 1, 2023-12-06T18:33:16.792 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:33:16.792 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:33:16.792 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:33:16.792 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:33:16.792 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:33:16.792 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:52.428840+0000", 2023-12-06T18:33:16.792 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:33:16.792 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.792 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.792 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 2, 2023-12-06T18:33:16.792 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:33:16.792 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:33:16.792 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:33:16.792 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:33:16.793 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:33:16.793 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:51.681954+0000", 2023-12-06T18:33:16.793 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:33:16.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.793 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 3, 2023-12-06T18:33:16.793 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:33:16.793 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:33:16.793 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:33:16.793 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:33:16.793 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:33:16.793 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:53.687417+0000", 2023-12-06T18:33:16.793 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:33:16.793 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.793 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.794 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 4, 2023-12-06T18:33:16.794 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:33:16.794 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:33:16.794 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:33:16.794 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:33:16.794 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:33:16.794 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:50.158682+0000", 2023-12-06T18:33:16.794 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:33:16.794 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.794 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.794 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 5, 2023-12-06T18:33:16.794 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:33:16.794 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:33:16.794 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:33:16.794 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:33:16.795 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:33:16.795 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:50.443807+0000", 2023-12-06T18:33:16.795 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:33:16.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.795 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 6, 2023-12-06T18:33:16.795 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:33:16.795 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:33:16.795 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:33:16.795 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:33:16.795 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 65536, 2023-12-06T18:33:16.795 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:49.750276+0000", 2023-12-06T18:33:16.795 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:33:16.795 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.795 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.796 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 7, 2023-12-06T18:33:16.796 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:33:16.796 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:33:16.796 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:33:16.796 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:33:16.796 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:33:16.796 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:49.770987+0000", 2023-12-06T18:33:16.796 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 38 2023-12-06T18:33:16.796 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.796 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:33:16.796 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_upmap": [ 2023-12-06T18:33:16.796 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.796 INFO:teuthology.orchestra.run.smithi061.stdout: "pgid": "2.7", 2023-12-06T18:33:16.796 INFO:teuthology.orchestra.run.smithi061.stdout: "osds": [ 2023-12-06T18:33:16.796 INFO:teuthology.orchestra.run.smithi061.stdout: 5, 2023-12-06T18:33:16.796 INFO:teuthology.orchestra.run.smithi061.stdout: 3 2023-12-06T18:33:16.797 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.797 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.797 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:33:16.797 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_upmap_items": [ 2023-12-06T18:33:16.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.797 INFO:teuthology.orchestra.run.smithi061.stdout: "pgid": "2.3", 2023-12-06T18:33:16.797 INFO:teuthology.orchestra.run.smithi061.stdout: "mappings": [ 2023-12-06T18:33:16.797 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:16.797 INFO:teuthology.orchestra.run.smithi061.stdout: "from": 3, 2023-12-06T18:33:16.797 INFO:teuthology.orchestra.run.smithi061.stdout: "to": 1 2023-12-06T18:33:16.797 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.797 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:16.797 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.797 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:33:16.797 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_upmap_primaries": [], 2023-12-06T18:33:16.797 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_temp": [], 2023-12-06T18:33:16.797 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_temp": [], 2023-12-06T18:33:16.798 INFO:teuthology.orchestra.run.smithi061.stdout: "blocklist": {}, 2023-12-06T18:33:16.798 INFO:teuthology.orchestra.run.smithi061.stdout: "range_blocklist": {}, 2023-12-06T18:33:16.798 INFO:teuthology.orchestra.run.smithi061.stdout: "erasure_code_profiles": { 2023-12-06T18:33:16.798 INFO:teuthology.orchestra.run.smithi061.stdout: "default": { 2023-12-06T18:33:16.798 INFO:teuthology.orchestra.run.smithi061.stdout: "crush-failure-domain": "osd", 2023-12-06T18:33:16.798 INFO:teuthology.orchestra.run.smithi061.stdout: "k": "2", 2023-12-06T18:33:16.798 INFO:teuthology.orchestra.run.smithi061.stdout: "m": "1", 2023-12-06T18:33:16.798 INFO:teuthology.orchestra.run.smithi061.stdout: "plugin": "jerasure", 2023-12-06T18:33:16.798 INFO:teuthology.orchestra.run.smithi061.stdout: "technique": "reed_sol_van" 2023-12-06T18:33:16.798 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.798 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:16.798 INFO:teuthology.orchestra.run.smithi061.stdout: "removed_snaps_queue": [], 2023-12-06T18:33:16.798 INFO:teuthology.orchestra.run.smithi061.stdout: "new_removed_snaps": [], 2023-12-06T18:33:16.798 INFO:teuthology.orchestra.run.smithi061.stdout: "new_purged_snaps": [], 2023-12-06T18:33:16.798 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_node_flags": {}, 2023-12-06T18:33:16.799 INFO:teuthology.orchestra.run.smithi061.stdout: "device_class_flags": {}, 2023-12-06T18:33:16.799 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode": { 2023-12-06T18:33:16.799 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode_enabled": false, 2023-12-06T18:33:16.799 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_bucket_count": 0, 2023-12-06T18:33:16.799 INFO:teuthology.orchestra.run.smithi061.stdout: "degraded_stretch_mode": 0, 2023-12-06T18:33:16.799 INFO:teuthology.orchestra.run.smithi061.stdout: "recovering_stretch_mode": 0, 2023-12-06T18:33:16.799 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode_bucket": 0 2023-12-06T18:33:16.799 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:16.799 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:16.801 INFO:tasks.thrashosds.thrasher:j is {'epoch': 107, 'fsid': '497e3283-cc16-40ff-a72c-a7487474fb4b', 'created': '2023-12-06T18:26:07.122862+0000', 'modified': '2023-12-06T18:33:14.786065+0000', 'last_up_change': '2023-12-06T18:30:04.956811+0000', 'last_in_change': '2023-12-06T18:32:58.826891+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 8, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-06T18:26:52.648377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7666606903076172, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.7699999809265137}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-12-06T18:27:01.712581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.7666606903076172, 'average_primary_affinity': 0.8799999952316284, 'average_primary_affinity_weighted': 0.7699999809265137}}], 'osds': [{'osd': 0, 'uuid': '5e6e17b9-e72e-4e34-a531-0830d659bfe0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.4124908447265625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6805', 'nonce': 281956997}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6806', 'nonce': 281956997}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6808', 'nonce': 281956997}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6807', 'nonce': 281956997}]}, 'public_addr': '172.21.15.61:6805/281956997', 'cluster_addr': '172.21.15.61:6806/281956997', 'heartbeat_back_addr': '172.21.15.61:6808/281956997', 'heartbeat_front_addr': '172.21.15.61:6807/281956997', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b78c49b0-360d-4351-b44e-65ccaaeb768f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 79, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6809', 'nonce': 3306625622}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6810', 'nonce': 3306625622}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6812', 'nonce': 3306625622}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6811', 'nonce': 3306625622}]}, 'public_addr': '172.21.15.61:6809/3306625622', 'cluster_addr': '172.21.15.61:6810/3306625622', 'heartbeat_back_addr': '172.21.15.61:6812/3306625622', 'heartbeat_front_addr': '172.21.15.61:6811/3306625622', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'a2af3151-a4da-4c34-8fb6-78abf035785b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 35, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6801', 'nonce': 3589270120}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6802', 'nonce': 3589270120}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6804', 'nonce': 3589270120}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6803', 'nonce': 3589270120}]}, 'public_addr': '172.21.15.61:6801/3589270120', 'cluster_addr': '172.21.15.61:6802/3589270120', 'heartbeat_back_addr': '172.21.15.61:6804/3589270120', 'heartbeat_front_addr': '172.21.15.61:6803/3589270120', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '1c754ed6-c535-454f-b35b-00f62cd6ce5a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 100, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6813', 'nonce': 2849920842}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6814', 'nonce': 2849920842}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6816', 'nonce': 2849920842}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6815', 'nonce': 2849920842}]}, 'public_addr': '172.21.15.61:6813/2849920842', 'cluster_addr': '172.21.15.61:6814/2849920842', 'heartbeat_back_addr': '172.21.15.61:6816/2849920842', 'heartbeat_front_addr': '172.21.15.61:6815/2849920842', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '766572af-b9ca-4f38-927d-36f26101ba51', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.720794677734375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 100, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6800', 'nonce': 2778770069}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6801', 'nonce': 2778770069}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6803', 'nonce': 2778770069}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6802', 'nonce': 2778770069}]}, 'public_addr': '172.21.15.152:6800/2778770069', 'cluster_addr': '172.21.15.152:6801/2778770069', 'heartbeat_back_addr': '172.21.15.152:6803/2778770069', 'heartbeat_front_addr': '172.21.15.152:6802/2778770069', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7a566b60-1543-41d7-894c-e2fbb63e48dc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 97, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6809', 'nonce': 263671335}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6810', 'nonce': 263671335}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6814', 'nonce': 263671335}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6812', 'nonce': 263671335}]}, 'public_addr': '172.21.15.152:6809/263671335', 'cluster_addr': '172.21.15.152:6810/263671335', 'heartbeat_back_addr': '172.21.15.152:6814/263671335', 'heartbeat_front_addr': '172.21.15.152:6812/263671335', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '28cc2358-4ac4-40b2-91a4-20edfe625a73', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6808', 'nonce': 4245639079}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6811', 'nonce': 4245639079}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6815', 'nonce': 4245639079}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6813', 'nonce': 4245639079}]}, 'public_addr': '172.21.15.152:6808/4245639079', 'cluster_addr': '172.21.15.152:6811/4245639079', 'heartbeat_back_addr': '172.21.15.152:6815/4245639079', 'heartbeat_front_addr': '172.21.15.152:6813/4245639079', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '8f79279b-d2d0-438e-b3c2-88df1241ff30', 'up': 1, 'in': 1, 'weight': 0.747711181640625, 'primary_affinity': 1, 'last_clean_begin': 14, 'last_clean_end': 37, 'up_from': 61, 'up_thru': 62, 'down_at': 38, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6804', 'nonce': 3075390540}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6805', 'nonce': 3075390540}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6807', 'nonce': 3075390540}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6806', 'nonce': 3075390540}]}, 'public_addr': '172.21.15.152:6804/3075390540', 'cluster_addr': '172.21.15.152:6805/3075390540', 'heartbeat_back_addr': '172.21.15.152:6807/3075390540', 'heartbeat_front_addr': '172.21.15.152:6806/3075390540', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:52.171904+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:52.428840+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:51.681954+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:53.687417+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:50.158682+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:50.443807+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2023-12-06T18:26:49.750276+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:49.770987+0000', 'dead_epoch': 38}], 'pg_upmap': [{'pgid': '2.7', 'osds': [5, 3]}], 'pg_upmap_items': [{'pgid': '2.3', 'mappings': [{'from': 3, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2023-12-06T18:33:16.801 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:33:16.801 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:16.797+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:16.844 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '17 of 30' 2023-12-06T18:33:16.844 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:16.845 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:16.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:16.845 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:33:16.845 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:16.845 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:33:16.845 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:16.845 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 175.177780652, 2023-12-06T18:33:16.845 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:33:16.845 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:16.845 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:16.845 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:16.845 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:16.846 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:16.846 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:33:16.846 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:16.846 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:16.846 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:16.846 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:16.846 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:16.846 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:16.846 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:16.846 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:16.846 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:16.846 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:16.846 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:16.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:16.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:16.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:16.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:33:16.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:33:16.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:16.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:16.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:16.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:33:16.847 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:33:16.847 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:16.847 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:16.847 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:16.847 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:33:16.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:33:16.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:16.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:16.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:16.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:33:16.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:33:16.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:16.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:16.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:16.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:33:16.848 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:33:16.848 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:16.848 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:16.848 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:16.848 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:33:16.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:33:16.849 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:16.849 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:16.849 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:16.849 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:16.849 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:16.849 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:12 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2023-12-06T18:33:16.849 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:16.849 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 55.027834026999997, 2023-12-06T18:33:16.849 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0032177460000000001, 2023-12-06T18:33:16.849 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:16.849 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:16.849 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:16.849 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:16.850 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:16.850 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 12 2023-12-06T18:33:16.850 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:16.850 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:16.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:16.850 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:16.850 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:16.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:16.850 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:16.850 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:16.850 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:16.850 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:16.850 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:16.850 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:16.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:16.851 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:16.851 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807030+0000", 2023-12-06T18:33:16.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6030000000000001e-05 2023-12-06T18:33:16.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:16.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:16.851 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:16.851 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807210+0000", 2023-12-06T18:33:16.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017936800000000001 2023-12-06T18:33:16.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:16.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:16.851 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:16.851 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807443+0000", 2023-12-06T18:33:16.851 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00023339699999999999 2023-12-06T18:33:16.851 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:16.851 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:16.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:16.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807614+0000", 2023-12-06T18:33:16.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000171032 2023-12-06T18:33:16.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:16.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:16.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:16.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.808696+0000", 2023-12-06T18:33:16.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010817190000000001 2023-12-06T18:33:16.852 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:16.852 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:16.852 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:16.852 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.810232+0000", 2023-12-06T18:33:16.852 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0015361999999999999 2023-12-06T18:33:16.852 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:16.852 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:16.852 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:16.853 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:16.853 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:16.853 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:16.854 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:33:16.899 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-12-06T18:33:16.901 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:16.898+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:17.003 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:16.999+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:17.052 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:17.052 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:17.052 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:17.052 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:17.062 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:33:17.105 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:17.100+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:17.205 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:17.201+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:17.239 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:17.239 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:17.239 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:17.239 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:17.239 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:17.248 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:33:17.333 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:17.329+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:17.406 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:17.403+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:17.432 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:17.432 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [] 2023-12-06T18:33:17.432 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:17.441 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:33:17.507 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:17.504+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:17.534 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:33:17.535 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:33:17.547 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2023-12-06T18:33:16.279736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":200304,"kb_used_data":5980,"kb_used_omap":13,"kb_used_meta":190002,"kb_avail":660402576,"statfs":{"total":676457349120,"available":676252237824,"internally_reserved":0,"allocated":6123520,"data_stored":2695639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13963,"internal_metadata":194562421},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.911920"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":59,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-12-06T18:33:08.876148+0000","last_change":"2023-12-06T18:29:09.051293+0000","last_active":"2023-12-06T18:33:08.876148+0000","last_peered":"2023-12-06T18:33:08.876148+0000","last_clean":"2023-12-06T18:33:08.876148+0000","last_became_active":"2023-12-06T18:29:09.045331+0000","last_became_peered":"2023-12-06T18:29:09.045331+0000","last_unstale":"2023-12-06T18:33:08.876148+0000","last_undegraded":"2023-12-06T18:33:08.876148+0000","last_fullsized":"2023-12-06T18:33:08.876148+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T02:46:01.854508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-12-06T18:33:08.865467+0000","last_change":"2023-12-06T18:33:00.623558+0000","last_active":"2023-12-06T18:33:08.865467+0000","last_peered":"2023-12-06T18:33:08.865467+0000","last_clean":"2023-12-06T18:33:08.865467+0000","last_became_active":"2023-12-06T18:33:00.616353+0000","last_became_peered":"2023-12-06T18:33:00.616353+0000","last_unstale":"2023-12-06T18:33:08.865467+0000","last_undegraded":"2023-12-06T18:33:08.865467+0000","last_fullsized":"2023-12-06T18:33:08.865467+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T18:33:31.069235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":105,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-12-06T18:33:05.781669+0000","last_change":"2023-12-06T18:33:00.592340+0000","last_active":"2023-12-06T18:33:05.781669+0000","last_peered":"2023-12-06T18:33:05.781669+0000","last_clean":"2023-12-06T18:33:05.781669+0000","last_became_active":"2023-12-06T18:33:00.581037+0000","last_became_peered":"2023-12-06T18:33:00.581037+0000","last_unstale":"2023-12-06T18:33:05.781669+0000","last_undegraded":"2023-12-06T18:33:05.781669+0000","last_fullsized":"2023-12-06T18:33:05.781669+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T06:13:54.498147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":75,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-12-06T18:33:08.878689+0000","last_change":"2023-12-06T18:32:23.586404+0000","last_active":"2023-12-06T18:33:08.878689+0000","last_peered":"2023-12-06T18:33:08.878689+0000","last_clean":"2023-12-06T18:33:08.878689+0000","last_became_active":"2023-12-06T18:32:23.580588+0000","last_became_peered":"2023-12-06T18:32:23.580588+0000","last_unstale":"2023-12-06T18:33:08.878689+0000","last_undegraded":"2023-12-06T18:33:08.878689+0000","last_fullsized":"2023-12-06T18:33:08.878689+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:31:58.277721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":98,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-12-06T18:33:05.783796+0000","last_change":"2023-12-06T18:32:23.632179+0000","last_active":"2023-12-06T18:33:05.783796+0000","last_peered":"2023-12-06T18:33:05.783796+0000","last_clean":"2023-12-06T18:33:05.783796+0000","last_became_active":"2023-12-06T18:32:23.623784+0000","last_became_peered":"2023-12-06T18:32:23.623784+0000","last_unstale":"2023-12-06T18:33:05.783796+0000","last_undegraded":"2023-12-06T18:33:05.783796+0000","last_fullsized":"2023-12-06T18:33:05.783796+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T21:18:05.348471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":73,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-12-06T18:33:06.105324+0000","last_change":"2023-12-06T18:30:08.762262+0000","last_active":"2023-12-06T18:33:06.105324+0000","last_peered":"2023-12-06T18:33:06.105324+0000","last_clean":"2023-12-06T18:33:06.105324+0000","last_became_active":"2023-12-06T18:30:08.733421+0000","last_became_peered":"2023-12-06T18:30:08.733421+0000","last_unstale":"2023-12-06T18:33:06.105324+0000","last_undegraded":"2023-12-06T18:33:06.105324+0000","last_fullsized":"2023-12-06T18:33:06.105324+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:43:31.796397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":106,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-12-06T18:33:07.292487+0000","last_change":"2023-12-06T18:28:47.876556+0000","last_active":"2023-12-06T18:33:07.292487+0000","last_peered":"2023-12-06T18:33:07.292487+0000","last_clean":"2023-12-06T18:33:07.292487+0000","last_became_active":"2023-12-06T18:28:47.865169+0000","last_became_peered":"2023-12-06T18:28:47.865169+0000","last_unstale":"2023-12-06T18:33:07.292487+0000","last_undegraded":"2023-12-06T18:33:07.292487+0000","last_fullsized":"2023-12-06T18:33:07.292487+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:16:40.095542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":36,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-12-06T18:33:08.351786+0000","last_change":"2023-12-06T18:31:57.187068+0000","last_active":"2023-12-06T18:33:08.351786+0000","last_peered":"2023-12-06T18:33:08.351786+0000","last_clean":"2023-12-06T18:33:08.351786+0000","last_became_active":"2023-12-06T18:31:57.175444+0000","last_became_peered":"2023-12-06T18:31:57.175444+0000","last_unstale":"2023-12-06T18:33:08.351786+0000","last_undegraded":"2023-12-06T18:33:08.351786+0000","last_fullsized":"2023-12-06T18:33:08.351786+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:25:09.547812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":227,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-12-06T18:33:08.354763+0000","last_change":"2023-12-06T18:28:48.207134+0000","last_active":"2023-12-06T18:33:08.354763+0000","last_peered":"2023-12-06T18:33:08.354763+0000","last_clean":"2023-12-06T18:33:08.354763+0000","last_became_active":"2023-12-06T18:28:47.875258+0000","last_became_peered":"2023-12-06T18:28:47.875258+0000","last_unstale":"2023-12-06T18:33:08.354763+0000","last_undegraded":"2023-12-06T18:33:08.354763+0000","last_fullsized":"2023-12-06T18:33:08.354763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:14:32.667039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1204224,"data_stored":1193520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411406,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":753664,"data_stored":268960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":10.098000000000001,"5min":10.752000000000001,"15min":10.554},"min":{"1min":6.3730000000000002,"5min":3.835,"15min":3.835},"max":{"1min":25.571000000000002,"5min":108.57599999999999,"15min":108.57599999999999},"last":16.699999999999999},{"interface":"front","average":{"1min":11.791,"5min":11.002000000000001,"15min":12.173999999999999},"min":{"1min":4.8639999999999999,"5min":3.7200000000000002,"15min":3.7200000000000002},"max":{"1min":20.649999999999999,"5min":128.09399999999999,"15min":128.09399999999999},"last":6.6749999999999998}]},{"osd":1,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":13.631,"5min":12.202,"15min":12.771000000000001},"min":{"1min":7.6390000000000002,"5min":6.2539999999999996,"15min":6.2539999999999996},"max":{"1min":26.565999999999999,"5min":70.072000000000003,"15min":70.072000000000003},"last":13.115},{"interface":"front","average":{"1min":14.135999999999999,"5min":12.827999999999999,"15min":15.066000000000001},"min":{"1min":6.8280000000000003,"5min":6.444,"15min":6.444},"max":{"1min":26.420000000000002,"5min":27.934000000000001,"15min":32.920000000000002},"last":7.8310000000000004}]},{"osd":2,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":13.890000000000001,"5min":14.91,"15min":14.795},"min":{"1min":8.8390000000000004,"5min":7.7690000000000001,"15min":7.5149999999999997},"max":{"1min":24.039000000000001,"5min":75.125,"15min":75.125},"last":14.702},{"interface":"front","average":{"1min":15.166,"5min":15.824999999999999,"15min":17.459},"min":{"1min":7.5389999999999997,"5min":7.5389999999999997,"15min":7.5389999999999997},"max":{"1min":24.375,"5min":69.403999999999996,"15min":69.403999999999996},"last":15.686999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":16.661000000000001,"5min":13.574,"15min":12.426},"min":{"1min":10.935,"5min":5.8879999999999999,"15min":5.8879999999999999},"max":{"1min":27.975000000000001,"5min":27.975000000000001,"15min":27.975000000000001},"last":24.617000000000001},{"interface":"front","average":{"1min":10.597,"5min":13.162000000000001,"15min":15.118},"min":{"1min":7.29,"5min":6.3099999999999996,"15min":6.3099999999999996},"max":{"1min":23.651,"5min":71.718000000000004,"15min":71.718000000000004},"last":23.651}]},{"osd":5,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":18.983000000000001,"5min":18.795999999999999,"15min":18.07},"min":{"1min":13.872,"5min":9.798,"15min":8.3179999999999996},"max":{"1min":28.015999999999998,"5min":75.751999999999995,"15min":75.751999999999995},"last":14.090999999999999},{"interface":"front","average":{"1min":18.129000000000001,"5min":17.302,"15min":17.986999999999998},"min":{"1min":10.148,"5min":7.9269999999999996,"15min":7.9269999999999996},"max":{"1min":28.039999999999999,"5min":70.933000000000007,"15min":70.933000000000007},"last":10.935}]},{"osd":6,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":13.880000000000001,"5min":14.038,"15min":15.648999999999999},"min":{"1min":10.093,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":22.777000000000001,"5min":28.469999999999999,"15min":29.245000000000001},"last":17.359000000000002},{"interface":"front","average":{"1min":19.372,"5min":19.238,"15min":20.812000000000001},"min":{"1min":14.287000000000001,"5min":10.973000000000001,"15min":10.973000000000001},"max":{"1min":29.497,"5min":74.224000000000004,"15min":74.224000000000004},"last":21.928999999999998}]},{"osd":7,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":17.841000000000001,"5min":19.835999999999999,"15min":21.25},"min":{"1min":9.1859999999999999,"5min":9.1859999999999999,"15min":9.1859999999999999},"max":{"1min":33.747,"5min":122.93000000000001,"15min":122.93000000000001},"last":9.1859999999999999},{"interface":"front","average":{"1min":22.260000000000002,"5min":22.786999999999999,"15min":23.417999999999999},"min":{"1min":15.458,"5min":13.196,"15min":13.196},"max":{"1min":31.978999999999999,"5min":124.473,"15min":124.473},"last":18.728000000000002}]}]},{"osd":2,"up_from":16,"seq":68719476814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":765952,"data_stored":273340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:33:08 2023","interfaces":[{"interface":"back","average":{"1min":12.861000000000001,"5min":10.439,"15min":11.157999999999999},"min":{"1min":6.0350000000000001,"5min":4.1639999999999997,"15min":4.1639999999999997},"max":{"1min":29.417000000000002,"5min":39.220999999999997,"15min":39.220999999999997},"last":8.8829999999999991},{"interface":"front","average":{"1min":14.871,"5min":11.584,"15min":11.446},"min":{"1min":7.1440000000000001,"5min":4.3280000000000003,"15min":4.3280000000000003},"max":{"1min":31.372,"5min":44.691000000000003,"15min":44.691000000000003},"last":8.5869999999999997}]},{"osd":1,"last update":"Wed Dec 6 18:33:08 2023","interfaces":[{"interface":"back","average":{"1min":9.8409999999999993,"5min":11.188000000000001,"15min":11.483000000000001},"min":{"1min":7.2530000000000001,"5min":5.9139999999999997,"15min":5.9139999999999997},"max":{"1min":15.18,"5min":42.920999999999999,"15min":42.920999999999999},"last":7.6029999999999998},{"interface":"front","average":{"1min":13.111000000000001,"5min":13.525,"15min":13.849},"min":{"1min":7.1230000000000002,"5min":6.1479999999999997,"15min":6.1479999999999997},"max":{"1min":18.948,"5min":34.607999999999997,"15min":34.607999999999997},"last":11.492000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:33:08 2023","interfaces":[{"interface":"back","average":{"1min":15.618,"5min":14.375999999999999,"15min":18.050000000000001},"min":{"1min":9.5869999999999997,"5min":6.1260000000000003,"15min":6.1260000000000003},"max":{"1min":26.550999999999998,"5min":49.609000000000002,"15min":112.129},"last":10.817},{"interface":"front","average":{"1min":16.542999999999999,"5min":16.094000000000001,"15min":22.181999999999999},"min":{"1min":10.18,"5min":7.5510000000000002,"15min":7.5510000000000002},"max":{"1min":22.042999999999999,"5min":61.243000000000002,"15min":181.601},"last":17.654}]},{"osd":4,"last update":"Wed Dec 6 18:33:08 2023","interfaces":[{"interface":"back","average":{"1min":14.792999999999999,"5min":14.91,"15min":17.263999999999999},"min":{"1min":10.734,"5min":6.609,"15min":6.609},"max":{"1min":21.219999999999999,"5min":60.332000000000001,"15min":97.088999999999999},"last":17.010000000000002},{"interface":"front","average":{"1min":11.157,"5min":13.459,"15min":19.145},"min":{"1min":7.2990000000000004,"5min":6.0720000000000001,"15min":6.0720000000000001},"max":{"1min":16.617999999999999,"5min":43.198,"15min":98.069000000000003},"last":11.965}]},{"osd":5,"last update":"Wed Dec 6 18:33:08 2023","interfaces":[{"interface":"back","average":{"1min":17.873999999999999,"5min":17.538,"15min":18.042000000000002},"min":{"1min":10.359,"5min":7.8170000000000002,"15min":7.8170000000000002},"max":{"1min":39.686,"5min":57.628999999999998,"15min":57.628999999999998},"last":18.285},{"interface":"front","average":{"1min":15.845000000000001,"5min":15.821999999999999,"15min":15.834},"min":{"1min":9.8049999999999997,"5min":8.5990000000000002,"15min":8.5990000000000002},"max":{"1min":23.103000000000002,"5min":47.749000000000002,"15min":47.749000000000002},"last":13.007999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:33:08 2023","interfaces":[{"interface":"back","average":{"1min":18.725000000000001,"5min":18.283000000000001,"15min":37.774999999999999},"min":{"1min":11.263,"5min":8.7289999999999992,"15min":8.7289999999999992},"max":{"1min":31.329000000000001,"5min":64.034999999999997,"15min":498.38499999999999},"last":14.154},{"interface":"front","average":{"1min":20.073,"5min":19.332000000000001,"15min":39.524999999999999},"min":{"1min":10.803000000000001,"5min":9.7010000000000005,"15min":9.7010000000000005},"max":{"1min":30.225000000000001,"5min":54.920000000000002,"15min":496.17399999999998},"last":18.896999999999998}]},{"osd":7,"last update":"Wed Dec 6 18:33:14 2023","interfaces":[{"interface":"back","average":{"1min":19.437999999999999,"5min":18.989999999999998,"15min":18.945},"min":{"1min":11.891999999999999,"5min":10.064,"15min":10.064},"max":{"1min":29.789000000000001,"5min":49.456000000000003,"15min":49.456000000000003},"last":15.183999999999999},{"interface":"front","average":{"1min":20.567,"5min":20.460000000000001,"15min":20.257000000000001},"min":{"1min":14.108000000000001,"5min":12.288,"15min":12.288},"max":{"1min":32.514000000000003,"5min":40.656999999999996,"15min":40.656999999999996},"last":20.337}]}]},{"osd":1,"up_from":16,"seq":68719476815,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1167360,"data_stored":671180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:33:06 2023","interfaces":[{"interface":"back","average":{"1min":11.112,"5min":10.651,"15min":19.792000000000002},"min":{"1min":7.1459999999999999,"5min":5.0890000000000004,"15min":5.0890000000000004},"max":{"1min":25.581,"5min":34.286000000000001,"15min":238.87100000000001},"last":16.594999999999999},{"interface":"front","average":{"1min":13.031000000000001,"5min":10.532,"15min":37.002000000000002},"min":{"1min":6.452,"5min":4.4269999999999996,"15min":4.4269999999999996},"max":{"1min":61.262,"5min":61.262,"15min":661.33600000000001},"last":12.699999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:33:06 2023","interfaces":[{"interface":"back","average":{"1min":11.746,"5min":12.837,"15min":28.960999999999999},"min":{"1min":7.9390000000000001,"5min":6.6340000000000003,"15min":6.6340000000000003},"max":{"1min":21.129000000000001,"5min":93.168000000000006,"15min":384.315},"last":15.019},{"interface":"front","average":{"1min":17.257999999999999,"5min":17.582000000000001,"15min":41.338999999999999},"min":{"1min":12.568,"5min":8.109,"15min":5.7009999999999996},"max":{"1min":27.864999999999998,"5min":94.230000000000004,"15min":656.98699999999997},"last":18.960000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:33:06 2023","interfaces":[{"interface":"back","average":{"1min":12.555999999999999,"5min":11.630000000000001,"15min":12.943},"min":{"1min":7.8159999999999998,"5min":5.548,"15min":5.548},"max":{"1min":28.707000000000001,"5min":31.16,"15min":34.057000000000002},"last":14.093},{"interface":"front","average":{"1min":16.106000000000002,"5min":13.483000000000001,"15min":15.641999999999999},"min":{"1min":6.9160000000000004,"5min":6.5730000000000004,"15min":6.5730000000000004},"max":{"1min":65.537000000000006,"5min":65.537000000000006,"15min":65.537000000000006},"last":8.0180000000000007}]},{"osd":4,"last update":"Wed Dec 6 18:33:06 2023","interfaces":[{"interface":"back","average":{"1min":14.526,"5min":12.695,"15min":35.174999999999997},"min":{"1min":8.3130000000000006,"5min":6.2930000000000001,"15min":6.2930000000000001},"max":{"1min":33.201999999999998,"5min":33.201999999999998,"15min":447.67200000000003},"last":9.1579999999999995},{"interface":"front","average":{"1min":17.960999999999999,"5min":16.978000000000002,"15min":42.109999999999999},"min":{"1min":10.378,"5min":9.6129999999999995,"15min":7.0919999999999996},"max":{"1min":32.469999999999999,"5min":44.043999999999997,"15min":630.572},"last":18.033000000000001}]},{"osd":5,"last update":"Wed Dec 6 18:33:06 2023","interfaces":[{"interface":"back","average":{"1min":16.893999999999998,"5min":16.215,"15min":37.536999999999999},"min":{"1min":9.5530000000000008,"5min":7.0540000000000003,"15min":7.0540000000000003},"max":{"1min":68.013999999999996,"5min":68.013999999999996,"15min":389.55000000000001},"last":31.116},{"interface":"front","average":{"1min":19.353000000000002,"5min":16.658999999999999,"15min":48.807000000000002},"min":{"1min":11.243,"5min":7.0410000000000004,"15min":7.0410000000000004},"max":{"1min":27.831,"5min":37.744,"15min":658.48599999999999},"last":32.429000000000002}]},{"osd":6,"last update":"Wed Dec 6 18:33:06 2023","interfaces":[{"interface":"back","average":{"1min":17.175000000000001,"5min":14.875,"15min":31.696999999999999},"min":{"1min":8.3640000000000008,"5min":7.5899999999999999,"15min":7.5899999999999999},"max":{"1min":35.575000000000003,"5min":54.383000000000003,"15min":439.56400000000002},"last":10.303000000000001},{"interface":"front","average":{"1min":20.062000000000001,"5min":19.577000000000002,"15min":33.789000000000001},"min":{"1min":11.862,"5min":10.641999999999999,"15min":10.641999999999999},"max":{"1min":44.154000000000003,"5min":55.423999999999999,"15min":386.71499999999997},"last":19.872}]},{"osd":7,"last update":"Wed Dec 6 18:33:11 2023","interfaces":[{"interface":"back","average":{"1min":19.498999999999999,"5min":18.276,"15min":18.617999999999999},"min":{"1min":9.3680000000000003,"5min":9.3680000000000003,"15min":9.3680000000000003},"max":{"1min":62.582999999999998,"5min":97.561999999999998,"15min":97.561999999999998},"last":13.404},{"interface":"front","average":{"1min":26.725000000000001,"5min":19.928999999999998,"15min":18.407},"min":{"1min":14.022,"5min":8.7219999999999995,"15min":8.7219999999999995},"max":{"1min":101.762,"5min":101.762,"15min":101.762},"last":15.727}]}]},{"osd":0,"up_from":16,"seq":68719476816,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":765952,"data_stored":273340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":11.467000000000001,"5min":10.736000000000001,"15min":15.943},"min":{"1min":5.6749999999999998,"5min":3.2360000000000002,"15min":3.2360000000000002},"max":{"1min":40.502000000000002,"5min":47.661000000000001,"15min":116.593},"last":7.9980000000000002},{"interface":"front","average":{"1min":14.481,"5min":12.929,"15min":16.911999999999999},"min":{"1min":5.5709999999999997,"5min":4.9039999999999999,"15min":4.9039999999999999},"max":{"1min":42.502000000000002,"5min":50.127000000000002,"15min":117.896},"last":9.9969999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":15.561,"5min":14.813000000000001,"15min":13.863},"min":{"1min":6.726,"5min":5.694,"15min":5.694},"max":{"1min":25.344000000000001,"5min":43.752000000000002,"15min":43.752000000000002},"last":22.675999999999998},{"interface":"front","average":{"1min":14.154999999999999,"5min":13.552,"15min":11.832000000000001},"min":{"1min":6.5220000000000002,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":27.962,"5min":49.670000000000002,"15min":49.670000000000002},"last":10.492000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:33:11 2023","interfaces":[{"interface":"back","average":{"1min":18.148,"5min":14.635999999999999,"15min":16.108000000000001},"min":{"1min":11.849,"5min":6.1379999999999999,"15min":6.1379999999999999},"max":{"1min":38.896000000000001,"5min":38.896000000000001,"15min":46.689999999999998},"last":20.103999999999999},{"interface":"front","average":{"1min":14.398999999999999,"5min":15.494,"15min":16.018000000000001},"min":{"1min":7.5510000000000002,"5min":7.5510000000000002,"15min":7.5510000000000002},"max":{"1min":26.088999999999999,"5min":50.545999999999999,"15min":50.545999999999999},"last":18.545000000000002}]},{"osd":4,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":12.005000000000001,"5min":12.326000000000001,"15min":18.361999999999998},"min":{"1min":6.9699999999999998,"5min":5.9349999999999996,"15min":5.9349999999999996},"max":{"1min":16.777000000000001,"5min":43.920999999999999,"15min":82.432000000000002},"last":12.305999999999999},{"interface":"front","average":{"1min":18.808,"5min":18.149999999999999,"15min":19.920000000000002},"min":{"1min":12.162000000000001,"5min":9.5239999999999991,"15min":6.2629999999999999},"max":{"1min":28.553999999999998,"5min":61.402999999999999,"15min":81.239999999999995},"last":15.577999999999999}]},{"osd":5,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":18.818999999999999,"5min":16.928999999999998,"15min":17.396999999999998},"min":{"1min":8.8119999999999994,"5min":8.0099999999999998,"15min":8.0099999999999998},"max":{"1min":36.323,"5min":71.367000000000004,"15min":71.367000000000004},"last":14.58},{"interface":"front","average":{"1min":17.681000000000001,"5min":16.157,"15min":17.521000000000001},"min":{"1min":9.2650000000000006,"5min":7.1779999999999999,"15min":7.1779999999999999},"max":{"1min":36.862000000000002,"5min":72.307000000000002,"15min":72.307000000000002},"last":13.625999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":20.559999999999999,"5min":19.556999999999999,"15min":19.190999999999999},"min":{"1min":13.153,"5min":8.9179999999999993,"15min":7.7560000000000002},"max":{"1min":44.655999999999999,"5min":62.932000000000002,"15min":62.932000000000002},"last":16.693999999999999},{"interface":"front","average":{"1min":17.888000000000002,"5min":17.213000000000001,"15min":18.768000000000001},"min":{"1min":10.164,"5min":9.3520000000000003,"15min":9.3520000000000003},"max":{"1min":29.501000000000001,"5min":43.088000000000001,"15min":52.627000000000002},"last":21.399999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:33:11 2023","interfaces":[{"interface":"back","average":{"1min":21.228999999999999,"5min":20.321000000000002,"15min":19.521000000000001},"min":{"1min":16.484999999999999,"5min":12.009,"15min":12.009},"max":{"1min":32.023000000000003,"5min":58.021000000000001,"15min":58.021000000000001},"last":25.071999999999999},{"interface":"front","average":{"1min":19.852,"5min":18.920999999999999,"15min":18.238},"min":{"1min":11.643000000000001,"5min":11.515000000000001,"15min":11.515000000000001},"max":{"1min":36.466000000000001,"5min":40.978999999999999,"15min":40.978999999999999},"last":23.587}]}]},{"osd":5,"up_from":15,"seq":64424509520,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1167360,"data_stored":671180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":8.9949999999999992,"5min":8.4920000000000009,"15min":13.872},"min":{"1min":4.1669999999999998,"5min":3.3660000000000001,"15min":3.3660000000000001},"max":{"1min":23.98,"5min":23.98,"15min":208.291},"last":7.6630000000000003},{"interface":"front","average":{"1min":11.83,"5min":10.465999999999999,"15min":17.478000000000002},"min":{"1min":6.3940000000000001,"5min":4.4299999999999997,"15min":4.4299999999999997},"max":{"1min":24.289999999999999,"5min":24.289999999999999,"15min":199.94900000000001},"last":9.0120000000000005}]},{"osd":1,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":14.759,"5min":13.478999999999999,"15min":48.890999999999998},"min":{"1min":7.0339999999999998,"5min":5.085,"15min":5.085},"max":{"1min":22.251000000000001,"5min":41.234999999999999,"15min":1211.6010000000001},"last":10.711},{"interface":"front","average":{"1min":10.929,"5min":10.045999999999999,"15min":44.293999999999997},"min":{"1min":7.0060000000000002,"5min":4.7199999999999998,"15min":4.7199999999999998},"max":{"1min":17.417999999999999,"5min":39.802999999999997,"15min":1189.3869999999999},"last":5.6260000000000003}]},{"osd":2,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":16.795000000000002,"5min":14.260999999999999,"15min":22.454999999999998},"min":{"1min":7.9980000000000002,"5min":6.5129999999999999,"15min":6.5129999999999999},"max":{"1min":68.617999999999995,"5min":68.617999999999995,"15min":283.46800000000002},"last":17.718},{"interface":"front","average":{"1min":21.155999999999999,"5min":17.585000000000001,"15min":26.276},"min":{"1min":8.173,"5min":8.173,"15min":8.173},"max":{"1min":69.513999999999996,"5min":69.513999999999996,"15min":296.47800000000001},"last":18.468}]},{"osd":3,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":12.999000000000001,"5min":14.625999999999999,"15min":17.956},"min":{"1min":6.0300000000000002,"5min":6.0300000000000002,"15min":6.0300000000000002},"max":{"1min":36.156999999999996,"5min":36.156999999999996,"15min":47.926000000000002},"last":13.016},{"interface":"front","average":{"1min":17.372,"5min":13.675000000000001,"15min":15.026999999999999},"min":{"1min":11.651,"5min":5.3979999999999997,"15min":5.3979999999999997},"max":{"1min":34.609000000000002,"5min":34.609000000000002,"15min":66.176000000000002},"last":14.977}]},{"osd":4,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":19.289999999999999,"5min":17.210000000000001,"15min":19.231999999999999},"min":{"1min":11.885,"5min":9.7210000000000001,"15min":9.7210000000000001},"max":{"1min":41.923999999999999,"5min":41.923999999999999,"15min":67.248000000000005},"last":15.702},{"interface":"front","average":{"1min":11.738,"5min":10.41,"15min":12.5},"min":{"1min":6.4450000000000003,"5min":5.3300000000000001,"15min":5.3300000000000001},"max":{"1min":43.101999999999997,"5min":43.101999999999997,"15min":77.266999999999996},"last":12.242000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":14.673,"5min":12.750999999999999,"15min":15.19},"min":{"1min":7.4160000000000004,"5min":5.915,"15min":5.915},"max":{"1min":44.587000000000003,"5min":44.587000000000003,"15min":76.751999999999995},"last":11.630000000000001},{"interface":"front","average":{"1min":20.553000000000001,"5min":18.672000000000001,"15min":20.558},"min":{"1min":12.913,"5min":8.8390000000000004,"15min":8.8390000000000004},"max":{"1min":45.366,"5min":45.366,"15min":57.101999999999997},"last":16.952999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":17.077999999999999,"5min":15.398999999999999,"15min":14.606999999999999},"min":{"1min":10.875,"5min":7.5519999999999996,"15min":7.5519999999999996},"max":{"1min":29.731000000000002,"5min":29.965,"15min":29.965},"last":14.364000000000001},{"interface":"front","average":{"1min":13.878,"5min":12.369999999999999,"15min":12.048},"min":{"1min":9.3049999999999997,"5min":8.2829999999999995,"15min":8.2829999999999995},"max":{"1min":27.423999999999999,"5min":27.423999999999999,"15min":27.423999999999999},"last":9.6530000000000005}]}]},{"osd":4,"up_from":15,"seq":64424509521,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":268679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":9.5589999999999993,"5min":9.7349999999999994,"15min":12.707000000000001},"min":{"1min":6.0220000000000002,"5min":3.7429999999999999,"15min":3.7429999999999999},"max":{"1min":17.300999999999998,"5min":41.076000000000001,"15min":53.298999999999999},"last":6.7169999999999996},{"interface":"front","average":{"1min":10.759,"5min":9.9280000000000008,"15min":10.686},"min":{"1min":4.9930000000000003,"5min":4.8140000000000001,"15min":3.9670000000000001},"max":{"1min":29.567,"5min":35.863,"15min":52.567},"last":8.4030000000000005}]},{"osd":1,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":11.622,"5min":11.404999999999999,"15min":13.449999999999999},"min":{"1min":8.0660000000000007,"5min":5.8390000000000004,"15min":5.835},"max":{"1min":18.09,"5min":27.251999999999999,"15min":49.183},"last":21.186},{"interface":"front","average":{"1min":12.454000000000001,"5min":12.068,"15min":14.85},"min":{"1min":6.6849999999999996,"5min":5.96,"15min":5.96},"max":{"1min":19.509,"5min":26.495999999999999,"15min":45.412999999999997},"last":9.3260000000000005}]},{"osd":2,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":15.634,"5min":12.238,"15min":20.167000000000002},"min":{"1min":7.4009999999999998,"5min":5.6390000000000002,"15min":5.6390000000000002},"max":{"1min":35.258000000000003,"5min":44.649000000000001,"15min":85.394000000000005},"last":13.558},{"interface":"front","average":{"1min":17.91,"5min":14.865,"15min":19.396999999999998},"min":{"1min":7.5209999999999999,"5min":6.3639999999999999,"15min":6.3639999999999999},"max":{"1min":34.097999999999999,"5min":42.500999999999998,"15min":69.530000000000001},"last":10.269}]},{"osd":3,"last update":"Wed Dec 6 18:32:12 2023","interfaces":[{"interface":"back","average":{"1min":14.212999999999999,"5min":14.561999999999999,"15min":14.454000000000001},"min":{"1min":9.8339999999999996,"5min":7.008,"15min":7.008},"max":{"1min":18.713000000000001,"5min":26.219000000000001,"15min":26.219000000000001},"last":15.335000000000001},{"interface":"front","average":{"1min":12.509,"5min":13.157,"15min":14.773},"min":{"1min":7.4219999999999997,"5min":5.5549999999999997,"15min":5.5549999999999997},"max":{"1min":19.228000000000002,"5min":29.902999999999999,"15min":29.902999999999999},"last":18.782}]},{"osd":5,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":13.163,"5min":13.558,"15min":16.303000000000001},"min":{"1min":9.2739999999999991,"5min":5.4969999999999999,"15min":5.4969999999999999},"max":{"1min":20.664000000000001,"5min":53.840000000000003,"15min":78.798000000000002},"last":14.154999999999999},{"interface":"front","average":{"1min":18.699999999999999,"5min":17.393000000000001,"15min":18.565999999999999},"min":{"1min":11.209,"5min":8.0519999999999996,"15min":6.4450000000000003},"max":{"1min":27.582000000000001,"5min":52.781999999999996,"15min":55.814999999999998},"last":11.164999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":19.576000000000001,"5min":17.699000000000002,"15min":17.986999999999998},"min":{"1min":9.2100000000000009,"5min":8.2300000000000004,"15min":7.7850000000000001},"max":{"1min":41.723999999999997,"5min":51.149000000000001,"15min":51.149000000000001},"last":16.244},{"interface":"front","average":{"1min":15.221,"5min":14.712999999999999,"15min":16.190999999999999},"min":{"1min":9.6809999999999992,"5min":8.5139999999999993,"15min":7.8700000000000001},"max":{"1min":27.405000000000001,"5min":40.426000000000002,"15min":40.426000000000002},"last":20}]},{"osd":7,"last update":"Wed Dec 6 18:32:12 2023","interfaces":[{"interface":"back","average":{"1min":15.081,"5min":18.016999999999999,"15min":18.506},"min":{"1min":8.8800000000000008,"5min":8.8800000000000008,"15min":8.8800000000000008},"max":{"1min":24.949000000000002,"5min":60.390000000000001,"15min":60.390000000000001},"last":12.066000000000001},{"interface":"front","average":{"1min":18.972999999999999,"5min":22.041,"15min":22.553000000000001},"min":{"1min":13.69,"5min":13.69,"15min":13.69},"max":{"1min":40.625999999999998,"5min":57.606999999999999,"15min":57.606999999999999},"last":17.129000000000001}]}]},{"osd":7,"up_from":61,"seq":261993005095,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32940,"kb_used_data":736,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94338900,"statfs":{"total":96636764160,"available":96603033600,"internally_reserved":0,"allocated":753664,"data_stored":268960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4395,"internal_metadata":28634837},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":9.8659999999999997,"5min":10.045999999999999,"15min":10.109},"min":{"1min":4.4359999999999999,"5min":4.1059999999999999,"15min":4.1059999999999999},"max":{"1min":23.795000000000002,"5min":37.481999999999999,"15min":37.481999999999999},"last":9.6929999999999996},{"interface":"front","average":{"1min":10.321999999999999,"5min":10.752000000000001,"15min":10.927},"min":{"1min":5.4909999999999997,"5min":4.2530000000000001,"15min":4.2530000000000001},"max":{"1min":29.928999999999998,"5min":38.475999999999999,"15min":38.475999999999999},"last":10.311}]},{"osd":1,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":12.515000000000001,"5min":14.898999999999999,"15min":16.606000000000002},"min":{"1min":4.6929999999999996,"5min":4.6929999999999996,"15min":4.6929999999999996},"max":{"1min":30.350000000000001,"5min":64.765000000000001,"15min":64.765000000000001},"last":10.739000000000001},{"interface":"front","average":{"1min":15.795,"5min":16.404,"15min":17.541},"min":{"1min":8.0419999999999998,"5min":7.0739999999999998,"15min":7.0739999999999998},"max":{"1min":34.390000000000001,"5min":65.994,"15min":65.994},"last":12.693}]},{"osd":2,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":16.215,"5min":16.803000000000001,"15min":17.611999999999998},"min":{"1min":8.734,"5min":7.9909999999999997,"15min":7.9909999999999997},"max":{"1min":36.622,"5min":46.478000000000002,"15min":46.478000000000002},"last":11.401},{"interface":"front","average":{"1min":14.967000000000001,"5min":14.003,"15min":14.08},"min":{"1min":6.5910000000000002,"5min":6.5910000000000002,"15min":6.5910000000000002},"max":{"1min":37.026000000000003,"5min":41.603999999999999,"15min":41.603999999999999},"last":15.18}]},{"osd":3,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":13.579000000000001,"5min":15.653,"15min":16.725000000000001},"min":{"1min":7.4950000000000001,"5min":6.8319999999999999,"15min":6.8319999999999999},"max":{"1min":45.731000000000002,"5min":60.463999999999999,"15min":60.463999999999999},"last":13.539999999999999},{"interface":"front","average":{"1min":18.077000000000002,"5min":18.809000000000001,"15min":19.835000000000001},"min":{"1min":10.327,"5min":9.7370000000000001,"15min":9.7370000000000001},"max":{"1min":47.265999999999998,"5min":56.286000000000001,"15min":56.286000000000001},"last":14.119}]},{"osd":4,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":19.510999999999999,"5min":20.824999999999999,"15min":21.161000000000001},"min":{"1min":9.6180000000000003,"5min":8.4960000000000004,"15min":8.4960000000000004},"max":{"1min":50.741999999999997,"5min":57.716999999999999,"15min":57.716999999999999},"last":16.129000000000001},{"interface":"front","average":{"1min":15.906000000000001,"5min":18.274000000000001,"15min":18.507000000000001},"min":{"1min":7.3840000000000003,"5min":7.3840000000000003,"15min":7.3840000000000003},"max":{"1min":46.180999999999997,"5min":67.159000000000006,"15min":67.159000000000006},"last":16.954999999999998}]},{"osd":5,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":14.403,"5min":14.342000000000001,"15min":14.606},"min":{"1min":7.7510000000000003,"5min":6.3399999999999999,"15min":6.3399999999999999},"max":{"1min":44.884,"5min":44.884,"15min":44.884},"last":17.625},{"interface":"front","average":{"1min":16.992999999999999,"5min":18.634,"15min":19.498000000000001},"min":{"1min":10.260999999999999,"5min":10.111000000000001,"15min":10.111000000000001},"max":{"1min":31.847999999999999,"5min":70.900000000000006,"15min":70.900000000000006},"last":18.387}]},{"osd":6,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":17.672000000000001,"5min":16.728999999999999,"15min":16.760000000000002},"min":{"1min":11.443,"5min":8.7170000000000005,"15min":8.7170000000000005},"max":{"1min":41.091000000000001,"5min":54.393000000000001,"15min":54.393000000000001},"last":20.126000000000001},{"interface":"front","average":{"1min":18.923999999999999,"5min":20.111000000000001,"15min":20.655999999999999},"min":{"1min":10.144,"5min":10.144,"15min":10.144},"max":{"1min":42.084000000000003,"5min":69.263000000000005,"15min":69.263000000000005},"last":19.548999999999999}]}]},{"osd":6,"up_from":0,"seq":60129542222,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:33:17.550 INFO:tasks.thrashosds.thrasher:Setting 1.0 to [2, 7, 1, 0] 2023-12-06T18:33:17.550 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '1.0', '2', '7', '1', '0'] 2023-12-06T18:33:17.550 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 1.0 2 7 1 0 2023-12-06T18:33:17.609 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:17.605+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:17.630 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:17.630 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:17.630 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:17.631 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:17.637 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:33:17.710 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:17.706+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:17.786 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:17.786 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:17.786 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:17.786 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:17.786 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:17.795 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:33:17.839 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:17.835+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:17.912 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:17.908+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:18.013 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:18.009+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:18.014 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:18.014 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:18.014 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:18.014 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:18.014 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.014 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:33:18.015 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:18.015 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 382.582723477, 2023-12-06T18:33:18.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:33:18.015 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.015 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.015 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.015 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.015 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.015 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:33:18.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.015 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:18.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:18.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:33:18.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:33:18.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:33:18.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:33:18.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:33:18.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:33:18.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:33:18.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:33:18.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:33:18.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:33:18.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:33:18.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:33:18.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:33:18.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:33:18.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:33:18.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:33:18.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:33:18.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:33:18.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:33:18.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:33:18.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:33:18.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:33:18.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:33:18.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:33:18.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:33:18.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:33:18.021 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:33:18.021 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.021 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.021 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.021 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.021 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.021 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:18.021 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:18.021 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 380.47415023799999, 2023-12-06T18:33:18.021 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:33:18.021 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.021 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.021 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.021 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.021 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.022 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:33:18.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.022 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:18.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:18.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:33:18.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:33:18.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:33:18.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:33:18.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:33:18.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:33:18.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:33:18.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:33:18.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:33:18.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:33:18.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:33:18.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:33:18.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:33:18.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:33:18.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:33:18.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:33:18.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:33:18.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:33:18.026 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.026 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.026 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.026 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:33:18.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:33:18.026 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.026 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.026 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.026 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:33:18.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:33:18.026 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.026 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.026 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.027 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:18.027 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:18.027 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 380.47340500600001, 2023-12-06T18:33:18.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:33:18.027 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.027 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.027 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.027 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.028 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.028 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:33:18.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.028 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:18.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:18.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:33:18.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:33:18.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:33:18.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:33:18.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:33:18.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:33:18.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:33:18.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:33:18.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:33:18.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:33:18.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:33:18.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:33:18.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:33:18.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:33:18.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:33:18.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:33:18.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:33:18.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:33:18.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:33:18.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:33:18.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:33:18.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:33:18.032 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.033 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.033 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.033 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:18.033 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:18.033 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 380.472721251, 2023-12-06T18:33:18.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:33:18.033 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.033 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.034 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.034 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.034 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.034 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:33:18.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.034 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.034 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:18.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:18.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:33:18.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:33:18.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:33:18.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:33:18.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:33:18.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:33:18.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:33:18.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:33:18.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:33:18.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:33:18.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.041 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.041 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:33:18.041 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:33:18.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.041 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.042 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:33:18.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:33:18.042 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.042 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.042 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.042 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:33:18.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:33:18.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:33:18.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:33:18.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:33:18.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:33:18.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:33:18.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:33:18.044 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.044 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.044 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.045 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:18.045 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:18.045 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 380.46714799599999, 2023-12-06T18:33:18.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:33:18.045 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.045 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.045 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.045 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.045 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.045 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:33:18.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.045 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.046 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.046 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:18.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.046 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.046 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:18.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.046 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.046 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:33:18.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:33:18.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:33:18.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:33:18.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:33:18.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:33:18.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:33:18.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:33:18.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:33:18.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:33:18.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:33:18.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:33:18.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:33:18.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:33:18.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:33:18.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:33:18.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:33:18.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:33:18.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:33:18.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:33:18.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:33:18.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:33:18.050 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.050 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.051 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.051 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:18.051 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:18.051 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 380.46647435599999, 2023-12-06T18:33:18.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:33:18.051 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.051 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.051 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.051 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.051 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.051 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:33:18.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.051 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:18.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:18.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:33:18.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:33:18.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:33:18.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:33:18.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:33:18.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:33:18.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.054 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:33:18.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:33:18.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.055 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.055 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:33:18.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:33:18.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.055 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.055 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:33:18.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:33:18.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:33:18.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:33:18.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:33:18.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:33:18.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:33:18.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:33:18.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:33:18.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:33:18.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:33:18.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:33:18.059 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.059 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.059 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.060 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:18.060 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:18.060 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 380.46581215700002, 2023-12-06T18:33:18.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:33:18.060 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.060 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.060 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.060 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.060 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.061 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:33:18.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.061 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.061 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.061 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:18.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:18.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:33:18.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:33:18.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:33:18.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:33:18.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:33:18.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:33:18.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:33:18.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:33:18.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:33:18.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:33:18.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:33:18.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:33:18.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:33:18.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:33:18.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:33:18.066 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:33:18.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.066 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.066 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.066 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:33:18.066 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:33:18.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.066 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.066 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.066 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:33:18.066 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:33:18.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.066 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.067 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.067 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:33:18.067 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:33:18.067 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.067 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.067 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.067 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.067 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.067 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:18.067 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:18.067 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 380.39310272199998, 2023-12-06T18:33:18.067 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:33:18.067 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.067 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.067 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.067 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.067 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.067 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:33:18.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.068 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:18.068 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:18.068 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:33:18.068 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:33:18.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:33:18.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:33:18.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:33:18.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:33:18.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:33:18.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:33:18.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.070 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.070 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:33:18.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:33:18.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.070 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.070 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:33:18.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:33:18.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.070 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.070 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:33:18.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:33:18.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.071 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.071 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:33:18.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:33:18.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.071 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.071 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:33:18.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:33:18.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.071 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.071 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:33:18.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:33:18.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.071 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.071 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:33:18.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:33:18.072 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.072 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.072 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.072 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.072 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:18.072 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:18.072 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 380.39242101299999, 2023-12-06T18:33:18.072 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:33:18.072 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.072 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.072 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.072 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.072 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.072 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:33:18.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.073 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:18.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:18.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:33:18.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:33:18.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.074 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:33:18.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:33:18.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.074 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:33:18.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:33:18.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.074 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:33:18.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:33:18.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.075 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:33:18.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:33:18.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.075 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:33:18.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:33:18.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.075 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:33:18.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:33:18.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.076 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:33:18.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:33:18.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.076 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:33:18.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:33:18.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.076 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:33:18.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:33:18.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.077 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:33:18.077 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:33:18.077 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.077 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.077 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.077 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.077 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.077 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:18.077 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:18.077 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 379.31361692500002, 2023-12-06T18:33:18.077 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:33:18.077 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.077 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.077 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.077 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.077 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.077 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:33:18.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.078 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.078 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.078 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:18.078 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.078 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.078 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:18.078 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.078 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:33:18.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:33:18.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:33:18.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:33:18.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:33:18.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:33:18.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:33:18.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:33:18.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:33:18.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:33:18.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:33:18.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:33:18.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:33:18.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:33:18.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.081 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:33:18.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:33:18.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.081 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:33:18.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:33:18.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.081 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:33:18.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:33:18.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.082 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.082 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:33:18.082 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:33:18.082 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.082 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.082 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.082 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.082 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.082 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:18.082 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:18.082 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 379.312200351, 2023-12-06T18:33:18.082 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:33:18.082 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.082 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.083 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.083 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.083 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.083 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:33:18.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.083 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.083 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.083 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.083 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:18.083 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.083 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.083 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.083 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:18.083 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:33:18.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:33:18.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:33:18.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:33:18.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:33:18.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:33:18.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:33:18.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:33:18.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:33:18.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:33:18.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:33:18.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:33:18.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:33:18.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:33:18.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:33:18.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:33:18.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:33:18.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:33:18.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.087 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:33:18.087 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:33:18.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.087 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:33:18.087 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:33:18.087 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.087 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.087 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.088 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.088 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:18.088 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:18.088 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 379.31111522600003, 2023-12-06T18:33:18.088 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:33:18.088 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.088 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.088 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.088 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.088 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.088 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:33:18.088 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.088 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.088 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.088 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.088 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:18.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:18.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:33:18.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:33:18.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:33:18.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:33:18.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:33:18.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:33:18.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:33:18.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:33:18.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:33:18.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:33:18.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:33:18.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:33:18.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:33:18.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:33:18.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:33:18.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:33:18.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:33:18.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:33:18.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:33:18.093 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:33:18.093 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.093 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.093 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.093 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:33:18.093 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:33:18.093 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.094 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.094 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.094 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.094 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.094 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:18.094 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:18.095 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 379.27596612399998, 2023-12-06T18:33:18.095 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:33:18.095 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.095 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.095 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.095 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.095 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.095 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:33:18.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.096 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.096 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.096 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:18.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.097 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.097 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:18.097 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.097 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.098 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:33:18.098 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:33:18.098 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.098 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.098 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.098 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:33:18.098 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:33:18.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.099 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.099 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.099 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:33:18.099 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:33:18.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.100 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:33:18.100 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:33:18.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:33:18.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:33:18.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:33:18.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:33:18.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:33:18.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:33:18.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.103 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.103 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.103 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:33:18.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:33:18.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.103 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.103 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.104 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:33:18.104 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:33:18.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.104 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.104 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:33:18.104 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:33:18.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.105 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.105 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:33:18.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:33:18.105 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.105 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.106 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.106 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:18.106 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:18.106 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 372.72623559900001, 2023-12-06T18:33:18.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:33:18.107 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.107 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.107 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.107 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.107 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.107 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:33:18.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.108 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.108 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.108 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:18.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.109 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.109 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.109 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:18.109 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.109 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.109 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.110 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:33:18.110 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:33:18.110 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.110 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.110 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.110 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:33:18.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:33:18.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:33:18.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:33:18.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:33:18.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:33:18.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:33:18.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:33:18.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:33:18.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:33:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:33:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:33:18.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:33:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:33:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:33:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:33:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:33:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:33:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:33:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:33:18.117 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 372.725429991, 2023-12-06T18:33:18.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:33:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:33:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:33:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:33:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:33:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:33:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:33:18.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:33:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:33:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:33:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:33:18.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:33:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:33:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:33:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:33:18.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:33:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:33:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:33:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:33:18.124 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:33:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:33:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:33:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.125 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:33:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:33:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 372.72473918200001, 2023-12-06T18:33:18.126 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:33:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:33:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.128 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:33:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:33:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:33:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:33:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.129 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:33:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:33:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.130 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:33:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:33:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.131 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:33:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:33:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:33:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:33:18.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:33:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:33:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:33:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:33:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:33:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:33:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:33:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:33:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:33:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:33:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.136 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 372.72044167400003, 2023-12-06T18:33:18.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:33:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:33:18.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:33:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:33:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:33:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:33:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.142 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:33:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:33:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:33:18.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:33:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:33:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:33:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:33:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:33:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:33:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:33:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:33:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:33:18.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:33:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:33:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:33:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:33:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:33:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:33:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 372.71977399899998, 2023-12-06T18:33:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:33:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.150 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:33:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:33:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:33:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:33:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:33:18.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:33:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:33:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:33:18.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:33:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:33:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:33:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:33:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:33:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:33:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:33:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:33:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:33:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:33:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:33:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:33:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:33:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:33:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:33:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 372.71911857100002, 2023-12-06T18:33:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:33:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.158 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:33:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.159 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:33:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:33:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:33:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:33:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.160 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:33:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:33:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:33:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:33:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:33:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:33:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.161 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:33:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:33:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:33:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:33:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:33:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:33:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.162 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:33:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:33:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:33:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:33:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:33:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:33:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.163 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 372.67846402399999, 2023-12-06T18:33:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:33:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:33:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:18.164 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:33:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:33:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:33:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:33:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.165 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:33:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:33:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:33:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:33:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:33:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:33:18.166 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:33:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:33:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:33:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:33:18.167 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:33:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:33:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:18.168 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:33:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:33:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:33:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:33:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:18.169 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:33:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:33:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.170 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:18.171 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:18.171 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:18.173 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:33:18.174 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is unset 2023-12-06T18:33:18.174 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:18.169+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:18.203 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:33:18.274 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:18.270+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:18.374 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:18.374 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:18.374 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:18.374 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:18.375 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:18.371+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:18.384 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:33:18.476 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:18.472+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:18.486 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:33:18.487 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:33:18.576 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:18.573+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:18.585 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:18.585 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:18.585 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:18.585 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:18.585 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:18.594 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:33:18.605 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:33:18.605 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:33:18.677 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:18.673+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:18.698 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:33:18.698 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:33:18.739 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:18.739 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:18.750 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:33:18.778 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:18.774+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:18.804 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:33:18.804 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:33:18.879 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:18.875+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:18.890 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:33:18.890 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:33:18.929 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:18.929 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:18.939 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:33:18.975 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:33:18.975 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:33:19.003 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:19.000+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:19.062 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:33:19.063 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:33:19.081 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:19.078+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:19.115 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:19.115 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:19.125 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:33:19.143 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '18 of 30' 2023-12-06T18:33:19.170 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:33:19.170 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:33:19.182 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:19.179+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:19.281 INFO:teuthology.orchestra.run.smithi061.stderr:set 1.0 pg_upmap_items mapping to [2->7,1->0] 2023-12-06T18:33:19.283 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:19.279+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:19.325 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:19.325 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:19.335 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:33:19.385 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:19.381+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:19.484 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:19.484 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:19.485 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:19.482+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:19.493 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:33:19.586 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:19.583+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:19.666 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:19.666 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:19.675 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:33:19.720 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:19.716+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:19.789 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:19.785+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:19.803 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:19.803 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:19.812 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:33:19.890 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:19.886+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:19.984 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:19.984 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:19.990 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:19.987+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:19.994 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:33:20.093 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:20.088+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:20.148 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '19 of 30' 2023-12-06T18:33:20.168 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:20.168 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:20.177 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:33:20.187 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-12-06T18:33:20.219 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:20.215+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:20.292 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:20.289+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:20.357 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:20.357 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:20.369 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:33:20.410 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:20.407+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:20.493 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:20.490+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:20.556 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:20.557 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:20.567 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:33:20.594 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:20.590+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:20.695 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:20.692+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:20.746 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:20.746 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:20.755 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:33:20.799 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:20.795+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:20.885 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:20.885 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:20.894 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:33:20.896 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:20.892+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:20.998 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:20.995+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:21.064 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:21.064 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:21.073 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:33:21.117 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:21.114+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:21.119 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '20 of 30' 2023-12-06T18:33:21.187 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:33:21.205 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:21.205 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:21.206 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:21.201+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:21.216 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:33:21.230 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is unset 2023-12-06T18:33:21.299 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:21.296+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:21.398 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:21.398 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:21.407 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:33:21.416 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:21.412+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:21.502 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:21.498+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:21.603 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:21.600+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:21.711 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:21.706+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:21.733 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:33:21.734 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:33:21.739 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:21.740 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:21.740 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:21.740 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:21.750 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:33:21.828 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:33:21.829 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:33:21.906 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:21.902+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:21.925 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:33:21.925 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:33:22.007 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:22.004+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:22.036 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:33:22.036 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:33:22.068 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:22.065+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:22.108 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:22.105+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:22.141 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:33:22.141 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:33:22.166 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:22.166 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:22.166 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:22.166 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:22.166 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:22.176 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:33:22.230 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:33:22.231 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:33:22.235 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:22.232+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:22.248 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '21 of 30' 2023-12-06T18:33:22.309 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:22.306+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:22.317 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:33:22.318 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:33:22.378 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:22.378 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:22.378 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:22.378 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:22.378 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.378 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 110/109 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 23'75, data_included: [0~4096,65536~66064], data_size: 70160, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@23'75, size: 131600, copy_subset: [(0, 131600)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 131600, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:33:22.378 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:33:21.824095+0000", 2023-12-06T18:33:22.379 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 0.53739623800000003, 2023-12-06T18:33:22.379 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.22522905200000001, 2023-12-06T18:33:22.379 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:22.379 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:33:22.379 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:22.379 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.379 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:22.379 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.824095+0000", 2023-12-06T18:33:22.379 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.379 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.379 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.379 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:22.379 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.824095+0000", 2023-12-06T18:33:22.379 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.379 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.380 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.380 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:22.380 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.824111+0000", 2023-12-06T18:33:22.380 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5596000000000001e-05 2023-12-06T18:33:22.380 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.380 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.380 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:22.380 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.866577+0000", 2023-12-06T18:33:22.380 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.042466773999999999 2023-12-06T18:33:22.380 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.380 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.380 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:22.380 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.871496+0000", 2023-12-06T18:33:22.380 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049188699999999997 2023-12-06T18:33:22.380 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.380 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.381 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:22.381 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.886281+0000", 2023-12-06T18:33:22.381 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.014784535 2023-12-06T18:33:22.381 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.381 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.381 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:22.381 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.903661+0000", 2023-12-06T18:33:22.381 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.017379950000000002 2023-12-06T18:33:22.381 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.381 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.381 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:22.381 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.908970+0000", 2023-12-06T18:33:22.381 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.005309322 2023-12-06T18:33:22.381 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.381 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.381 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:22.382 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.049324+0000", 2023-12-06T18:33:22.382 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.140354005 2023-12-06T18:33:22.382 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.382 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:22.382 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.382 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.382 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.382 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 110/109 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:33:22.382 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:33:22.129048+0000", 2023-12-06T18:33:22.382 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 0.23244305200000001, 2023-12-06T18:33:22.382 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.029478353999999998, 2023-12-06T18:33:22.382 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:22.382 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:33:22.382 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:22.383 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.383 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:22.383 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.129048+0000", 2023-12-06T18:33:22.383 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.383 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.383 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.383 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:22.383 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.129048+0000", 2023-12-06T18:33:22.383 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.383 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.383 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.383 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:22.383 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.129058+0000", 2023-12-06T18:33:22.383 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 9.8630000000000003e-06 2023-12-06T18:33:22.383 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.383 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.384 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:22.384 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.133519+0000", 2023-12-06T18:33:22.384 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0044613229999999997 2023-12-06T18:33:22.384 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.384 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.384 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:22.384 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.133935+0000", 2023-12-06T18:33:22.384 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00041585999999999999 2023-12-06T18:33:22.384 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.384 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.384 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:22.384 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.134363+0000", 2023-12-06T18:33:22.384 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00042782200000000001 2023-12-06T18:33:22.384 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.384 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.385 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:22.385 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.136423+0000", 2023-12-06T18:33:22.385 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002059759 2023-12-06T18:33:22.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.385 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.385 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:22.385 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.136759+0000", 2023-12-06T18:33:22.385 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000335898 2023-12-06T18:33:22.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.385 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.385 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:22.385 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.158527+0000", 2023-12-06T18:33:22.385 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021767828999999999 2023-12-06T18:33:22.385 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.385 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:22.386 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.386 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.386 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:22.386 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:22.387 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:33:22.405 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:33:22.406 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:33:22.410 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:22.407+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:22.511 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:22.509+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:22.543 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:22.544 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:22.544 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:22.544 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:22.553 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:33:22.612 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:22.609+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:22.713 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:22.710+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:22.736 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:22.736 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:22.736 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:22.736 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:22.736 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:22.746 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:33:22.815 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:22.811+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:22.916 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:22.913+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:22.943 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:22.943 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:22.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:22.943 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:22.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.943 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:33:22.943 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:22.943 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 387.53082962299999, 2023-12-06T18:33:22.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:33:22.943 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:22.943 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:22.944 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:22.944 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:22.944 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:22.944 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:33:22.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.944 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:22.944 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.944 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:22.944 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:22.944 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.944 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.944 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:22.944 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:22.944 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.945 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.945 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:22.945 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:33:22.945 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:33:22.945 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.945 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.945 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:22.945 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:33:22.945 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:33:22.945 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.945 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.945 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:22.945 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:33:22.945 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:33:22.945 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.945 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.945 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:22.945 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:33:22.946 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:33:22.946 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.946 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.946 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:22.946 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:33:22.946 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:33:22.946 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.946 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.946 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:33:22.946 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:33:22.946 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:33:22.946 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.946 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.946 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:22.946 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:33:22.946 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:33:22.947 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.947 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.947 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:22.947 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:33:22.947 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:33:22.947 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.947 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.947 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:22.947 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:33:22.947 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:33:22.947 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.947 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.947 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:22.947 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:33:22.947 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:33:22.947 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.948 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.948 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:22.948 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:33:22.948 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:33:22.948 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.948 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.948 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:22.948 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:33:22.948 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:33:22.948 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.948 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.948 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:22.948 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:33:22.948 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:33:22.948 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.948 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:22.949 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.949 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.949 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.949 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:22.949 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:22.949 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 385.42225638399998, 2023-12-06T18:33:22.949 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:33:22.949 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:22.949 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:22.949 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:22.949 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:22.949 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:22.949 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:33:22.949 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.949 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:22.949 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.949 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:22.949 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:22.950 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.950 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.950 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.950 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:22.950 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:22.950 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.950 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.950 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.950 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:22.950 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:33:22.950 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:33:22.950 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.950 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.950 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:22.950 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:33:22.951 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:33:22.951 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.951 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.951 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:22.951 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:33:22.951 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:33:22.951 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.951 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.951 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:22.951 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:33:22.951 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:33:22.951 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.951 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.951 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:22.951 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:33:22.952 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:33:22.952 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.952 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.952 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:22.952 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:33:22.952 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:33:22.952 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.952 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.952 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:22.952 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:33:22.953 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:33:22.953 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.953 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.953 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:22.953 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:33:22.953 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:33:22.953 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.953 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.953 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:22.953 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:33:22.953 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:33:22.953 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.953 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.953 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:22.953 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:33:22.953 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:33:22.954 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.954 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.954 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:22.954 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:33:22.954 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:33:22.954 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.954 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:22.954 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.954 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.954 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.954 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:22.954 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:22.954 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 385.42151115199999, 2023-12-06T18:33:22.954 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:33:22.954 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:22.954 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:22.955 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:22.955 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:22.955 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:22.955 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:33:22.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.955 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:22.955 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.955 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:22.955 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:22.955 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.955 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.956 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:22.956 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:22.956 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.956 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.956 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.956 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:22.956 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:33:22.956 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:33:22.956 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.956 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.956 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:22.956 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:33:22.956 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:33:22.956 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.956 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.956 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:22.957 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:33:22.957 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:33:22.957 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.957 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.957 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:22.957 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:33:22.957 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:33:22.957 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.957 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.957 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:22.957 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:33:22.957 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:33:22.957 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.957 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.958 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:22.958 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:33:22.958 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:33:22.958 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.958 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.958 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:22.958 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:33:22.958 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:33:22.958 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.958 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.958 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:22.958 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:33:22.958 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:33:22.958 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.958 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.958 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:22.959 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:33:22.959 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:33:22.959 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.959 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.959 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:22.959 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:33:22.959 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:33:22.959 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.959 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.959 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:22.959 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:33:22.959 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:33:22.959 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.959 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:22.959 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.959 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.959 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.960 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:22.960 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:22.960 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 385.42082739699998, 2023-12-06T18:33:22.960 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:33:22.960 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:22.960 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:22.960 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:22.960 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:22.960 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:22.960 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:33:22.960 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.960 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:22.960 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.960 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:22.960 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:22.961 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.961 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.961 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.961 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:22.961 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:22.961 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.961 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.961 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.961 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:22.961 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:33:22.961 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:33:22.961 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.961 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.961 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:22.961 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:33:22.962 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:33:22.962 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.962 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.962 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:22.962 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:33:22.962 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:33:22.962 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.962 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.962 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:22.962 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:33:22.962 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:33:22.962 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.962 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.962 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:22.963 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:33:22.963 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:33:22.963 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.963 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.963 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:22.963 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:33:22.963 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:33:22.963 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.963 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.963 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:22.963 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:33:22.963 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:33:22.963 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.963 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.963 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:22.963 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:33:22.963 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:33:22.964 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.964 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.964 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:22.964 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:33:22.964 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:33:22.964 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.964 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.964 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:22.964 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:33:22.964 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:33:22.964 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.964 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.964 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:22.964 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:33:22.964 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:33:22.964 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.964 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:22.965 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.965 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.965 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.965 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:22.965 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:22.965 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 385.41525414199998, 2023-12-06T18:33:22.965 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:33:22.965 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:22.965 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:22.965 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:22.965 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:22.965 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:22.965 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:33:22.965 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.966 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:22.966 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.966 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:22.966 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:22.966 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.966 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.966 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:22.966 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:22.966 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.966 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.966 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:22.967 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:33:22.967 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:33:22.967 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.967 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.967 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:22.967 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:33:22.967 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:33:22.967 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.967 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.967 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:22.967 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:33:22.967 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:33:22.967 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.967 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.967 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:22.967 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:33:22.968 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:33:22.968 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.968 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.968 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:22.968 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:33:22.968 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:33:22.968 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.968 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.968 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:22.968 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:33:22.968 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:33:22.968 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.968 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.968 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:22.968 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:33:22.969 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:33:22.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.969 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.969 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:22.969 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:33:22.969 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:33:22.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.969 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.969 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:22.969 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:33:22.969 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:33:22.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.969 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.969 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:22.970 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:33:22.970 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:33:22.970 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.970 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.970 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:22.970 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:33:22.970 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:33:22.970 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.970 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:22.970 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.970 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.970 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.970 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:22.971 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:22.971 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 385.41458050199998, 2023-12-06T18:33:22.971 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:33:22.971 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:22.971 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:22.971 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:22.971 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:22.971 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:22.971 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:33:22.971 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.971 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:22.971 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.971 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:22.971 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:22.971 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.971 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.972 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.972 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:22.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:22.972 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.972 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.972 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.972 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:22.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:33:22.972 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:33:22.972 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.972 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.972 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:22.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:33:22.972 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:33:22.972 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.972 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.972 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:22.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:33:22.973 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:33:22.973 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.973 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.973 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:22.973 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:33:22.973 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:33:22.973 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.973 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.973 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:22.973 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:33:22.973 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:33:22.973 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.973 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.973 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:22.973 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:33:22.973 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:33:22.973 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.974 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.974 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:22.974 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:33:22.974 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:33:22.974 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.974 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.974 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:22.974 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:33:22.974 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:33:22.974 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.974 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.974 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:22.974 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:33:22.974 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:33:22.974 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.975 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.975 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:22.975 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:33:22.975 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:33:22.975 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.975 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.975 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:22.975 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:33:22.975 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:33:22.975 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.975 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:22.975 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.975 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.975 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.975 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:22.975 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:22.976 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 385.413918303, 2023-12-06T18:33:22.976 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:33:22.976 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:22.976 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:22.976 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:22.976 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:22.976 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:22.976 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:33:22.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.977 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:22.977 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.977 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:22.977 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:22.977 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.977 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.977 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:22.977 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:22.978 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.978 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.978 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.978 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:22.978 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:33:22.978 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:33:22.978 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.978 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.978 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:22.978 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:33:22.978 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:33:22.978 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.978 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.978 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:22.979 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:33:22.979 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:33:22.979 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.979 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.979 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:22.979 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:33:22.979 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:33:22.979 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.979 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.979 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:22.979 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:33:22.979 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:33:22.979 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.979 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.979 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:22.979 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:33:22.979 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:33:22.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.980 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:22.980 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:33:22.980 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:33:22.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.980 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:22.980 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:33:22.980 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:33:22.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.980 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:22.980 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:33:22.980 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:33:22.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.981 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:22.981 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:33:22.981 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:33:22.981 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.981 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.981 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:22.981 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:33:22.981 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:33:22.981 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.981 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:22.981 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.981 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.981 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.981 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:22.981 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:22.982 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 385.34120886800002, 2023-12-06T18:33:22.982 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:33:22.982 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:22.982 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:22.982 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:22.982 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:22.982 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:22.982 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:33:22.982 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.982 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:22.982 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.982 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:22.982 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:22.982 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.982 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.983 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.983 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:22.983 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:22.983 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.983 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.983 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.983 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:22.983 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:33:22.983 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:33:22.983 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.983 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.983 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:22.983 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:33:22.983 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:33:22.983 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.983 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.984 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:22.984 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:33:22.984 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:33:22.984 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.984 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.984 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:22.984 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:33:22.984 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:33:22.984 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.984 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.984 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:22.984 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:33:22.984 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:33:22.984 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.984 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.984 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:22.985 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:33:22.985 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:33:22.985 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.985 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.985 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:22.985 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:33:22.985 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:33:22.985 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.985 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.985 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:22.985 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:33:22.985 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:33:22.985 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.985 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.985 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:22.985 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:33:22.985 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:33:22.986 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.986 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.986 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:22.986 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:33:22.986 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:33:22.986 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.986 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.986 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:22.986 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:33:22.987 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:33:22.987 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.987 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:22.987 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.987 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.987 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.987 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:22.987 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:22.988 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 385.34052715899998, 2023-12-06T18:33:22.988 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:33:22.988 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:22.988 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:22.988 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:22.988 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:22.988 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:22.989 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:33:22.989 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.989 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:22.989 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.989 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:22.989 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:22.989 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.990 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.990 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.990 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:22.990 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:22.990 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.990 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.990 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.991 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:22.991 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:33:22.991 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:33:22.991 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.991 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.991 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:22.991 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:33:22.991 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:33:22.991 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.992 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.992 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:22.992 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:33:22.992 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:33:22.992 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.992 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.992 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:22.992 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:33:22.993 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:33:22.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.993 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.993 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:22.993 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:33:22.993 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:33:22.994 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.994 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.994 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:22.994 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:33:22.994 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:33:22.994 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.995 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.995 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:22.995 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:33:22.995 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:33:22.995 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.995 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.995 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:22.996 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:33:22.996 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:33:22.996 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.996 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.996 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:22.996 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:33:22.996 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:33:22.996 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.996 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.996 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:22.996 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:33:22.996 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:33:22.997 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.997 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.997 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:22.997 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:33:22.997 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:33:22.997 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.997 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:22.997 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:22.997 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.997 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.997 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:22.997 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:22.997 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 384.26172307100001, 2023-12-06T18:33:22.997 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:33:22.997 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:22.998 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:22.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:22.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:22.998 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:22.998 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:33:22.998 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.998 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:22.998 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.998 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:22.998 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:22.998 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.998 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.998 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.998 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:22.999 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:22.999 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:22.999 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.999 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.999 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:22.999 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:33:22.999 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:33:22.999 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.999 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.999 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:22.999 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:33:22.999 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:33:22.999 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:22.999 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:22.999 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:23.000 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:33:23.000 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:33:23.000 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.000 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.000 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:23.000 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:33:23.000 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:33:23.000 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.000 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.000 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:23.000 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:33:23.001 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:33:23.001 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.001 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.001 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:23.001 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:33:23.001 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:33:23.001 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.001 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.001 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:23.001 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:33:23.001 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:33:23.001 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.001 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.002 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:23.002 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:33:23.002 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:33:23.002 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.002 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.002 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:23.002 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:33:23.002 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:33:23.002 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.002 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.002 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:23.002 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:33:23.002 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:33:23.003 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.003 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.003 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:23.003 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:33:23.003 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:33:23.003 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.003 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:23.003 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.003 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.003 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.003 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:23.003 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:23.003 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 384.26030649699999, 2023-12-06T18:33:23.003 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:33:23.003 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:23.003 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:23.004 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:23.004 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:23.004 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:23.004 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:33:23.004 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.004 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:23.004 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.004 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:23.004 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:23.004 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.004 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.004 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.005 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:23.005 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:23.005 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.005 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.005 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.005 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:23.005 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:33:23.005 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:33:23.005 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.005 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.005 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:23.005 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:33:23.005 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:33:23.005 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.005 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.005 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:23.006 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:33:23.006 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:33:23.006 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.006 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.006 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:23.006 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:33:23.006 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:33:23.006 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.006 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.006 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:23.006 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:33:23.006 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:33:23.006 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.006 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.007 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:23.007 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:33:23.007 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:33:23.007 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.007 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.007 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:23.007 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:33:23.007 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:33:23.007 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.007 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.007 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:23.007 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:33:23.007 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:33:23.007 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.007 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.008 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:23.008 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:33:23.008 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:33:23.008 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.008 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.008 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:23.008 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:33:23.008 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:33:23.008 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.008 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.008 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:23.008 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:33:23.009 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:33:23.009 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.009 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:23.009 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.009 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.009 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.009 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:23.009 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:23.009 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 384.25922137200001, 2023-12-06T18:33:23.009 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:33:23.009 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:23.009 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:23.009 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:23.009 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:23.009 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:23.010 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:33:23.010 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.010 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:23.010 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.010 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:23.010 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:23.010 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.010 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.010 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.010 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:23.010 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:23.010 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.010 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.010 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.011 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:23.011 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:33:23.011 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:33:23.011 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.011 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.011 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:23.011 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:33:23.011 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:33:23.011 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.011 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.011 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:23.011 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:33:23.011 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:33:23.011 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.011 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.011 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:23.011 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:33:23.012 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:33:23.012 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.012 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.012 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:23.012 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:33:23.012 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:33:23.012 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.012 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.012 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:23.012 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:33:23.012 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:33:23.012 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.012 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.012 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:23.012 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:33:23.013 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:33:23.013 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.013 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.013 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:23.013 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:33:23.013 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:33:23.013 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.013 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.013 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:23.013 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:33:23.013 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:33:23.013 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.013 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.013 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:23.013 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:33:23.013 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:33:23.014 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.014 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.014 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:23.014 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:33:23.014 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:33:23.014 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.014 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:23.014 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.014 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.014 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.014 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:23.015 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:23.015 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 384.22407227000002, 2023-12-06T18:33:23.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:33:23.015 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:23.015 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:23.015 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:23.015 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:23.015 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:23.015 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:33:23.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.015 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:23.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:23.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:23.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:23.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:23.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:23.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:33:23.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:33:23.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:23.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:33:23.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:33:23.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:23.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:33:23.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:33:23.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:23.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:33:23.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:33:23.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:23.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:33:23.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:33:23.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:23.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:33:23.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:33:23.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:23.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:33:23.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:33:23.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:23.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:33:23.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:33:23.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:23.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:33:23.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:33:23.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:23.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:33:23.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:33:23.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.021 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.021 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:23.021 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:33:23.021 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:33:23.021 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.021 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:23.021 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.021 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.022 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:23.022 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:23.022 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 377.67434174499999, 2023-12-06T18:33:23.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:33:23.022 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:23.022 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:23.023 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:23.023 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:23.023 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:23.023 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:33:23.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.023 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:23.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.023 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:23.023 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:23.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:23.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:23.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:23.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:33:23.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:33:23.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:23.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:33:23.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:33:23.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:23.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:33:23.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:33:23.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:23.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:33:23.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:33:23.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.026 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:23.026 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:33:23.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:33:23.026 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.026 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.026 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:23.026 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:33:23.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:33:23.026 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.026 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.026 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:23.026 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:33:23.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:33:23.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.027 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:23.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:33:23.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:33:23.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.027 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:23.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:33:23.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:33:23.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.027 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:23.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:33:23.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:33:23.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:23.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:33:23.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:33:23.028 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.028 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:23.028 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.028 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:23.028 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:23.028 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 377.67353613699999, 2023-12-06T18:33:23.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:33:23.029 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:23.029 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:23.029 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:23.029 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:23.029 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:23.029 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:33:23.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.029 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:23.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:23.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:23.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:23.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:23.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:23.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:33:23.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:33:23.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:23.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:33:23.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:33:23.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:23.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:33:23.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:33:23.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:23.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:33:23.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:33:23.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:23.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:33:23.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:33:23.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:23.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:33:23.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:33:23.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:23.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:33:23.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:33:23.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:23.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:33:23.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:33:23.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:23.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:33:23.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:33:23.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:23.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:33:23.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:33:23.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:23.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:33:23.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:33:23.033 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.033 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:23.033 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.033 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:23.034 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:23.034 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 377.67284532799999, 2023-12-06T18:33:23.034 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:33:23.034 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:23.034 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:23.034 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:23.034 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:23.034 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:23.034 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:33:23.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.034 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:23.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:23.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:23.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:23.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:23.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:23.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:33:23.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:33:23.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:23.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:33:23.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:33:23.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:23.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:33:23.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:33:23.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.037 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:23.037 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:33:23.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:33:23.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.037 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:23.037 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:33:23.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:33:23.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:23.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:33:23.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:33:23.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:23.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:33:23.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:33:23.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:23.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:33:23.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:33:23.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:23.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:33:23.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:33:23.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:23.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:33:23.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:33:23.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.040 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:23.040 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:33:23.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:33:23.040 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.040 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:23.040 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.040 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.040 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.040 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:23.040 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:23.040 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 377.66854782000001, 2023-12-06T18:33:23.040 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:33:23.041 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:23.041 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:23.041 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:23.041 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:23.041 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:23.041 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:33:23.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.041 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:23.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.041 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:23.041 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:23.041 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.041 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.041 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.041 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:23.042 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:23.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.042 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.042 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.042 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:23.042 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:33:23.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:33:23.042 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.042 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.042 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:23.042 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:33:23.042 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:33:23.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:23.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:33:23.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:33:23.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:23.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:33:23.043 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:33:23.043 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.043 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.043 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:23.043 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:33:23.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:33:23.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:23.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:33:23.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:33:23.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:23.044 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:33:23.044 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:33:23.044 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.044 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.044 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:23.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:33:23.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:33:23.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:23.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:33:23.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:33:23.045 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.045 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.045 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:23.045 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:33:23.045 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:33:23.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.046 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:23.046 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:33:23.046 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:33:23.046 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.046 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:23.046 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.046 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.046 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.046 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:23.046 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:23.046 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 377.66788014500003, 2023-12-06T18:33:23.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:33:23.047 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:23.047 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:23.047 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:23.047 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:23.047 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:23.047 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:33:23.047 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.047 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:23.047 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.047 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:23.047 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:23.047 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:23.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:23.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:23.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:33:23.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:33:23.048 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.048 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:23.048 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:33:23.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:33:23.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:23.049 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:33:23.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:33:23.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:23.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:33:23.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:33:23.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:23.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:33:23.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:33:23.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:23.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:33:23.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:33:23.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:23.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:33:23.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:33:23.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:23.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:33:23.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:33:23.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:23.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:33:23.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:33:23.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:23.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:33:23.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:33:23.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.054 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:23.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:33:23.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:33:23.055 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.055 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:23.055 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.055 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:23.055 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:23.055 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 377.66722471700001, 2023-12-06T18:33:23.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:33:23.056 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:23.056 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:23.056 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:23.056 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:23.056 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:23.056 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:33:23.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.056 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:23.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:23.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:23.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:23.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:23.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:23.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:33:23.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:33:23.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:23.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:33:23.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:33:23.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:23.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:33:23.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:33:23.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:23.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:33:23.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:33:23.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:23.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:33:23.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:33:23.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:23.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:33:23.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:33:23.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:23.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:33:23.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:33:23.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:23.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:33:23.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:33:23.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:23.060 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:33:23.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:33:23.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:23.060 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:33:23.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:33:23.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:23.060 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:33:23.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:33:23.061 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.061 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:23.061 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.061 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:23.061 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:23.061 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 377.62657016999998, 2023-12-06T18:33:23.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:33:23.061 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:23.061 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:23.061 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:23.061 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:23.061 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:23.061 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:33:23.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.062 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:23.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:23.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:23.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:23.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:23.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:23.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:33:23.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:33:23.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:23.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:33:23.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:33:23.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:23.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:33:23.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:33:23.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:23.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:33:23.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:33:23.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:23.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:33:23.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:33:23.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:23.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:33:23.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:33:23.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:23.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:33:23.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:33:23.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:23.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:33:23.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:33:23.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:23.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:33:23.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:33:23.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.066 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.066 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:23.066 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:33:23.066 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:33:23.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.066 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.066 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:23.066 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:33:23.066 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:33:23.066 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.066 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:23.066 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.066 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.066 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:23.067 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:23.068 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:33:23.068 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:23.065+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:23.170 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:23.167+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:23.246 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:23.246 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:23.247 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:23.247 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:23.249 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-12-06T18:33:23.256 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:33:23.271 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:23.268+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:23.302 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '22 of 30' 2023-12-06T18:33:23.388 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:23.385+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:23.452 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:23.453 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:23.453 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:23.453 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:23.453 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:23.462 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:33:23.474 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:23.470+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:23.574 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:23.571+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:23.642 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:23.642 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:23.642 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:23.643 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:23.643 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.643 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:33:23.643 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:23.643 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 361.82642812500001, 2023-12-06T18:33:23.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:33:23.643 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:23.643 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:33:23.643 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:23.643 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.643 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:23.643 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:23.643 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:23.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:23.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:23.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:23.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:33:23.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:33:23.644 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.644 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.644 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:23.644 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:33:23.644 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:33:23.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.645 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:23.645 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:33:23.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:33:23.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.645 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:23.645 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:33:23.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:33:23.645 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.645 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.645 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:23.645 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:33:23.645 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:33:23.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:23.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:33:23.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:33:23.646 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:23.646 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:23.646 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:23.646 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:33:23.646 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:33:23.646 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.646 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:23.646 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.646 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:23.646 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:23.647 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:23.653 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:33:23.675 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:23.672+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:23.776 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:23.773+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:23.793 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:23.794 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:23.794 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:23.794 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:23.803 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:33:23.877 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:23.874+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:23.944 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:23.944 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:23.944 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:23.944 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:23.944 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:23.953 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:33:23.995 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:23.992+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:24.078 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:24.076+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:24.142 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:24.143 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:24.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:24.143 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:24.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.143 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:33:24.143 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:24.143 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 243.04120871200001, 2023-12-06T18:33:24.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:33:24.143 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:24.143 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:24.143 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:24.143 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:24.143 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:24.143 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:33:24.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.144 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:24.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:24.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:24.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:24.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:24.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:24.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:24.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:24.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:33:24.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:33:24.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:24.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:33:24.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:33:24.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:24.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:33:24.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:33:24.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:24.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:33:24.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:33:24.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:24.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:33:24.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:33:24.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:24.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:33:24.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:33:24.146 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:24.146 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:24.147 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:24.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.147 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:33:24.147 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:24.147 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 183.02295140999999, 2023-12-06T18:33:24.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:33:24.147 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:24.147 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:24.147 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:24.147 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:24.147 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:24.147 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:33:24.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.148 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:24.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:24.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:24.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:24.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:24.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:24.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:24.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:24.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:33:24.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:33:24.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:24.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:33:24.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:33:24.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:24.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:33:24.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:33:24.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:24.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:33:24.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:33:24.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:24.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:33:24.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:33:24.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:24.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:33:24.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:33:24.150 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:24.150 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:24.150 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:24.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.151 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:11 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2023-12-06T18:33:24.151 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:24.151 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 62.998530293000002, 2023-12-06T18:33:24.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0086075449999999994, 2023-12-06T18:33:24.151 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:24.151 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:24.151 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:24.151 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:24.151 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:24.151 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 11 2023-12-06T18:33:24.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.151 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:24.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.151 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:24.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:24.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:24.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:24.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:24.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:24.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:24.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135441+0000", 2023-12-06T18:33:24.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.8695999999999999e-05 2023-12-06T18:33:24.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.153 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:24.153 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135650+0000", 2023-12-06T18:33:24.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020883699999999999 2023-12-06T18:33:24.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.153 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:24.153 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135863+0000", 2023-12-06T18:33:24.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000213931 2023-12-06T18:33:24.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.153 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:24.153 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.136054+0000", 2023-12-06T18:33:24.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000190416 2023-12-06T18:33:24.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:24.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.142056+0000", 2023-12-06T18:33:24.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0060019330000000001 2023-12-06T18:33:24.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:24.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:24.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:24.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.144030+0000", 2023-12-06T18:33:24.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0019737320000000002 2023-12-06T18:33:24.154 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:24.154 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:24.154 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:24.154 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:24.154 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:24.154 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:24.156 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:33:24.197 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:24.194+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:24.280 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:24.277+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:24.288 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:24.288 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:24.288 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:24.288 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:24.295 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 7, 5, 4] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:33:24.295 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:33:24.296 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:33:24.297 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:33:24.318 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '23 of 30' 2023-12-06T18:33:24.358 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is set 2023-12-06T18:33:24.381 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:24.378+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:24.419 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:33:24.468 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:24.468 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:24.468 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:24.468 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:24.468 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:24.478 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:33:24.519 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:24.515+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:24.583 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:24.579+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:24.651 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:24.652 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:24.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:24.652 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:33:24.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:24.652 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:33:24.652 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:33:24.652 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 303.15803552699998, 2023-12-06T18:33:24.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:33:24.652 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:24.652 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:24.652 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:24.652 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:24.653 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:24.653 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:33:24.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:24.653 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:24.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:24.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:24.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:33:24.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:24.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:24.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:24.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:24.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:33:24.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:24.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:24.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:24.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:24.654 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:33:24.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:33:24.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:24.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:24.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:24.654 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:33:24.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:33:24.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:24.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:24.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:24.654 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:33:24.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:33:24.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:24.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:24.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:24.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:33:24.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:33:24.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:24.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:24.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:24.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:33:24.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:33:24.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:24.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:24.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:24.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:33:24.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:33:24.655 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:24.655 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:24.655 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:24.656 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:24.656 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:24.656 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:24.663 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:33:24.703 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:24.700+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:24.785 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:24.782+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:24.830 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:24.831 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:24.831 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:24.831 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:24.840 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:33:24.886 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:24.884+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:24.915 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:33:24.916 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:33:24.930 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2023-12-06T18:33:24.420291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":200588,"kb_used_data":6256,"kb_used_omap":13,"kb_used_meta":190002,"kb_avail":660402292,"statfs":{"total":676457349120,"available":676251947008,"internally_reserved":0,"allocated":6406144,"data_stored":2796685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13963,"internal_metadata":194562421},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":62,"apply_latency_ms":62,"commit_latency_ns":62000000,"apply_latency_ns":62000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.076522"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":61,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-06T18:33:16.047254+0000","last_change":"2023-12-06T18:29:09.051293+0000","last_active":"2023-12-06T18:33:16.047254+0000","last_peered":"2023-12-06T18:33:16.047254+0000","last_clean":"2023-12-06T18:33:16.047254+0000","last_became_active":"2023-12-06T18:29:09.045331+0000","last_became_peered":"2023-12-06T18:29:09.045331+0000","last_unstale":"2023-12-06T18:33:16.047254+0000","last_undegraded":"2023-12-06T18:33:16.047254+0000","last_fullsized":"2023-12-06T18:33:16.047254+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T02:46:01.854508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-06T18:33:16.444592+0000","last_change":"2023-12-06T18:33:00.623558+0000","last_active":"2023-12-06T18:33:16.444592+0000","last_peered":"2023-12-06T18:33:16.444592+0000","last_clean":"2023-12-06T18:33:16.444592+0000","last_became_active":"2023-12-06T18:33:00.616353+0000","last_became_peered":"2023-12-06T18:33:00.616353+0000","last_unstale":"2023-12-06T18:33:16.444592+0000","last_undegraded":"2023-12-06T18:33:16.444592+0000","last_fullsized":"2023-12-06T18:33:16.444592+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T18:33:31.069235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":112,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-12-06T18:33:22.067779+0000","last_change":"2023-12-06T18:33:00.592340+0000","last_active":"2023-12-06T18:33:22.067779+0000","last_peered":"2023-12-06T18:33:22.067779+0000","last_clean":"2023-12-06T18:33:22.067779+0000","last_became_active":"2023-12-06T18:33:00.581037+0000","last_became_peered":"2023-12-06T18:33:00.581037+0000","last_unstale":"2023-12-06T18:33:22.067779+0000","last_undegraded":"2023-12-06T18:33:22.067779+0000","last_fullsized":"2023-12-06T18:33:22.067779+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T06:13:54.498147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-06T18:33:16.049287+0000","last_change":"2023-12-06T18:32:23.586404+0000","last_active":"2023-12-06T18:33:16.049287+0000","last_peered":"2023-12-06T18:33:16.049287+0000","last_clean":"2023-12-06T18:33:16.049287+0000","last_became_active":"2023-12-06T18:32:23.580588+0000","last_became_peered":"2023-12-06T18:32:23.580588+0000","last_unstale":"2023-12-06T18:33:16.049287+0000","last_undegraded":"2023-12-06T18:33:16.049287+0000","last_fullsized":"2023-12-06T18:33:16.049287+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:31:58.277721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":105,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-12-06T18:33:22.063821+0000","last_change":"2023-12-06T18:32:23.632179+0000","last_active":"2023-12-06T18:33:22.063821+0000","last_peered":"2023-12-06T18:33:22.063821+0000","last_clean":"2023-12-06T18:33:22.063821+0000","last_became_active":"2023-12-06T18:32:23.623784+0000","last_became_peered":"2023-12-06T18:32:23.623784+0000","last_unstale":"2023-12-06T18:33:22.063821+0000","last_undegraded":"2023-12-06T18:33:22.063821+0000","last_fullsized":"2023-12-06T18:33:22.063821+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T21:18:05.348471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":78,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-12-06T18:33:20.574490+0000","last_change":"2023-12-06T18:30:08.762262+0000","last_active":"2023-12-06T18:33:20.574490+0000","last_peered":"2023-12-06T18:33:20.574490+0000","last_clean":"2023-12-06T18:33:20.574490+0000","last_became_active":"2023-12-06T18:30:08.733421+0000","last_became_peered":"2023-12-06T18:30:08.733421+0000","last_unstale":"2023-12-06T18:33:20.574490+0000","last_undegraded":"2023-12-06T18:33:20.574490+0000","last_fullsized":"2023-12-06T18:33:20.574490+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:43:31.796397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":112,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-12-06T18:33:21.245413+0000","last_change":"2023-12-06T18:28:47.876556+0000","last_active":"2023-12-06T18:33:21.245413+0000","last_peered":"2023-12-06T18:33:21.245413+0000","last_clean":"2023-12-06T18:33:21.245413+0000","last_became_active":"2023-12-06T18:28:47.865169+0000","last_became_peered":"2023-12-06T18:28:47.865169+0000","last_unstale":"2023-12-06T18:33:21.245413+0000","last_undegraded":"2023-12-06T18:33:21.245413+0000","last_fullsized":"2023-12-06T18:33:21.245413+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:16:40.095542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":39,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-12-06T18:33:18.255832+0000","last_change":"2023-12-06T18:31:57.187068+0000","last_active":"2023-12-06T18:33:18.255832+0000","last_peered":"2023-12-06T18:33:18.255832+0000","last_clean":"2023-12-06T18:33:18.255832+0000","last_became_active":"2023-12-06T18:31:57.175444+0000","last_became_peered":"2023-12-06T18:31:57.175444+0000","last_unstale":"2023-12-06T18:33:18.255832+0000","last_undegraded":"2023-12-06T18:33:18.255832+0000","last_fullsized":"2023-12-06T18:33:18.255832+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:25:09.547812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":2,"reported_epoch":109,"state":"peering","last_fresh":"2023-12-06T18:33:18.249644+0000","last_change":"2023-12-06T18:33:20.909755+0000","last_active":"2023-12-06T18:33:20.909755+0000","last_peered":"2023-12-06T18:33:18.249644+0000","last_clean":"2023-12-06T18:33:18.249644+0000","last_became_active":"2023-12-06T18:28:47.875258+0000","last_became_peered":"2023-12-06T18:28:47.875258+0000","last_unstale":"2023-12-06T18:33:18.249644+0000","last_undegraded":"2023-12-06T18:33:18.249644+0000","last_fullsized":"2023-12-06T18:33:18.249644+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":795680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1204224,"data_stored":1193520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411408,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":290955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":10.098000000000001,"5min":10.752000000000001,"15min":10.554},"min":{"1min":6.3730000000000002,"5min":3.835,"15min":3.835},"max":{"1min":25.571000000000002,"5min":108.57599999999999,"15min":108.57599999999999},"last":17.792999999999999},{"interface":"front","average":{"1min":11.791,"5min":11.002000000000001,"15min":12.173999999999999},"min":{"1min":4.8639999999999999,"5min":3.7200000000000002,"15min":3.7200000000000002},"max":{"1min":20.649999999999999,"5min":128.09399999999999,"15min":128.09399999999999},"last":21.036999999999999}]},{"osd":1,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":13.631,"5min":12.202,"15min":12.771000000000001},"min":{"1min":7.6390000000000002,"5min":6.2539999999999996,"15min":6.2539999999999996},"max":{"1min":26.565999999999999,"5min":70.072000000000003,"15min":70.072000000000003},"last":15.797000000000001},{"interface":"front","average":{"1min":14.135999999999999,"5min":12.827999999999999,"15min":15.066000000000001},"min":{"1min":6.8280000000000003,"5min":6.444,"15min":6.444},"max":{"1min":26.420000000000002,"5min":27.934000000000001,"15min":32.920000000000002},"last":9.9870000000000001}]},{"osd":2,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":13.890000000000001,"5min":14.91,"15min":14.795},"min":{"1min":8.8390000000000004,"5min":7.7690000000000001,"15min":7.5149999999999997},"max":{"1min":24.039000000000001,"5min":75.125,"15min":75.125},"last":18.948},{"interface":"front","average":{"1min":15.166,"5min":15.824999999999999,"15min":17.459},"min":{"1min":7.5389999999999997,"5min":7.5389999999999997,"15min":7.5389999999999997},"max":{"1min":24.375,"5min":69.403999999999996,"15min":69.403999999999996},"last":23.766999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":16.661000000000001,"5min":13.574,"15min":12.426},"min":{"1min":10.935,"5min":5.8879999999999999,"15min":5.8879999999999999},"max":{"1min":27.975000000000001,"5min":27.975000000000001,"15min":27.975000000000001},"last":20.061},{"interface":"front","average":{"1min":10.597,"5min":13.162000000000001,"15min":15.118},"min":{"1min":7.29,"5min":6.3099999999999996,"15min":6.3099999999999996},"max":{"1min":23.651,"5min":71.718000000000004,"15min":71.718000000000004},"last":6.4240000000000004}]},{"osd":5,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":18.983000000000001,"5min":18.795999999999999,"15min":18.07},"min":{"1min":13.872,"5min":9.798,"15min":8.3179999999999996},"max":{"1min":28.015999999999998,"5min":75.751999999999995,"15min":75.751999999999995},"last":21.655000000000001},{"interface":"front","average":{"1min":18.129000000000001,"5min":17.302,"15min":17.986999999999998},"min":{"1min":10.148,"5min":7.9269999999999996,"15min":7.9269999999999996},"max":{"1min":28.039999999999999,"5min":70.933000000000007,"15min":70.933000000000007},"last":15.127000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":13.880000000000001,"5min":14.038,"15min":15.648999999999999},"min":{"1min":10.093,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":22.777000000000001,"5min":28.469999999999999,"15min":29.245000000000001},"last":21.686},{"interface":"front","average":{"1min":19.372,"5min":19.238,"15min":20.812000000000001},"min":{"1min":14.287000000000001,"5min":10.973000000000001,"15min":10.973000000000001},"max":{"1min":29.497,"5min":74.224000000000004,"15min":74.224000000000004},"last":27.911999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":17.841000000000001,"5min":19.835999999999999,"15min":21.25},"min":{"1min":9.1859999999999999,"5min":9.1859999999999999,"15min":9.1859999999999999},"max":{"1min":33.747,"5min":122.93000000000001,"15min":122.93000000000001},"last":16.355},{"interface":"front","average":{"1min":22.260000000000002,"5min":22.786999999999999,"15min":23.417999999999999},"min":{"1min":15.458,"5min":13.196,"15min":13.196},"max":{"1min":31.978999999999999,"5min":124.473,"15min":124.473},"last":22.273}]}]},{"osd":2,"up_from":16,"seq":68719476815,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":286538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:33:08 2023","interfaces":[{"interface":"back","average":{"1min":12.861000000000001,"5min":10.439,"15min":11.157999999999999},"min":{"1min":6.0350000000000001,"5min":4.1639999999999997,"15min":4.1639999999999997},"max":{"1min":29.417000000000002,"5min":39.220999999999997,"15min":39.220999999999997},"last":18.291},{"interface":"front","average":{"1min":14.871,"5min":11.584,"15min":11.446},"min":{"1min":7.1440000000000001,"5min":4.3280000000000003,"15min":4.3280000000000003},"max":{"1min":31.372,"5min":44.691000000000003,"15min":44.691000000000003},"last":9.6869999999999994}]},{"osd":1,"last update":"Wed Dec 6 18:33:08 2023","interfaces":[{"interface":"back","average":{"1min":9.8409999999999993,"5min":11.188000000000001,"15min":11.483000000000001},"min":{"1min":7.2530000000000001,"5min":5.9139999999999997,"15min":5.9139999999999997},"max":{"1min":15.18,"5min":42.920999999999999,"15min":42.920999999999999},"last":28.242999999999999},{"interface":"front","average":{"1min":13.111000000000001,"5min":13.525,"15min":13.849},"min":{"1min":7.1230000000000002,"5min":6.1479999999999997,"15min":6.1479999999999997},"max":{"1min":18.948,"5min":34.607999999999997,"15min":34.607999999999997},"last":29.478999999999999}]},{"osd":3,"last update":"Wed Dec 6 18:33:08 2023","interfaces":[{"interface":"back","average":{"1min":15.618,"5min":14.375999999999999,"15min":18.050000000000001},"min":{"1min":9.5869999999999997,"5min":6.1260000000000003,"15min":6.1260000000000003},"max":{"1min":26.550999999999998,"5min":49.609000000000002,"15min":112.129},"last":9.0139999999999993},{"interface":"front","average":{"1min":16.542999999999999,"5min":16.094000000000001,"15min":22.181999999999999},"min":{"1min":10.18,"5min":7.5510000000000002,"15min":7.5510000000000002},"max":{"1min":22.042999999999999,"5min":61.243000000000002,"15min":181.601},"last":10.819000000000001}]},{"osd":4,"last update":"Wed Dec 6 18:33:08 2023","interfaces":[{"interface":"back","average":{"1min":14.792999999999999,"5min":14.91,"15min":17.263999999999999},"min":{"1min":10.734,"5min":6.609,"15min":6.609},"max":{"1min":21.219999999999999,"5min":60.332000000000001,"15min":97.088999999999999},"last":13.500999999999999},{"interface":"front","average":{"1min":11.157,"5min":13.459,"15min":19.145},"min":{"1min":7.2990000000000004,"5min":6.0720000000000001,"15min":6.0720000000000001},"max":{"1min":16.617999999999999,"5min":43.198,"15min":98.069000000000003},"last":15.869}]},{"osd":5,"last update":"Wed Dec 6 18:33:08 2023","interfaces":[{"interface":"back","average":{"1min":17.873999999999999,"5min":17.538,"15min":18.042000000000002},"min":{"1min":10.359,"5min":7.8170000000000002,"15min":7.8170000000000002},"max":{"1min":39.686,"5min":57.628999999999998,"15min":57.628999999999998},"last":12.102},{"interface":"front","average":{"1min":15.845000000000001,"5min":15.821999999999999,"15min":15.834},"min":{"1min":9.8049999999999997,"5min":8.5990000000000002,"15min":8.5990000000000002},"max":{"1min":23.103000000000002,"5min":47.749000000000002,"15min":47.749000000000002},"last":17.100999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:33:08 2023","interfaces":[{"interface":"back","average":{"1min":18.725000000000001,"5min":18.283000000000001,"15min":37.774999999999999},"min":{"1min":11.263,"5min":8.7289999999999992,"15min":8.7289999999999992},"max":{"1min":31.329000000000001,"5min":64.034999999999997,"15min":498.38499999999999},"last":19.532},{"interface":"front","average":{"1min":20.073,"5min":19.332000000000001,"15min":39.524999999999999},"min":{"1min":10.803000000000001,"5min":9.7010000000000005,"15min":9.7010000000000005},"max":{"1min":30.225000000000001,"5min":54.920000000000002,"15min":496.17399999999998},"last":14.798999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:33:14 2023","interfaces":[{"interface":"back","average":{"1min":19.437999999999999,"5min":18.989999999999998,"15min":18.945},"min":{"1min":11.891999999999999,"5min":10.064,"15min":10.064},"max":{"1min":29.789000000000001,"5min":49.456000000000003,"15min":49.456000000000003},"last":20.867999999999999},{"interface":"front","average":{"1min":20.567,"5min":20.460000000000001,"15min":20.257000000000001},"min":{"1min":14.108000000000001,"5min":12.288,"15min":12.288},"max":{"1min":32.514000000000003,"5min":40.656999999999996,"15min":40.656999999999996},"last":12.81}]}]},{"osd":1,"up_from":16,"seq":68719476816,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1191936,"data_stored":679940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:33:06 2023","interfaces":[{"interface":"back","average":{"1min":11.112,"5min":10.651,"15min":19.792000000000002},"min":{"1min":7.1459999999999999,"5min":5.0890000000000004,"15min":5.0890000000000004},"max":{"1min":25.581,"5min":34.286000000000001,"15min":238.87100000000001},"last":11.448},{"interface":"front","average":{"1min":13.031000000000001,"5min":10.532,"15min":37.002000000000002},"min":{"1min":6.452,"5min":4.4269999999999996,"15min":4.4269999999999996},"max":{"1min":61.262,"5min":61.262,"15min":661.33600000000001},"last":5.0629999999999997}]},{"osd":2,"last update":"Wed Dec 6 18:33:06 2023","interfaces":[{"interface":"back","average":{"1min":11.746,"5min":12.837,"15min":28.960999999999999},"min":{"1min":7.9390000000000001,"5min":6.6340000000000003,"15min":6.6340000000000003},"max":{"1min":21.129000000000001,"5min":93.168000000000006,"15min":384.315},"last":21.116},{"interface":"front","average":{"1min":17.257999999999999,"5min":17.582000000000001,"15min":41.338999999999999},"min":{"1min":12.568,"5min":8.109,"15min":5.7009999999999996},"max":{"1min":27.864999999999998,"5min":94.230000000000004,"15min":656.98699999999997},"last":23.954000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:33:06 2023","interfaces":[{"interface":"back","average":{"1min":12.555999999999999,"5min":11.630000000000001,"15min":12.943},"min":{"1min":7.8159999999999998,"5min":5.548,"15min":5.548},"max":{"1min":28.707000000000001,"5min":31.16,"15min":34.057000000000002},"last":10.532},{"interface":"front","average":{"1min":16.106000000000002,"5min":13.483000000000001,"15min":15.641999999999999},"min":{"1min":6.9160000000000004,"5min":6.5730000000000004,"15min":6.5730000000000004},"max":{"1min":65.537000000000006,"5min":65.537000000000006,"15min":65.537000000000006},"last":8.4770000000000003}]},{"osd":4,"last update":"Wed Dec 6 18:33:06 2023","interfaces":[{"interface":"back","average":{"1min":14.526,"5min":12.695,"15min":35.174999999999997},"min":{"1min":8.3130000000000006,"5min":6.2930000000000001,"15min":6.2930000000000001},"max":{"1min":33.201999999999998,"5min":33.201999999999998,"15min":447.67200000000003},"last":9.4979999999999993},{"interface":"front","average":{"1min":17.960999999999999,"5min":16.978000000000002,"15min":42.109999999999999},"min":{"1min":10.378,"5min":9.6129999999999995,"15min":7.0919999999999996},"max":{"1min":32.469999999999999,"5min":44.043999999999997,"15min":630.572},"last":21.771999999999998}]},{"osd":5,"last update":"Wed Dec 6 18:33:06 2023","interfaces":[{"interface":"back","average":{"1min":16.893999999999998,"5min":16.215,"15min":37.536999999999999},"min":{"1min":9.5530000000000008,"5min":7.0540000000000003,"15min":7.0540000000000003},"max":{"1min":68.013999999999996,"5min":68.013999999999996,"15min":389.55000000000001},"last":18.321999999999999},{"interface":"front","average":{"1min":19.353000000000002,"5min":16.658999999999999,"15min":48.807000000000002},"min":{"1min":11.243,"5min":7.0410000000000004,"15min":7.0410000000000004},"max":{"1min":27.831,"5min":37.744,"15min":658.48599999999999},"last":26.794}]},{"osd":6,"last update":"Wed Dec 6 18:33:06 2023","interfaces":[{"interface":"back","average":{"1min":17.175000000000001,"5min":14.875,"15min":31.696999999999999},"min":{"1min":8.3640000000000008,"5min":7.5899999999999999,"15min":7.5899999999999999},"max":{"1min":35.575000000000003,"5min":54.383000000000003,"15min":439.56400000000002},"last":19.606000000000002},{"interface":"front","average":{"1min":20.062000000000001,"5min":19.577000000000002,"15min":33.789000000000001},"min":{"1min":11.862,"5min":10.641999999999999,"15min":10.641999999999999},"max":{"1min":44.154000000000003,"5min":55.423999999999999,"15min":386.71499999999997},"last":25.914000000000001}]},{"osd":7,"last update":"Wed Dec 6 18:33:11 2023","interfaces":[{"interface":"back","average":{"1min":19.498999999999999,"5min":18.276,"15min":18.617999999999999},"min":{"1min":9.3680000000000003,"5min":9.3680000000000003,"15min":9.3680000000000003},"max":{"1min":62.582999999999998,"5min":97.561999999999998,"15min":97.561999999999998},"last":22.530000000000001},{"interface":"front","average":{"1min":26.725000000000001,"5min":19.928999999999998,"15min":18.407},"min":{"1min":14.022,"5min":8.7219999999999995,"15min":8.7219999999999995},"max":{"1min":101.762,"5min":101.762,"15min":101.762},"last":23.183}]}]},{"osd":0,"up_from":16,"seq":68719476817,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":286538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":11.467000000000001,"5min":10.736000000000001,"15min":15.943},"min":{"1min":5.6749999999999998,"5min":3.2360000000000002,"15min":3.2360000000000002},"max":{"1min":40.502000000000002,"5min":47.661000000000001,"15min":116.593},"last":9.2899999999999991},{"interface":"front","average":{"1min":14.481,"5min":12.929,"15min":16.911999999999999},"min":{"1min":5.5709999999999997,"5min":4.9039999999999999,"15min":4.9039999999999999},"max":{"1min":42.502000000000002,"5min":50.127000000000002,"15min":117.896},"last":10.064}]},{"osd":2,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":15.561,"5min":14.813000000000001,"15min":13.863},"min":{"1min":6.726,"5min":5.694,"15min":5.694},"max":{"1min":25.344000000000001,"5min":43.752000000000002,"15min":43.752000000000002},"last":12.503},{"interface":"front","average":{"1min":14.154999999999999,"5min":13.552,"15min":11.832000000000001},"min":{"1min":6.5220000000000002,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":27.962,"5min":49.670000000000002,"15min":49.670000000000002},"last":10.866}]},{"osd":3,"last update":"Wed Dec 6 18:33:11 2023","interfaces":[{"interface":"back","average":{"1min":18.148,"5min":14.635999999999999,"15min":16.108000000000001},"min":{"1min":11.849,"5min":6.1379999999999999,"15min":6.1379999999999999},"max":{"1min":38.896000000000001,"5min":38.896000000000001,"15min":46.689999999999998},"last":19.039000000000001},{"interface":"front","average":{"1min":14.398999999999999,"5min":15.494,"15min":16.018000000000001},"min":{"1min":7.5510000000000002,"5min":7.5510000000000002,"15min":7.5510000000000002},"max":{"1min":26.088999999999999,"5min":50.545999999999999,"15min":50.545999999999999},"last":16.515999999999998}]},{"osd":4,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":12.005000000000001,"5min":12.326000000000001,"15min":18.361999999999998},"min":{"1min":6.9699999999999998,"5min":5.9349999999999996,"15min":5.9349999999999996},"max":{"1min":16.777000000000001,"5min":43.920999999999999,"15min":82.432000000000002},"last":14.492000000000001},{"interface":"front","average":{"1min":18.808,"5min":18.149999999999999,"15min":19.920000000000002},"min":{"1min":12.162000000000001,"5min":9.5239999999999991,"15min":6.2629999999999999},"max":{"1min":28.553999999999998,"5min":61.402999999999999,"15min":81.239999999999995},"last":21.542000000000002}]},{"osd":5,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":18.818999999999999,"5min":16.928999999999998,"15min":17.396999999999998},"min":{"1min":8.8119999999999994,"5min":8.0099999999999998,"15min":8.0099999999999998},"max":{"1min":36.323,"5min":71.367000000000004,"15min":71.367000000000004},"last":18.190000000000001},{"interface":"front","average":{"1min":17.681000000000001,"5min":16.157,"15min":17.521000000000001},"min":{"1min":9.2650000000000006,"5min":7.1779999999999999,"15min":7.1779999999999999},"max":{"1min":36.862000000000002,"5min":72.307000000000002,"15min":72.307000000000002},"last":17.507000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":20.559999999999999,"5min":19.556999999999999,"15min":19.190999999999999},"min":{"1min":13.153,"5min":8.9179999999999993,"15min":7.7560000000000002},"max":{"1min":44.655999999999999,"5min":62.932000000000002,"15min":62.932000000000002},"last":23.733000000000001},{"interface":"front","average":{"1min":17.888000000000002,"5min":17.213000000000001,"15min":18.768000000000001},"min":{"1min":10.164,"5min":9.3520000000000003,"15min":9.3520000000000003},"max":{"1min":29.501000000000001,"5min":43.088000000000001,"15min":52.627000000000002},"last":20.515000000000001}]},{"osd":7,"last update":"Wed Dec 6 18:33:11 2023","interfaces":[{"interface":"back","average":{"1min":21.228999999999999,"5min":20.321000000000002,"15min":19.521000000000001},"min":{"1min":16.484999999999999,"5min":12.009,"15min":12.009},"max":{"1min":32.023000000000003,"5min":58.021000000000001,"15min":58.021000000000001},"last":25.375},{"interface":"front","average":{"1min":19.852,"5min":18.920999999999999,"15min":18.238},"min":{"1min":11.643000000000001,"5min":11.515000000000001,"15min":11.515000000000001},"max":{"1min":36.466000000000001,"5min":40.978999999999999,"15min":40.978999999999999},"last":22.803000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509521,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1191936,"data_stored":679940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":8.9949999999999992,"5min":8.4920000000000009,"15min":13.872},"min":{"1min":4.1669999999999998,"5min":3.3660000000000001,"15min":3.3660000000000001},"max":{"1min":23.98,"5min":23.98,"15min":208.291},"last":5.2140000000000004},{"interface":"front","average":{"1min":11.83,"5min":10.465999999999999,"15min":17.478000000000002},"min":{"1min":6.3940000000000001,"5min":4.4299999999999997,"15min":4.4299999999999997},"max":{"1min":24.289999999999999,"5min":24.289999999999999,"15min":199.94900000000001},"last":8.4109999999999996}]},{"osd":1,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":14.759,"5min":13.478999999999999,"15min":48.890999999999998},"min":{"1min":7.0339999999999998,"5min":5.085,"15min":5.085},"max":{"1min":22.251000000000001,"5min":41.234999999999999,"15min":1211.6010000000001},"last":10.393000000000001},{"interface":"front","average":{"1min":10.929,"5min":10.045999999999999,"15min":44.293999999999997},"min":{"1min":7.0060000000000002,"5min":4.7199999999999998,"15min":4.7199999999999998},"max":{"1min":17.417999999999999,"5min":39.802999999999997,"15min":1189.3869999999999},"last":14.201000000000001}]},{"osd":2,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":16.795000000000002,"5min":14.260999999999999,"15min":22.454999999999998},"min":{"1min":7.9980000000000002,"5min":6.5129999999999999,"15min":6.5129999999999999},"max":{"1min":68.617999999999995,"5min":68.617999999999995,"15min":283.46800000000002},"last":9.0459999999999994},{"interface":"front","average":{"1min":21.155999999999999,"5min":17.585000000000001,"15min":26.276},"min":{"1min":8.173,"5min":8.173,"15min":8.173},"max":{"1min":69.513999999999996,"5min":69.513999999999996,"15min":296.47800000000001},"last":12.851000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":12.999000000000001,"5min":14.625999999999999,"15min":17.956},"min":{"1min":6.0300000000000002,"5min":6.0300000000000002,"15min":6.0300000000000002},"max":{"1min":36.156999999999996,"5min":36.156999999999996,"15min":47.926000000000002},"last":11.77},{"interface":"front","average":{"1min":17.372,"5min":13.675000000000001,"15min":15.026999999999999},"min":{"1min":11.651,"5min":5.3979999999999997,"15min":5.3979999999999997},"max":{"1min":34.609000000000002,"5min":34.609000000000002,"15min":66.176000000000002},"last":13.574}]},{"osd":4,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":19.289999999999999,"5min":17.210000000000001,"15min":19.231999999999999},"min":{"1min":11.885,"5min":9.7210000000000001,"15min":9.7210000000000001},"max":{"1min":41.923999999999999,"5min":41.923999999999999,"15min":67.248000000000005},"last":16.044},{"interface":"front","average":{"1min":11.738,"5min":10.41,"15min":12.5},"min":{"1min":6.4450000000000003,"5min":5.3300000000000001,"15min":5.3300000000000001},"max":{"1min":43.101999999999997,"5min":43.101999999999997,"15min":77.266999999999996},"last":9.7729999999999997}]},{"osd":6,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":14.673,"5min":12.750999999999999,"15min":15.19},"min":{"1min":7.4160000000000004,"5min":5.915,"15min":5.915},"max":{"1min":44.587000000000003,"5min":44.587000000000003,"15min":76.751999999999995},"last":14.932},{"interface":"front","average":{"1min":20.553000000000001,"5min":18.672000000000001,"15min":20.558},"min":{"1min":12.913,"5min":8.8390000000000004,"15min":8.8390000000000004},"max":{"1min":45.366,"5min":45.366,"15min":57.101999999999997},"last":16.609999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":17.077999999999999,"5min":15.398999999999999,"15min":14.606999999999999},"min":{"1min":10.875,"5min":7.5519999999999996,"15min":7.5519999999999996},"max":{"1min":29.731000000000002,"5min":29.965,"15min":29.965},"last":17.585000000000001},{"interface":"front","average":{"1min":13.878,"5min":12.369999999999999,"15min":12.048},"min":{"1min":9.3049999999999997,"5min":8.2829999999999995,"15min":8.2829999999999995},"max":{"1min":27.423999999999999,"5min":27.423999999999999,"15min":27.423999999999999},"last":7.133}]}]},{"osd":4,"up_from":15,"seq":64424509522,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":786432,"data_stored":281819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":9.5589999999999993,"5min":9.7349999999999994,"15min":12.707000000000001},"min":{"1min":6.0220000000000002,"5min":3.7429999999999999,"15min":3.7429999999999999},"max":{"1min":17.300999999999998,"5min":41.076000000000001,"15min":53.298999999999999},"last":11.803000000000001},{"interface":"front","average":{"1min":10.759,"5min":9.9280000000000008,"15min":10.686},"min":{"1min":4.9930000000000003,"5min":4.8140000000000001,"15min":3.9670000000000001},"max":{"1min":29.567,"5min":35.863,"15min":52.567},"last":7.835}]},{"osd":1,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":11.622,"5min":11.404999999999999,"15min":13.449999999999999},"min":{"1min":8.0660000000000007,"5min":5.8390000000000004,"15min":5.835},"max":{"1min":18.09,"5min":27.251999999999999,"15min":49.183},"last":9.4719999999999995},{"interface":"front","average":{"1min":12.454000000000001,"5min":12.068,"15min":14.85},"min":{"1min":6.6849999999999996,"5min":5.96,"15min":5.96},"max":{"1min":19.509,"5min":26.495999999999999,"15min":45.412999999999997},"last":5.9850000000000003}]},{"osd":2,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":15.634,"5min":12.238,"15min":20.167000000000002},"min":{"1min":7.4009999999999998,"5min":5.6390000000000002,"15min":5.6390000000000002},"max":{"1min":35.258000000000003,"5min":44.649000000000001,"15min":85.394000000000005},"last":8.8599999999999994},{"interface":"front","average":{"1min":17.91,"5min":14.865,"15min":19.396999999999998},"min":{"1min":7.5209999999999999,"5min":6.3639999999999999,"15min":6.3639999999999999},"max":{"1min":34.097999999999999,"5min":42.500999999999998,"15min":69.530000000000001},"last":12.576000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:33:14 2023","interfaces":[{"interface":"back","average":{"1min":17.870999999999999,"5min":15.256,"15min":14.686},"min":{"1min":8.0250000000000004,"5min":8.0039999999999996,"15min":7.008},"max":{"1min":37.561999999999998,"5min":37.561999999999998,"15min":37.561999999999998},"last":35.286999999999999},{"interface":"front","average":{"1min":15.468,"5min":13.134,"15min":14.765000000000001},"min":{"1min":7.3650000000000002,"5min":5.5549999999999997,"15min":5.5549999999999997},"max":{"1min":36.103999999999999,"5min":36.103999999999999,"15min":36.103999999999999},"last":32.893000000000001}]},{"osd":5,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":13.163,"5min":13.558,"15min":16.303000000000001},"min":{"1min":9.2739999999999991,"5min":5.4969999999999999,"15min":5.4969999999999999},"max":{"1min":20.664000000000001,"5min":53.840000000000003,"15min":78.798000000000002},"last":11.202},{"interface":"front","average":{"1min":18.699999999999999,"5min":17.393000000000001,"15min":18.565999999999999},"min":{"1min":11.209,"5min":8.0519999999999996,"15min":6.4450000000000003},"max":{"1min":27.582000000000001,"5min":52.781999999999996,"15min":55.814999999999998},"last":13.438000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":19.576000000000001,"5min":17.699000000000002,"15min":17.986999999999998},"min":{"1min":9.2100000000000009,"5min":8.2300000000000004,"15min":7.7850000000000001},"max":{"1min":41.723999999999997,"5min":51.149000000000001,"15min":51.149000000000001},"last":14.374000000000001},{"interface":"front","average":{"1min":15.221,"5min":14.712999999999999,"15min":16.190999999999999},"min":{"1min":9.6809999999999992,"5min":8.5139999999999993,"15min":7.8700000000000001},"max":{"1min":27.405000000000001,"5min":40.426000000000002,"15min":40.426000000000002},"last":16.048999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:33:14 2023","interfaces":[{"interface":"back","average":{"1min":15.782999999999999,"5min":17.422999999999998,"15min":18.308},"min":{"1min":8.093,"5min":8.093,"15min":8.093},"max":{"1min":30.408000000000001,"5min":60.390000000000001,"15min":60.390000000000001},"last":18.125},{"interface":"front","average":{"1min":20.154,"5min":21.510000000000002,"15min":22.376000000000001},"min":{"1min":15.909000000000001,"5min":13.69,"15min":13.69},"max":{"1min":29.472000000000001,"5min":57.606999999999999,"15min":57.606999999999999},"last":16.759}]}]},{"osd":7,"up_from":61,"seq":261993005097,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33000,"kb_used_data":796,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":815104,"data_stored":290955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4395,"internal_metadata":28634837},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":9.8659999999999997,"5min":10.045999999999999,"15min":10.109},"min":{"1min":4.4359999999999999,"5min":4.1059999999999999,"15min":4.1059999999999999},"max":{"1min":23.795000000000002,"5min":37.481999999999999,"15min":37.481999999999999},"last":18.253},{"interface":"front","average":{"1min":10.321999999999999,"5min":10.752000000000001,"15min":10.927},"min":{"1min":5.4909999999999997,"5min":4.2530000000000001,"15min":4.2530000000000001},"max":{"1min":29.928999999999998,"5min":38.475999999999999,"15min":38.475999999999999},"last":6.8680000000000003}]},{"osd":1,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":12.515000000000001,"5min":14.898999999999999,"15min":16.606000000000002},"min":{"1min":4.6929999999999996,"5min":4.6929999999999996,"15min":4.6929999999999996},"max":{"1min":30.350000000000001,"5min":64.765000000000001,"15min":64.765000000000001},"last":9.3650000000000002},{"interface":"front","average":{"1min":15.795,"5min":16.404,"15min":17.541},"min":{"1min":8.0419999999999998,"5min":7.0739999999999998,"15min":7.0739999999999998},"max":{"1min":34.390000000000001,"5min":65.994,"15min":65.994},"last":19.817}]},{"osd":2,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":16.215,"5min":16.803000000000001,"15min":17.611999999999998},"min":{"1min":8.734,"5min":7.9909999999999997,"15min":7.9909999999999997},"max":{"1min":36.622,"5min":46.478000000000002,"15min":46.478000000000002},"last":21.34},{"interface":"front","average":{"1min":14.967000000000001,"5min":14.003,"15min":14.08},"min":{"1min":6.5910000000000002,"5min":6.5910000000000002,"15min":6.5910000000000002},"max":{"1min":37.026000000000003,"5min":41.603999999999999,"15min":41.603999999999999},"last":23.727}]},{"osd":3,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":13.579000000000001,"5min":15.653,"15min":16.725000000000001},"min":{"1min":7.4950000000000001,"5min":6.8319999999999999,"15min":6.8319999999999999},"max":{"1min":45.731000000000002,"5min":60.463999999999999,"15min":60.463999999999999},"last":7.8879999999999999},{"interface":"front","average":{"1min":18.077000000000002,"5min":18.809000000000001,"15min":19.835000000000001},"min":{"1min":10.327,"5min":9.7370000000000001,"15min":9.7370000000000001},"max":{"1min":47.265999999999998,"5min":56.286000000000001,"15min":56.286000000000001},"last":27.667000000000002}]},{"osd":4,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":19.510999999999999,"5min":20.824999999999999,"15min":21.161000000000001},"min":{"1min":9.6180000000000003,"5min":8.4960000000000004,"15min":8.4960000000000004},"max":{"1min":50.741999999999997,"5min":57.716999999999999,"15min":57.716999999999999},"last":12.624000000000001},{"interface":"front","average":{"1min":15.906000000000001,"5min":18.274000000000001,"15min":18.507000000000001},"min":{"1min":7.3840000000000003,"5min":7.3840000000000003,"15min":7.3840000000000003},"max":{"1min":46.180999999999997,"5min":67.159000000000006,"15min":67.159000000000006},"last":7.2699999999999996}]},{"osd":5,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":14.403,"5min":14.342000000000001,"15min":14.606},"min":{"1min":7.7510000000000003,"5min":6.3399999999999999,"15min":6.3399999999999999},"max":{"1min":44.884,"5min":44.884,"15min":44.884},"last":24.658000000000001},{"interface":"front","average":{"1min":16.992999999999999,"5min":18.634,"15min":19.498000000000001},"min":{"1min":10.260999999999999,"5min":10.111000000000001,"15min":10.111000000000001},"max":{"1min":31.847999999999999,"5min":70.900000000000006,"15min":70.900000000000006},"last":11.122}]},{"osd":6,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":17.672000000000001,"5min":16.728999999999999,"15min":16.760000000000002},"min":{"1min":11.443,"5min":8.7170000000000005,"15min":8.7170000000000005},"max":{"1min":41.091000000000001,"5min":54.393000000000001,"15min":54.393000000000001},"last":22.789999999999999},{"interface":"front","average":{"1min":18.923999999999999,"5min":20.111000000000001,"15min":20.655999999999999},"min":{"1min":10.144,"5min":10.144,"15min":10.144},"max":{"1min":42.084000000000003,"5min":69.263000000000005,"15min":69.263000000000005},"last":21.969999999999999}]}]},{"osd":6,"up_from":0,"seq":60129542224,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:33:24.980 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:33:24.980 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:33:24.988 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:24.984+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:25.009 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:25.009 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:25.009 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:25.009 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:25.009 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:25.019 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:33:25.088 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:25.086+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:25.092 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:33:25.092 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:33:25.189 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:25.186+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:25.189 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:33:25.189 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:33:25.229 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:25.229 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:25.229 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:25.229 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:33:25.229 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.229 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:33:25.229 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:33:25.229 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 388.61376438799999, 2023-12-06T18:33:25.229 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:33:25.229 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.229 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.229 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.230 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.230 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.230 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:33:25.230 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.230 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.230 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.230 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.230 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:33:25.230 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.230 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.230 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.230 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.230 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:33:25.230 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:33:25.231 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.231 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.231 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.231 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:33:25.231 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:33:25.231 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.231 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.231 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.231 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:33:25.231 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:33:25.231 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.231 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.231 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.231 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:33:25.231 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:33:25.231 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.231 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.232 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.232 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:33:25.232 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:33:25.232 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.232 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.232 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.232 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:33:25.232 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:33:25.232 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.232 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.232 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.232 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:33:25.232 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:33:25.232 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.232 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.233 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.233 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:33:25.233 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:33:25.233 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.233 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.233 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.233 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.233 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.233 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:33:25.233 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:33:25.233 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 387.59093737900002, 2023-12-06T18:33:25.233 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:33:25.233 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.233 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.233 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.233 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.233 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.234 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:33:25.234 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.234 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.234 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.234 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.234 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:33:25.234 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.234 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.234 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.234 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.234 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:33:25.234 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:33:25.234 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.234 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.234 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.235 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:33:25.235 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:33:25.235 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.235 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.235 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.235 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:33:25.235 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:33:25.235 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.235 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.235 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.235 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:33:25.235 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:33:25.235 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.235 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.235 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.236 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:33:25.236 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:33:25.236 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.236 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.236 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.236 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:33:25.236 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:33:25.236 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.236 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.236 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.236 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:33:25.236 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:33:25.236 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.236 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.236 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.237 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:33:25.237 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:33:25.237 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.237 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.237 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.237 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.237 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.237 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:33:25.237 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:33:25.237 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 387.57889074100001, 2023-12-06T18:33:25.237 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:33:25.237 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.237 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.238 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.238 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.238 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.238 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:33:25.238 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.238 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.238 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.238 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.238 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:33:25.238 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.238 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.238 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.238 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.238 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:33:25.239 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:33:25.239 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.239 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.239 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.239 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:33:25.239 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:33:25.239 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.239 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.239 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.239 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:33:25.239 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:33:25.239 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.239 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.239 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.239 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:33:25.239 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:33:25.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.240 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.240 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.240 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:33:25.240 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:33:25.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.240 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.240 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.240 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:33:25.240 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:33:25.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.240 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.240 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.240 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:33:25.240 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:33:25.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.241 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.241 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.241 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:33:25.241 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:33:25.241 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.241 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.241 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.241 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.241 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.241 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:33:25.241 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:33:25.241 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 387.55749013399998, 2023-12-06T18:33:25.241 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:33:25.241 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.241 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.242 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.242 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.242 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.242 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:33:25.242 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.242 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.242 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.242 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:33:25.242 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.242 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.242 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.242 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.242 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:33:25.243 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:33:25.243 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.243 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.243 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.243 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:33:25.243 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:33:25.243 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.243 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.243 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.243 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:33:25.243 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:33:25.243 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.243 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.243 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.243 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:33:25.243 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:33:25.244 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.244 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.244 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.244 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:33:25.244 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:33:25.244 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.244 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.244 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.244 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:33:25.244 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:33:25.244 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.244 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.244 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.244 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:33:25.245 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:33:25.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.245 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.245 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.245 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:33:25.245 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:33:25.245 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.245 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.245 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.245 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.245 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.245 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:33:25.245 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:33:25.245 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 387.556310811, 2023-12-06T18:33:25.245 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:33:25.246 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.246 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.246 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.246 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.246 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.246 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:33:25.246 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.246 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.246 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.246 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:33:25.246 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.246 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.246 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.247 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.247 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:33:25.247 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:33:25.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.247 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.247 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.247 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:33:25.247 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:33:25.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.247 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.247 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.247 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:33:25.247 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:33:25.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.247 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.247 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.247 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:33:25.248 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:33:25.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.248 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.248 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.248 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:33:25.248 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:33:25.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.248 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.248 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.248 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:33:25.248 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:33:25.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.248 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.248 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.248 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:33:25.248 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:33:25.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.249 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.249 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.249 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:33:25.249 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:33:25.249 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.249 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.249 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.249 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.249 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.249 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:33:25.249 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:33:25.249 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 387.55573883, 2023-12-06T18:33:25.249 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:33:25.249 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.249 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.249 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.249 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.250 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.250 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:33:25.250 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.250 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.250 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.250 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.250 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:33:25.250 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.250 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.250 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.250 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.250 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:33:25.250 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:33:25.250 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.250 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.251 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.251 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:33:25.251 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:33:25.251 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.251 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.251 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.251 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:33:25.251 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:33:25.251 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.251 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.251 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.251 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:33:25.251 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:33:25.251 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.251 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.252 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.252 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:33:25.252 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:33:25.252 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.252 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.252 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.252 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:33:25.252 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:33:25.252 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.252 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.252 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.252 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:33:25.252 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:33:25.252 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.252 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.253 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.253 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:33:25.253 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:33:25.253 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.253 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.253 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.253 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.253 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.253 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:33:25.253 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:33:25.253 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 387.10990335999998, 2023-12-06T18:33:25.253 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:33:25.253 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.253 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.253 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.253 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.254 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.254 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:33:25.254 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.254 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.254 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.254 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.254 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:33:25.254 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.254 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.254 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.254 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.254 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:33:25.254 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:33:25.255 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.255 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.255 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.255 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:33:25.255 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:33:25.255 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.255 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.255 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.255 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:33:25.255 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:33:25.255 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.255 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.255 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.255 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:33:25.256 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:33:25.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.256 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.256 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.256 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:33:25.256 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:33:25.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.256 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.256 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.256 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:33:25.256 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:33:25.256 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.256 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.256 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.256 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:33:25.257 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:33:25.257 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.257 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.257 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.257 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:33:25.257 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:33:25.257 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.257 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.258 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.258 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.258 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.258 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:33:25.258 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:33:25.258 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 387.10830350399999, 2023-12-06T18:33:25.258 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:33:25.258 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.258 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.258 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.259 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.259 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.259 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:33:25.259 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.259 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.259 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.259 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.259 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:33:25.259 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.259 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.259 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.259 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.259 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:33:25.259 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:33:25.259 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.259 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.260 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.260 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:33:25.260 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:33:25.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.260 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.260 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.260 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:33:25.260 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:33:25.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.260 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.260 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.260 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:33:25.260 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:33:25.260 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.260 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.260 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.261 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:33:25.261 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:33:25.261 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.261 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.261 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.261 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:33:25.261 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:33:25.261 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.261 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.261 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.261 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:33:25.261 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:33:25.261 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.261 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.261 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.262 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:33:25.262 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:33:25.262 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.262 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.262 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.262 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.262 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.262 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:33:25.262 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:33:25.262 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 386.72909788800001, 2023-12-06T18:33:25.262 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:33:25.262 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.262 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.263 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.263 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.263 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.263 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:33:25.263 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.263 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.263 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.263 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.263 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:33:25.263 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.263 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.263 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.263 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.263 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:33:25.263 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:33:25.264 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.264 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.264 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.264 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:33:25.264 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:33:25.264 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.264 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.264 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.264 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:33:25.264 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:33:25.264 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.264 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.264 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.264 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:33:25.264 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:33:25.264 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.264 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.265 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.265 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:33:25.265 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:33:25.265 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.265 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.265 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.265 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:33:25.265 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:33:25.265 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.265 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.265 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.265 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:33:25.265 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:33:25.265 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.265 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.265 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.266 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:33:25.266 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:33:25.266 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.266 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.266 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.266 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.266 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.266 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:33:25.266 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:33:25.266 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 386.491595706, 2023-12-06T18:33:25.266 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:33:25.266 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.266 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.266 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.266 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.267 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.267 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:33:25.267 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.267 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.267 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.267 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.267 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:33:25.267 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.267 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.267 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.267 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.267 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:33:25.267 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:33:25.267 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.267 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.268 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.268 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:33:25.268 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:33:25.268 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.268 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.268 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.268 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:33:25.268 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:33:25.268 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.268 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.268 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.268 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:33:25.268 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:33:25.268 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.268 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.268 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.269 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:33:25.269 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:33:25.269 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.269 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.269 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.269 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:33:25.269 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:33:25.269 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.269 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.269 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.269 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:33:25.269 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:33:25.269 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.269 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.269 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.270 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:33:25.270 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:33:25.270 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.270 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.270 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.270 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.270 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.270 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:33:25.270 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:33:25.270 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 386.49086770700001, 2023-12-06T18:33:25.270 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:33:25.270 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.270 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.270 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.271 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.271 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.271 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:33:25.271 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.271 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.271 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.271 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:33:25.271 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.271 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.271 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.271 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:33:25.271 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:33:25.271 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.272 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.272 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.272 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:33:25.272 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:33:25.272 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.272 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.272 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.272 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:33:25.272 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:33:25.272 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.272 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.272 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.273 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:33:25.273 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:33:25.273 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.273 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.273 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.273 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:33:25.273 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:33:25.273 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.273 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.273 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.273 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:33:25.273 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:33:25.273 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.273 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.274 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.274 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:33:25.274 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:33:25.274 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.274 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.274 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.274 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:33:25.274 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:33:25.274 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.274 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.274 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.275 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.275 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.275 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:33:25.275 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:33:25.275 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 386.485855297, 2023-12-06T18:33:25.275 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:33:25.275 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.275 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.275 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.275 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.275 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.275 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:33:25.275 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.275 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.276 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.276 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.276 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:33:25.276 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.276 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.276 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.276 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:33:25.276 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:33:25.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.276 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.276 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.276 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:33:25.276 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:33:25.276 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.277 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.277 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.277 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:33:25.277 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:33:25.277 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.277 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.277 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.277 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:33:25.277 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:33:25.277 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.277 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.277 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.277 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:33:25.277 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:33:25.277 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.278 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.278 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.278 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:33:25.278 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:33:25.278 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.278 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.278 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.278 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:33:25.278 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:33:25.278 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.278 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.278 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.278 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:33:25.279 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:33:25.279 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.279 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.279 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.279 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.279 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:33:25.279 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:33:25.279 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 386.48032821800001, 2023-12-06T18:33:25.279 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:33:25.279 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.279 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.279 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.279 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.280 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.280 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:33:25.280 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.280 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.280 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:33:25.280 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.280 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.280 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:33:25.280 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:33:25.280 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.280 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.281 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:33:25.281 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:33:25.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.281 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:33:25.281 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:33:25.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.281 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:33:25.281 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:33:25.281 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.281 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.281 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:33:25.282 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:33:25.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.282 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.282 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:33:25.282 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:33:25.282 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.282 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.282 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.282 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:33:25.283 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:33:25.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.283 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.283 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.283 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:33:25.283 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:33:25.283 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.283 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.283 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.283 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.283 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.283 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:33:25.283 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:33:25.283 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 386.47981738099998, 2023-12-06T18:33:25.284 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:33:25.284 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.284 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.284 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.284 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.284 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.284 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:33:25.284 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.284 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.284 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.284 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.284 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:33:25.284 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.285 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.285 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:33:25.285 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:33:25.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.285 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.285 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:33:25.285 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:33:25.285 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.285 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.285 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:33:25.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:33:25.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:33:25.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:33:25.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.286 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.286 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:33:25.286 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:33:25.286 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.286 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:33:25.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:33:25.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:33:25.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:33:25.287 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.287 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.287 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.287 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:33:25.287 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:33:25.287 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.287 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.288 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.288 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.288 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:33:25.288 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:33:25.288 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 386.47935811999997, 2023-12-06T18:33:25.288 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:33:25.288 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.288 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.288 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.288 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.288 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.288 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:33:25.288 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.288 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.289 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.289 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.289 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:33:25.289 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.289 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.289 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.289 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:33:25.289 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:33:25.289 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.290 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.290 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:33:25.290 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:33:25.290 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.290 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.290 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:33:25.290 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:33:25.290 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.290 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.290 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:33:25.291 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:33:25.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:33:25.291 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:33:25.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.291 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:33:25.291 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:33:25.291 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.291 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.291 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.292 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:33:25.292 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:33:25.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.292 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.292 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.292 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:33:25.292 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:33:25.292 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.292 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.292 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.292 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.292 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.292 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:33:25.292 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:33:25.293 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 379.89944231300001, 2023-12-06T18:33:25.293 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:33:25.293 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.293 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.293 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.293 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.293 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.293 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:33:25.293 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.293 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.293 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.293 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.293 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:33:25.293 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.294 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.294 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.294 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.294 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:33:25.294 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:33:25.294 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.294 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.295 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.295 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:33:25.295 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:33:25.295 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.295 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.295 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.295 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:33:25.295 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:33:25.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.296 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.296 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.296 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:33:25.296 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:33:25.296 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.296 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.296 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.297 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:33:25.297 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:33:25.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.297 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.297 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.297 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:33:25.297 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:33:25.297 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.298 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.298 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.298 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:33:25.298 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:33:25.298 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.298 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.298 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.299 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:33:25.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:33:25.299 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.299 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.299 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.299 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.299 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:33:25.299 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:33:25.299 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 379.893204508, 2023-12-06T18:33:25.299 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:33:25.299 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.299 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.299 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.299 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:33:25.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:33:25.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.300 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:33:25.300 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:33:25.300 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.300 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.300 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.301 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:33:25.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:33:25.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.301 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.301 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:33:25.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:33:25.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.301 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.301 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:33:25.301 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:33:25.301 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.301 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.301 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.302 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:33:25.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:33:25.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.302 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.302 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:33:25.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:33:25.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.302 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.302 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:33:25.302 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:33:25.302 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.302 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.303 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.303 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:33:25.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:33:25.303 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.303 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.303 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.303 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.303 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.303 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:33:25.303 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:33:25.303 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 379.887141429, 2023-12-06T18:33:25.303 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:33:25.303 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.303 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.303 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.304 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:33:25.304 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.304 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:33:25.304 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.304 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.304 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:33:25.304 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:33:25.304 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.304 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.305 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.305 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:33:25.305 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:33:25.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.305 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.305 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.305 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:33:25.305 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:33:25.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.305 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.305 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.305 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:33:25.305 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:33:25.305 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.305 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.305 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.306 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:33:25.306 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:33:25.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.306 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.306 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.306 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:33:25.306 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:33:25.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.306 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.306 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.306 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:33:25.306 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:33:25.306 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.307 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.307 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:33:25.307 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:33:25.307 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.307 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.307 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.307 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.307 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.307 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:33:25.307 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:33:25.307 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 379.749754344, 2023-12-06T18:33:25.307 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:33:25.307 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.307 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.307 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.308 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.308 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:33:25.308 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.308 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.308 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:33:25.308 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.308 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.308 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:33:25.308 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:33:25.308 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.308 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.309 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.309 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:33:25.309 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:33:25.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.309 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.309 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.309 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:33:25.309 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:33:25.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.309 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.309 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.309 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:33:25.309 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:33:25.309 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.309 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.309 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.310 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:33:25.310 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:33:25.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.310 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.310 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.310 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:33:25.310 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:33:25.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.310 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.310 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.310 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:33:25.310 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:33:25.310 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.311 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.311 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:33:25.311 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:33:25.311 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.311 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.311 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.311 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.311 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.311 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:33:25.311 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:33:25.312 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 379.74308361700002, 2023-12-06T18:33:25.312 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:33:25.312 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:25.312 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:25.312 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:25.312 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.312 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:25.312 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:33:25.313 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.313 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:25.313 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:33:25.313 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:25.313 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.313 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.313 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:25.313 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:33:25.313 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:33:25.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.314 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:25.314 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:33:25.314 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:33:25.314 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.314 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.314 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:25.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:33:25.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:33:25.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:25.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:33:25.315 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:33:25.315 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.315 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.315 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:25.315 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:33:25.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:33:25.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.316 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:25.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:33:25.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:33:25.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.316 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.316 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:25.316 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:33:25.316 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:33:25.316 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:25.317 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:25.317 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:25.317 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:33:25.317 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:33:25.317 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.317 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.317 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.317 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:25.317 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:25.317 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:25.319 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:33:25.319 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:33:25.320 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:25.316+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:25.320 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:33:25.374 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '24 of 30' 2023-12-06T18:33:25.375 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:33:25.375 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:33:25.420 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:25.417+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:25.483 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:25.483 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:25.483 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:25.483 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:25.494 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:33:25.497 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:33:25.498 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:33:25.536 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:25.532+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:25.588 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:33:25.589 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:33:25.625 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:25.621+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:25.679 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:25.679 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:25.688 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:33:25.695 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:33:25.695 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:33:25.731 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:25.726+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:25.824 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:25.821+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:25.869 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:25.869 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:25.878 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:33:25.925 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:25.922+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:26.005 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:26.005 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:26.014 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:33:26.026 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:26.023+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:26.127 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:26.124+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:26.141 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:26.141 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:26.149 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:33:26.227 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:26.225+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:26.277 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:26.278 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:26.287 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:33:26.331 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:26.328+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:26.374 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-12-06T18:33:26.390 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '25 of 30' 2023-12-06T18:33:26.432 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:26.428+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:26.438 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:26.438 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:26.450 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:33:26.530 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:26.527+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:26.631 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:26.628+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:26.633 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:26.633 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:26.643 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:33:26.732 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:26.729+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:26.822 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:26.822 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:26.831 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:33:26.833 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:26.830+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:26.934 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:26.932+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:26.991 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:26.991 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:27.000 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:33:27.041 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:27.039+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:27.057 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is set 2023-12-06T18:33:27.136 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:27.133+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:27.182 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:27.182 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:27.191 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:33:27.236 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:27.233+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:27.337 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:27.335+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:27.369 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:27.369 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:27.379 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:33:27.438 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:27.435+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:27.452 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '26 of 30' 2023-12-06T18:33:27.539 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:27.537+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:27.559 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:27.559 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:27.570 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:33:27.656 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:27.653+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:27.707 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:33:27.742 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:27.738+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:27.747 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:27.747 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:27.757 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:33:27.843 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:27.840+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:27.943 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:27.941+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:27.955 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:27.955 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:27.966 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:33:27.976 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:33:27.976 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:33:28.045 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:28.041+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:28.088 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:33:28.088 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:33:28.130 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:28.130 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:28.143 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:33:28.145 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:28.142+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:28.193 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:33:28.193 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:33:28.247 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:28.244+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:28.291 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:33:28.292 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:33:28.331 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:28.332 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:28.332 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:28.332 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:28.332 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:28.341 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:33:28.379 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:33:28.379 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:33:28.380 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:28.377+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:28.449 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:28.446+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:28.482 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:33:28.483 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:33:28.528 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:28.528 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:28.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:28.528 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:28.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.528 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:33:28.528 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:28.528 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 247.425194323, 2023-12-06T18:33:28.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:33:28.529 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:28.529 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:28.529 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:28.529 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:28.529 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:28.529 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:33:28.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.529 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:28.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:28.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:28.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:28.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:28.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:28.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:28.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:28.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:33:28.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:33:28.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:28.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:33:28.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:33:28.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:28.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:33:28.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:33:28.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:28.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:33:28.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:33:28.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:28.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:33:28.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:33:28.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:28.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:33:28.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:33:28.532 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:28.532 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:28.533 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:28.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.533 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:33:28.533 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:28.533 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 187.406937021, 2023-12-06T18:33:28.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:33:28.533 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:28.533 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:28.533 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:28.533 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:28.533 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:28.533 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:33:28.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.534 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:28.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:28.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:28.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:28.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:28.534 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:28.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:28.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:28.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:33:28.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:33:28.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:28.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:33:28.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:33:28.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:28.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:33:28.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:33:28.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:28.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:33:28.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:33:28.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:28.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:33:28.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:33:28.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:28.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:33:28.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:33:28.537 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:28.537 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:28.537 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:28.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.537 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:11 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2023-12-06T18:33:28.537 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:28.537 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 67.382515904000002, 2023-12-06T18:33:28.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0086075449999999994, 2023-12-06T18:33:28.537 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:28.538 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:28.538 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:28.538 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:28.538 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:28.538 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 11 2023-12-06T18:33:28.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.538 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:28.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:28.538 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:28.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:28.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:28.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:28.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:28.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:28.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135441+0000", 2023-12-06T18:33:28.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.8695999999999999e-05 2023-12-06T18:33:28.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:28.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135650+0000", 2023-12-06T18:33:28.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020883699999999999 2023-12-06T18:33:28.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:28.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135863+0000", 2023-12-06T18:33:28.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000213931 2023-12-06T18:33:28.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:28.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.136054+0000", 2023-12-06T18:33:28.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000190416 2023-12-06T18:33:28.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:28.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.142056+0000", 2023-12-06T18:33:28.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0060019330000000001 2023-12-06T18:33:28.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:28.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:28.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:28.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.144030+0000", 2023-12-06T18:33:28.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0019737320000000002 2023-12-06T18:33:28.541 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:28.541 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:28.541 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:28.541 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:28.541 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:28.542 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:28.543 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:33:28.580 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:33:28.580 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:33:28.581 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:28.579+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:28.585 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '27 of 30' 2023-12-06T18:33:28.666 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:33:28.667 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:33:28.667 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:28.663+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:28.682 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:28.682 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:28.683 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:28.683 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:28.692 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:33:28.777 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:28.773+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:28.852 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:28.849+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:28.861 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:28.862 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:28.862 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:28.862 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:28.862 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:28.871 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:33:28.955 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:28.952+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:29.040 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:29.040 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:29.040 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:29.040 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:33:29.040 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.040 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:33:29.040 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:33:29.040 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 307.54690898699999, 2023-12-06T18:33:29.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:33:29.041 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.041 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:29.041 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:29.041 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:29.041 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:29.041 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:33:29.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.041 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.041 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.041 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.041 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:33:29.041 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.041 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:33:29.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:33:29.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:33:29.042 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.042 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.042 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:33:29.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:33:29.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:33:29.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:33:29.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:33:29.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:33:29.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:33:29.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:33:29.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:33:29.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:33:29.044 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.044 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.044 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.044 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.044 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.044 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:29.049 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:33:29.070 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-12-06T18:33:29.092 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:29.087+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:29.156 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:29.152+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:29.218 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:29.219 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:29.219 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:29.219 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:29.228 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:33:29.256 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:29.254+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:29.358 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:29.354+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:29.399 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:29.399 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:29.399 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:29.399 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:29.399 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:29.408 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:33:29.458 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:29.455+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:29.543 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '28 of 30' 2023-12-06T18:33:29.585 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:29.582+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:29.621 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:29.621 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:29.621 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:29.621 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:33:29.621 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.621 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:33:29.621 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:33:29.621 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 393.00553777099998, 2023-12-06T18:33:29.622 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:33:29.622 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.622 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.622 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.622 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.622 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.622 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:33:29.622 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.622 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.622 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.622 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:33:29.622 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.622 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:33:29.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:33:29.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:33:29.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:33:29.623 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.623 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.623 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.623 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:33:29.623 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:33:29.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.624 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.624 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.624 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:33:29.624 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:33:29.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.624 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.624 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.624 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:33:29.624 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:33:29.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.624 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.624 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.624 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:33:29.624 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:33:29.624 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.625 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.625 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:33:29.625 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:33:29.625 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.625 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.625 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:33:29.625 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:33:29.625 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.625 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.625 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.625 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.625 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.625 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:33:29.625 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:33:29.625 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 391.98271076200001, 2023-12-06T18:33:29.626 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:33:29.626 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.626 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.626 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.626 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.626 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:33:29.626 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.626 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.626 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:33:29.626 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.626 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.626 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.627 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.627 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:33:29.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:33:29.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.627 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.627 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:33:29.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:33:29.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.627 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.627 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:33:29.627 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:33:29.627 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.627 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.627 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:33:29.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:33:29.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:33:29.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:33:29.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:33:29.628 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:33:29.628 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.628 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.628 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.628 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:33:29.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:33:29.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.629 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.629 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:33:29.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:33:29.629 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.629 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.629 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.629 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.629 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.629 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:33:29.629 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:33:29.629 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 391.970664124, 2023-12-06T18:33:29.629 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:33:29.629 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.630 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.630 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.630 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:33:29.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.630 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.630 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:33:29.630 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.630 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.630 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:33:29.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:33:29.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:33:29.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:33:29.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.631 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:33:29.631 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:33:29.631 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.631 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.631 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.632 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:33:29.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:33:29.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.632 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.632 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:33:29.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:33:29.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.632 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.632 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:33:29.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:33:29.632 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.632 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.632 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.632 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:33:29.632 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:33:29.633 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.633 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.633 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:33:29.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:33:29.633 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.633 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.633 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.633 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.633 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.633 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:33:29.633 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:33:29.633 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 391.94926351700002, 2023-12-06T18:33:29.633 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:33:29.633 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.633 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.634 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:33:29.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:33:29.634 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.634 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.634 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.634 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.634 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:33:29.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:33:29.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:33:29.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:33:29.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:33:29.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:33:29.635 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.635 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.635 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.635 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:33:29.635 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:33:29.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:33:29.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:33:29.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:33:29.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:33:29.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.636 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.636 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.636 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:33:29.636 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:33:29.636 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.637 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.637 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.637 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:33:29.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:33:29.637 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.637 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.637 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.637 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.637 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.637 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:33:29.637 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:33:29.637 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 391.94808419399999, 2023-12-06T18:33:29.637 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:33:29.637 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.637 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.637 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.638 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:33:29.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.638 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:33:29.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.638 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.638 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.638 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:33:29.638 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:33:29.638 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:33:29.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:33:29.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:33:29.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:33:29.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.639 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.639 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:33:29.639 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:33:29.639 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.639 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:33:29.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:33:29.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:33:29.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:33:29.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.640 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:33:29.640 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:33:29.640 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.640 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.640 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.641 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:33:29.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:33:29.641 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.641 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.641 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.641 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.641 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:33:29.641 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:33:29.641 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 391.94751221299998, 2023-12-06T18:33:29.641 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:33:29.641 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.641 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.641 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.641 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.641 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:33:29.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:33:29.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.642 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:33:29.642 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:33:29.642 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.642 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.642 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:33:29.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:33:29.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.643 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:33:29.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:33:29.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.643 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:33:29.643 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:33:29.643 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.643 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.643 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.643 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:33:29.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:33:29.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:33:29.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:33:29.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:33:29.644 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:33:29.644 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.644 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.644 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.644 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:33:29.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:33:29.645 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.645 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.645 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.645 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.645 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:33:29.645 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:33:29.645 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 391.50167674300002, 2023-12-06T18:33:29.645 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:33:29.645 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.645 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.645 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.645 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.645 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.645 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:33:29.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:33:29.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.646 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.646 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:33:29.646 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:33:29.646 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.646 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:33:29.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:33:29.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:33:29.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:33:29.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.647 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:33:29.647 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:33:29.647 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.647 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.647 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:33:29.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:33:29.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.648 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:33:29.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:33:29.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.648 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:33:29.648 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:33:29.648 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.648 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.648 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.648 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:33:29.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:33:29.649 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.649 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.649 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.649 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.649 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.649 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:33:29.649 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:33:29.649 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 391.50007688699998, 2023-12-06T18:33:29.649 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:33:29.649 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.649 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.649 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.649 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.649 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.649 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:33:29.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.650 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:33:29.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.650 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:33:29.650 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:33:29.650 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.650 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.650 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.650 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:33:29.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:33:29.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:33:29.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:33:29.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:33:29.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:33:29.651 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.651 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.651 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.651 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:33:29.651 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:33:29.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:33:29.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:33:29.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:33:29.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:33:29.652 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.652 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.652 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.652 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:33:29.652 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:33:29.652 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.652 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.653 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.653 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:33:29.653 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:33:29.653 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 391.120871271, 2023-12-06T18:33:29.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:33:29.653 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.653 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.653 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.653 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.653 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.653 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:33:29.653 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.653 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.654 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:33:29.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.654 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:33:29.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:33:29.654 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.654 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.654 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.654 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:33:29.654 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:33:29.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:33:29.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:33:29.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:33:29.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:33:29.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.655 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.655 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.655 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:33:29.655 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:33:29.655 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:33:29.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:33:29.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:33:29.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:33:29.656 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.656 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.656 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.656 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:33:29.656 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:33:29.656 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.656 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.657 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.657 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.657 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:33:29.657 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:33:29.657 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 390.88336908899998, 2023-12-06T18:33:29.657 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:33:29.657 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.657 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.657 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.657 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.657 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:33:29.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:33:29.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.658 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:33:29.658 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:33:29.658 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.658 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.658 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:33:29.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:33:29.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:33:29.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:33:29.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:33:29.659 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:33:29.659 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.659 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.659 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.659 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:33:29.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:33:29.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:33:29.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:33:29.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:33:29.660 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:33:29.660 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.660 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.660 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.660 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:33:29.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:33:29.661 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.661 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.661 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.661 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.661 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:33:29.661 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:33:29.661 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 390.88264108999999, 2023-12-06T18:33:29.661 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:33:29.661 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.661 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.661 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.661 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.661 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.661 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:33:29.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:33:29.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:33:29.662 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:33:29.662 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.662 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.662 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.662 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:33:29.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:33:29.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:33:29.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:33:29.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:33:29.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:33:29.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.663 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.663 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.663 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:33:29.663 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:33:29.663 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:33:29.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:33:29.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:33:29.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:33:29.664 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.664 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.664 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.664 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:33:29.664 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:33:29.664 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.664 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.665 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.665 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:33:29.665 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:33:29.665 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 390.87762867999999, 2023-12-06T18:33:29.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:33:29.665 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.665 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.665 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.665 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.665 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:33:29.665 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.665 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.665 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:33:29.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:33:29.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:33:29.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.666 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.666 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.666 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:33:29.666 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:33:29.666 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:33:29.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:33:29.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:33:29.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:33:29.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.667 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.667 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:33:29.667 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:33:29.667 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.667 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:33:29.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:33:29.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:33:29.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:33:29.668 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.668 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.668 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.668 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:33:29.668 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:33:29.668 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.668 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.669 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.669 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:33:29.669 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:33:29.669 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 390.872101601, 2023-12-06T18:33:29.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:33:29.669 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.669 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.669 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.669 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:33:29.669 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.669 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.669 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.669 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:33:29.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:33:29.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:33:29.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.670 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.670 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:33:29.670 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:33:29.670 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.670 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:33:29.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:33:29.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:33:29.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:33:29.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.671 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:33:29.671 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:33:29.671 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.671 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.671 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:33:29.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:33:29.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:33:29.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:33:29.672 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.672 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.672 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.672 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:33:29.672 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:33:29.672 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.672 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.672 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.673 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:33:29.673 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:33:29.673 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 390.87159076400002, 2023-12-06T18:33:29.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:33:29.673 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.673 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.673 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.673 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:33:29.673 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.673 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.673 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.673 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:33:29.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:33:29.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:33:29.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.674 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:33:29.674 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:33:29.674 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.674 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.674 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:33:29.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:33:29.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:33:29.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:33:29.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:33:29.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:33:29.675 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.675 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.675 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.675 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:33:29.675 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:33:29.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:33:29.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:33:29.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.676 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.676 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:33:29.676 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:33:29.676 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.676 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.676 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.676 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.676 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.676 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:33:29.676 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:33:29.677 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 390.87113150300002, 2023-12-06T18:33:29.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:33:29.677 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.677 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.677 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.677 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.677 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.677 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:33:29.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.677 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.677 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.677 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.677 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:33:29.677 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:33:29.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:33:29.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:33:29.678 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:33:29.678 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.678 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.678 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.678 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:33:29.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:33:29.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:33:29.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:33:29.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:33:29.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:33:29.679 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.679 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.679 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.679 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:33:29.679 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:33:29.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:33:29.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:33:29.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.680 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.680 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:33:29.680 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:33:29.680 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.680 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.680 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.680 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.680 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.680 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:33:29.680 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:33:29.681 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 384.29121569599999, 2023-12-06T18:33:29.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:33:29.681 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.681 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.681 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.681 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.681 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:33:29.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.681 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.681 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:33:29.681 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.681 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.681 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.681 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:33:29.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:33:29.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:33:29.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:33:29.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.682 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:33:29.682 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:33:29.682 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.682 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.682 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:33:29.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:33:29.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:33:29.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:33:29.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:33:29.683 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:33:29.683 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.683 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.683 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.683 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:33:29.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:33:29.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.684 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.684 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:33:29.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:33:29.684 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.684 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.684 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.684 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.684 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.684 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:33:29.684 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:33:29.684 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 384.28497789099998, 2023-12-06T18:33:29.684 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:33:29.685 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.685 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.685 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:33:29.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.685 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:33:29.685 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.685 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.685 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.685 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.686 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:33:29.686 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:33:29.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.686 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.686 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.686 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:33:29.686 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:33:29.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.686 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.686 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.686 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:33:29.686 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:33:29.686 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.686 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.687 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.687 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:33:29.687 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:33:29.687 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.687 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.687 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.687 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:33:29.687 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:33:29.687 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.687 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.687 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.687 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:33:29.687 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:33:29.687 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.687 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.688 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.688 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:33:29.688 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:33:29.688 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.688 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.688 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.688 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:33:29.688 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:33:29.688 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.688 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.688 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.688 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.688 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.688 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:33:29.688 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:33:29.688 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 384.27891481199998, 2023-12-06T18:33:29.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:33:29.689 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.689 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.689 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.689 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:33:29.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.689 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.689 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:33:29.689 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.689 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.689 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.689 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:33:29.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:33:29.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:33:29.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:33:29.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.690 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.690 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.690 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:33:29.690 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:33:29.690 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:33:29.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:33:29.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:33:29.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:33:29.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.691 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.691 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.691 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:33:29.691 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:33:29.691 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:33:29.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:33:29.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.692 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.692 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:33:29.692 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:33:29.692 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.692 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.692 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.692 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.692 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.693 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:33:29.693 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:33:29.693 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 384.14152772699998, 2023-12-06T18:33:29.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:33:29.693 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.693 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.693 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.693 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:33:29.693 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.693 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.693 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.693 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.693 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:33:29.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:33:29.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:33:29.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:33:29.694 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:33:29.694 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.694 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.694 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.694 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:33:29.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:33:29.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:33:29.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:33:29.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:33:29.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:33:29.695 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.695 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.695 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.695 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:33:29.695 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:33:29.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:33:29.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:33:29.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.696 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.696 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.696 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:33:29.696 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:33:29.696 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.696 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.696 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.696 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.697 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:33:29.697 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:33:29.697 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 384.13485700000001, 2023-12-06T18:33:29.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:33:29.697 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:29.697 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:29.697 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:29.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:29.697 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:33:29.697 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.697 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.697 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.697 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:29.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:33:29.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:29.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.698 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:29.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:33:29.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:33:29.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.698 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:29.698 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:33:29.698 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:33:29.698 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.698 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:29.699 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:33:29.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:33:29.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:29.699 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:33:29.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:33:29.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:29.699 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:33:29.699 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:33:29.699 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.699 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.699 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:29.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:33:29.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:33:29.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:29.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:33:29.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:33:29.700 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:29.700 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:29.700 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:29.700 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:33:29.700 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:33:29.700 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.700 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.700 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.700 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:29.701 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:29.701 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:29.702 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:33:29.702 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:29.699+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:29.803 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:29.800+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:29.831 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:29.831 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:29.831 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:29.831 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:29.840 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:33:29.904 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:29.900+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:29.932 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 7, 5, 4] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:33:29.932 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:33:29.932 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2023-12-06T18:33:29.933 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2023-12-06T18:33:29.933 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2023-12-06T18:33:29.933 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2023-12-06T18:33:29.970 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:29.970 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:29.970 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:29.970 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:29.970 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:29.980 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:33:30.022 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:30.018+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:30.069 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:30.070 INFO:teuthology.orchestra.run.smithi152.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2023-12-06T18:33:30.070 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:30.106 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:30.104+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:30.150 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:30.150 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:30.150 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:30.150 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:33:30.151 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:30.158 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:33:30.207 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:30.204+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:30.286 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:30.286 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:30.286 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:30.286 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:30.295 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:33:30.336 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:30.333+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:30.410 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:30.407+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:30.468 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:30.468 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:30.468 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:30.468 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:30.468 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:30.478 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:33:30.511 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:30.508+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:30.610 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:30.610 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:30.610 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:30.610 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:33:30.610 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:30.610 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:33:30.610 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:30.610 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 188.94631079000001, 2023-12-06T18:33:30.611 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:33:30.611 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:30.611 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:30.611 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:30.611 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:30.611 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:30.611 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:33:30.611 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:30.611 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:30.611 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:30.611 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:30.611 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:30.611 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:30.611 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:30.611 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:30.612 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:30.612 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:30.612 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:30.612 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:30.612 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:30.612 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:30.612 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:33:30.612 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:33:30.612 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:30.612 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:30.612 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:30.612 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:33:30.612 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:33:30.612 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:30.612 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:30.613 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:30.613 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:33:30.613 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:33:30.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:30.613 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:30.613 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:30.613 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:33:30.613 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:33:30.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:30.613 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:30.613 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:30.613 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:33:30.613 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:33:30.613 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:30.613 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:30.614 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:30.614 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:33:30.614 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:33:30.614 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:30.614 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:30.614 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:30.614 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:30.614 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:30.614 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:12 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2023-12-06T18:33:30.614 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:30.614 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 68.796364165, 2023-12-06T18:33:30.614 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0032177460000000001, 2023-12-06T18:33:30.614 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:30.614 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:30.615 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:30.615 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:30.615 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:30.615 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 12 2023-12-06T18:33:30.615 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:30.615 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:30.615 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:30.615 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:30.615 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:30.615 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:30.615 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:30.615 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:30.615 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:30.615 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:30.616 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:30.616 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:30.616 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:30.616 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:30.616 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807030+0000", 2023-12-06T18:33:30.616 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6030000000000001e-05 2023-12-06T18:33:30.616 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:30.616 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:30.616 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:30.616 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807210+0000", 2023-12-06T18:33:30.616 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017936800000000001 2023-12-06T18:33:30.616 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:30.616 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:30.616 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:30.617 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807443+0000", 2023-12-06T18:33:30.617 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00023339699999999999 2023-12-06T18:33:30.617 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:30.617 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:30.617 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:30.617 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807614+0000", 2023-12-06T18:33:30.617 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000171032 2023-12-06T18:33:30.617 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:30.617 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:30.617 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:30.617 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.808696+0000", 2023-12-06T18:33:30.617 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010817190000000001 2023-12-06T18:33:30.617 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:30.617 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:30.617 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:30.617 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.810232+0000", 2023-12-06T18:33:30.618 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0015361999999999999 2023-12-06T18:33:30.618 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:30.618 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:30.618 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:30.618 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:30.618 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:30.618 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:30.619 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:33:30.620 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:30.617+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:30.648 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is unset 2023-12-06T18:33:30.675 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '29 of 30' 2023-12-06T18:33:30.679 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:33:30.720 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:30.717+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:30.795 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:30.796 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:30.796 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:30.796 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:30.809 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:33:30.821 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:30.818+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:30.922 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:30.919+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:30.989 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:33:30.989 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:33:31.023 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:31.021+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:31.027 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:31.027 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:31.036 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:33:31.082 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:33:31.082 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:33:31.124 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:31.121+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:31.190 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:33:31.190 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:33:31.225 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:31.222+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:31.237 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:31.237 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:31.248 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:33:31.296 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:33:31.297 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:33:31.325 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:31.322+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:31.427 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:31.427 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:31.436 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:33:31.446 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:33:31.448 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:31.445+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:31.527 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:31.524+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:31.543 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:33:31.543 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:33:31.589 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:31.589 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:31.598 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:33:31.633 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:33:31.633 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:33:31.634 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:31.630+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:31.677 INFO:tasks.mon_recovery:adding mon c back in 2023-12-06T18:33:31.678 INFO:tasks.ceph.mon.c:Restarting daemon 2023-12-06T18:33:31.678 DEBUG:teuthology.orchestra.run.smithi061:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2023-12-06T18:33:31.680 INFO:tasks.ceph.mon.c:Started 2023-12-06T18:33:31.680 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2023-12-06T18:33:31.680 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:33:31.729 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:31.725+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:31.744 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:33:31.744 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:33:31.782 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:31.785 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:31.793 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:33:31.830 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:31.826+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:31.931 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:31.927+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:31.975 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:31.976 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:31.985 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:33:32.032 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:32.028+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:32.133 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:32.129+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:32.202 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:32.202 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:32.213 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:33:32.233 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:32.230+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:32.334 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:32.331+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:32.364 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":44,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":35,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:33:32.401 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:32.401 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:32.410 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:33:32.436 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:32.432+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:32.536 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:32.533+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:32.589 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:32.590 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:32.599 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:33:32.643 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:32.639+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:32.666 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-12-06T18:33:32.738 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:32.735+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:32.763 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:32.763 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:32.774 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:33:32.839 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:32.836+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:32.916 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:32.916 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:32.925 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:33:32.967 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:32.963+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:33.044 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:33.040+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:33.057 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:33.057 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:33.066 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:33:33.142 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:33.138+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:33.234 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:33.234 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:33.243 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:33:33.244 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:33.239+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:33.344 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:33.341+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:33.373 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:33.373 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:33.382 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:33:33.445 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:33.441+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:33.546 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:33.542+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:33.553 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:33.554 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:33.562 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:33:33.647 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:33.644+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:33.692 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:33.692 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:33.701 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:33:33.763 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:33:33.786 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:33.782+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:33.850 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:33.846+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:33.870 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:33.870 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:33.879 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:33:33.964 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:33.960+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:34.016 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:33:34.016 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:33:34.048 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:34.049 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:34.051 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:34.048+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:34.058 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:33:34.109 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:33:34.109 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:33:34.152 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:34.149+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:34.206 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:33:34.206 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:33:34.230 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:34.230 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:34.240 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:33:34.245 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is unset 2023-12-06T18:33:34.281 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:34.278+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:34.308 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:33:34.308 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:33:34.354 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:34.350+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:34.409 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:33:34.409 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:33:34.415 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:34.416 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:34.425 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:33:34.455 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:34.451+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:34.506 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:33:34.507 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:33:34.555 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:34.552+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:34.556 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:34.556 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:34.565 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:33:34.594 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:33:34.594 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:33:34.656 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:34.652+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:34.682 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:33:34.682 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:33:34.720 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:34.720 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:34.720 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:34.720 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:34.720 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:34.729 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:33:34.772 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:34.769+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:34.858 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:34.854+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:34.872 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:34.872 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:34.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:34.872 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:33:34.872 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:34.872 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:33:34.872 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:34.872 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 193.20850842300001, 2023-12-06T18:33:34.872 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:33:34.872 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:34.872 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:34.873 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:34.873 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:34.873 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:34.873 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:33:34.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:34.873 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:34.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:34.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:34.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:34.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:34.873 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:34.873 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:34.873 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:34.873 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:34.873 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:34.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:34.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:34.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:34.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:33:34.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:33:34.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:34.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:34.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:34.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:33:34.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:33:34.874 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:34.874 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:34.874 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:34.874 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:33:34.874 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:33:34.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:34.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:34.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:34.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:33:34.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:33:34.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:34.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:34.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:34.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:33:34.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:33:34.875 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:34.875 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:34.875 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:34.875 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:33:34.875 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:33:34.875 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:34.876 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:34.876 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:34.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:34.876 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:34.876 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:12 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2023-12-06T18:33:34.876 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:34.876 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 73.058561797999999, 2023-12-06T18:33:34.876 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0032177460000000001, 2023-12-06T18:33:34.876 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:34.876 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:34.876 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:34.876 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:34.876 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:34.876 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 12 2023-12-06T18:33:34.876 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:34.877 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:34.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:34.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:34.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:34.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:34.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:34.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:34.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:34.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:34.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:34.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:34.877 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:34.877 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:34.877 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807030+0000", 2023-12-06T18:33:34.877 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6030000000000001e-05 2023-12-06T18:33:34.877 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:34.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:34.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:34.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807210+0000", 2023-12-06T18:33:34.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017936800000000001 2023-12-06T18:33:34.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:34.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:34.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:34.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807443+0000", 2023-12-06T18:33:34.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00023339699999999999 2023-12-06T18:33:34.878 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:34.878 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:34.878 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:34.878 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807614+0000", 2023-12-06T18:33:34.878 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000171032 2023-12-06T18:33:34.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:34.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:34.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:34.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.808696+0000", 2023-12-06T18:33:34.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010817190000000001 2023-12-06T18:33:34.879 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:34.879 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:34.879 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:34.879 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.810232+0000", 2023-12-06T18:33:34.879 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0015361999999999999 2023-12-06T18:33:34.879 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:34.879 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:34.879 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:34.879 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:34.879 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:34.879 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:34.882 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:33:34.958 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:34.954+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:35.026 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:35.026 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:35.027 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:35.027 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:35.036 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:33:35.059 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:35.056+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:35.079 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 7, 5, 4] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:33:35.079 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:33:35.079 INFO:tasks.thrashosds.thrasher:primary_affinity 2023-12-06T18:33:35.080 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 0.000000 2023-12-06T18:33:35.080 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 0 2023-12-06T18:33:35.160 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:35.157+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:35.214 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:35.214 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:35.214 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:35.214 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:35.215 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:35.224 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:33:35.261 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:35.258+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:35.362 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:35.359+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:35.372 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:35.372 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:35.372 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:35.372 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:35.372 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.372 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 110/109 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 23'75, data_included: [0~4096,65536~66064], data_size: 70160, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@23'75, size: 131600, copy_subset: [(0, 131600)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 131600, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:33:35.372 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:33:21.824095+0000", 2023-12-06T18:33:35.372 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 13.540384538, 2023-12-06T18:33:35.372 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.22522905200000001, 2023-12-06T18:33:35.372 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:35.373 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:33:35.373 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:35.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.373 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:35.373 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.824095+0000", 2023-12-06T18:33:35.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.373 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.373 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:35.373 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.824095+0000", 2023-12-06T18:33:35.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.373 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:35.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.824111+0000", 2023-12-06T18:33:35.374 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5596000000000001e-05 2023-12-06T18:33:35.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:35.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.866577+0000", 2023-12-06T18:33:35.374 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.042466773999999999 2023-12-06T18:33:35.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:35.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.871496+0000", 2023-12-06T18:33:35.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049188699999999997 2023-12-06T18:33:35.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:35.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.886281+0000", 2023-12-06T18:33:35.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.014784535 2023-12-06T18:33:35.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:35.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.903661+0000", 2023-12-06T18:33:35.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.017379950000000002 2023-12-06T18:33:35.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:35.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.908970+0000", 2023-12-06T18:33:35.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.005309322 2023-12-06T18:33:35.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:35.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.049324+0000", 2023-12-06T18:33:35.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.140354005 2023-12-06T18:33:35.376 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.376 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:35.376 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.376 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 110/109 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:33:35.377 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:33:22.129048+0000", 2023-12-06T18:33:35.377 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 13.235431352000001, 2023-12-06T18:33:35.377 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.029478353999999998, 2023-12-06T18:33:35.377 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:35.377 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:33:35.377 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:35.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.377 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:35.377 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.129048+0000", 2023-12-06T18:33:35.377 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.377 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.377 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:35.377 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.129048+0000", 2023-12-06T18:33:35.378 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.378 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.378 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.378 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:35.378 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.129058+0000", 2023-12-06T18:33:35.378 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 9.8630000000000003e-06 2023-12-06T18:33:35.378 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.378 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.378 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:35.378 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.133519+0000", 2023-12-06T18:33:35.379 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0044613229999999997 2023-12-06T18:33:35.379 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.379 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.379 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:35.379 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.133935+0000", 2023-12-06T18:33:35.379 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00041585999999999999 2023-12-06T18:33:35.379 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.379 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.379 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:35.379 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.134363+0000", 2023-12-06T18:33:35.379 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00042782200000000001 2023-12-06T18:33:35.379 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.379 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.379 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:35.380 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.136423+0000", 2023-12-06T18:33:35.380 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002059759 2023-12-06T18:33:35.380 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.380 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.380 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:35.380 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.136759+0000", 2023-12-06T18:33:35.380 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000335898 2023-12-06T18:33:35.380 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.380 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.380 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:35.380 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.158527+0000", 2023-12-06T18:33:35.380 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021767828999999999 2023-12-06T18:33:35.380 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.380 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:35.380 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.381 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.381 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:35.381 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:35.382 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:33:35.383 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:33:35.463 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:35.460+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:35.524 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:35.524 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:35.524 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:35.525 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:35.534 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:33:35.579 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:35.575+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:35.665 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:35.662+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:35.672 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:35.672 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:35.672 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:35.672 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:35.673 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:35.682 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:33:35.766 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:35.763+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:35.867 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:35.864+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:35.928 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:35.928 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:35.928 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:35.928 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:35.928 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.928 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:33:35.928 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:35.928 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 400.51417366999999, 2023-12-06T18:33:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:33:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:33:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.929 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:33:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:33:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:33:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:33:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.930 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:33:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:33:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:33:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:33:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:33:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:33:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.931 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:33:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:33:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:33:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:33:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:33:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:33:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:33:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:33:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:33:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:33:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:33:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:33:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:33:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:33:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:33:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:33:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:33:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.934 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 398.40560043099998, 2023-12-06T18:33:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:33:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:33:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:35.935 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:33:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:33:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:35.936 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:33:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:33:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:33:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:33:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:33:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:33:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:35.937 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:33:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:33:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:33:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:33:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:33:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:33:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:35.938 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:33:35.939 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:33:35.939 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.939 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.939 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:35.939 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:33:35.939 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:33:35.939 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.939 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.939 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:35.939 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:33:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:33:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:33:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:33:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 398.404855199, 2023-12-06T18:33:35.940 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:33:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:33:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.941 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:35.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:35.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:35.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:33:35.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:33:35.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.942 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:35.942 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:33:35.942 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:33:35.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:33:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:33:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:33:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:33:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:33:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:33:35.943 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:33:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:33:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:33:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:33:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:33:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:33:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.944 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:33:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:33:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:33:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:33:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:33:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:33:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.945 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 398.40417144399999, 2023-12-06T18:33:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:33:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:33:35.946 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:33:35.947 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:33:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:33:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:33:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:33:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:33:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:33:35.948 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:33:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:33:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:33:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:33:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:33:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:33:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:33:35.949 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:33:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:33:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:33:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:33:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:33:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:33:35.950 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:33:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:33:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 398.39859818899998, 2023-12-06T18:33:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:33:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:35.951 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:33:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:35.952 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:33:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:33:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:33:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:33:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:35.953 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:33:35.954 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:33:35.954 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.954 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.954 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:35.954 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:33:35.954 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:33:35.954 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.954 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.954 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:35.954 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:33:35.954 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:33:35.954 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.954 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:33:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:33:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:33:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:33:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:33:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:33:35.955 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:33:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:33:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:33:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:33:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:33:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:33:35.956 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.957 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:35.957 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.957 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.957 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.957 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:35.957 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:35.957 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 398.39792454899998, 2023-12-06T18:33:35.957 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:33:35.957 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:35.957 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:35.957 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:35.957 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:35.957 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:35.957 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:33:35.957 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.958 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:35.958 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.958 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:35.958 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:35.958 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.958 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.958 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.958 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:35.958 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:35.958 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.958 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.958 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.958 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:35.958 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:33:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:33:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:33:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:33:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:33:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:33:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:35.959 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:33:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:33:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:33:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:33:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:33:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:33:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.960 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:33:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:33:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:33:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:33:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:33:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:33:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.961 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:33:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:33:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:33:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:33:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:35.962 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 398.39726235000001, 2023-12-06T18:33:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:33:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:33:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.963 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:33:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:33:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:33:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:33:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.964 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:33:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:33:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:33:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:33:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:33:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:33:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.965 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:33:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:33:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:33:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:33:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:33:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:33:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.966 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:33:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:33:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:33:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:33:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:33:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:33:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.967 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 398.32455291500003, 2023-12-06T18:33:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:33:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:33:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.968 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:33:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:33:35.969 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:33:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:33:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:33:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:33:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:33:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:33:35.970 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:33:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:33:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:33:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:33:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:33:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:33:35.971 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:33:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:33:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:33:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:33:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:33:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:33:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.972 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:33:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:33:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 398.32387120599998, 2023-12-06T18:33:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:33:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:35.973 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:35.974 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:35.974 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:35.974 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:33:35.974 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.974 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:35.974 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.974 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:35.974 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:35.974 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.974 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.974 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.974 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:35.974 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:35.974 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:33:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:33:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:33:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:33:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:33:35.975 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:33:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:33:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:33:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:33:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:33:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:33:35.976 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:33:35.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.977 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.977 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:35.977 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:33:35.977 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:33:35.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.977 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.977 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:35.977 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:33:35.977 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:33:35.977 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.977 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.977 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:35.977 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:33:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:33:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:33:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:33:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:33:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:33:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.978 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 397.24506711800001, 2023-12-06T18:33:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:33:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:33:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.979 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:33:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:33:35.980 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:33:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:33:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:33:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:33:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:33:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:33:35.981 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:33:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:33:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:33:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:33:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:33:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:33:35.982 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:33:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:33:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:33:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:33:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:33:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:33:35.983 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:33:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:33:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 397.24365054399999, 2023-12-06T18:33:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:33:35.984 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:33:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:35.985 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:33:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:33:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:33:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:33:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:35.986 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:33:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:33:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:33:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:33:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:33:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:33:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:35.987 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:33:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:33:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:33:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:33:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:33:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:33:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:35.988 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:33:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:33:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:33:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:33:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:33:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:33:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.989 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 397.24256541900002, 2023-12-06T18:33:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:33:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:33:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:35.990 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:33:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:33:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:35.991 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:33:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:33:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:33:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:33:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:33:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:33:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.992 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:33:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:33:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:33:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:33:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:33:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:33:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:35.993 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:33:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:33:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:33:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:33:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:33:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:33:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.994 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:33:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:33:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 397.20741631700002, 2023-12-06T18:33:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:33:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:35.995 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:33:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.996 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:33:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:33:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:33:35.997 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:33:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:33:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:33:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:33:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:33:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:35.998 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:33:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:33:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:33:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:33:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:33:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:33:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:35.999 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:33:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:33:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:33:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:33:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:33:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:33:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:36.000 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:33:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:33:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 390.657685792, 2023-12-06T18:33:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:33:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:36.001 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:33:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.002 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:33:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:33:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:33:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:33:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:33:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:33:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.003 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:33:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:33:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:33:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:33:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:33:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:33:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.004 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:33:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:33:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:33:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:33:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:33:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:33:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.005 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:33:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:33:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:33:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:33:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:36.006 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 390.65688018399999, 2023-12-06T18:33:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:33:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:33:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:36.007 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:33:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:33:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:33:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:33:36.008 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:33:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:33:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:33:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:33:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:33:36.009 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:33:36.010 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.010 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.010 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:36.010 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:33:36.010 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:33:36.010 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.010 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.010 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:36.010 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:33:36.010 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:33:36.010 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.010 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.010 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:36.010 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:33:36.011 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:33:36.011 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.011 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.011 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:36.011 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:33:36.011 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:33:36.011 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.011 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.011 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:36.011 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:33:36.011 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:33:36.011 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.011 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.011 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:36.011 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:33:36.012 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:33:36.012 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.012 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:36.012 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.012 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.012 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.012 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:36.012 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:36.012 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 390.656189375, 2023-12-06T18:33:36.012 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:33:36.012 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:36.012 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:36.012 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:36.012 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:36.012 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:36.013 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:33:36.013 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.013 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:36.013 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.013 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:36.013 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:36.013 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:36.013 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.013 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.013 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:36.013 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:36.013 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:36.013 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.013 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.013 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:36.014 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:33:36.014 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:33:36.014 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.014 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.014 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:36.014 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:33:36.014 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:33:36.014 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.014 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.014 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:36.014 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:33:36.014 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:33:36.014 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.014 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:33:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:33:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:33:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:33:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:33:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:33:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.015 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:33:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:33:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:33:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:33:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:33:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:33:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:36.016 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:33:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:33:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:33:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:33:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 390.65189186700002, 2023-12-06T18:33:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:33:36.018 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:36.018 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:36.018 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:36.018 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:36.018 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:36.018 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:33:36.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.018 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:36.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.018 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:36.018 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:36.018 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:36.018 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.018 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:36.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:36.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:36.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:36.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:33:36.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:33:36.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.019 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.019 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:36.019 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:33:36.019 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:33:36.019 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:33:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:33:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:33:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:33:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:33:36.020 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:33:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:33:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:33:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:33:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:33:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:33:36.021 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:33:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:33:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:33:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:33:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:33:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:36.022 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:33:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:33:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 390.65122419199997, 2023-12-06T18:33:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:33:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:36.023 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:33:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:36.024 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:33:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:33:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:33:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:33:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:33:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:33:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:36.025 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:33:36.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:33:36.026 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.026 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.026 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:36.026 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:33:36.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:33:36.026 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.026 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.026 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:36.026 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:33:36.026 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:33:36.026 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.026 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.026 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:33:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:33:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:33:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:33:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:33:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:33:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.027 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:33:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:33:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:33:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:33:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:36.028 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 390.65056876400001, 2023-12-06T18:33:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:33:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:33:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.029 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:33:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:33:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:33:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:33:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.030 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:33:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:33:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:33:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:33:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:33:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:33:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.031 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:33:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:33:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:33:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:33:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:33:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:33:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.032 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:33:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:33:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:33:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:33:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:33:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:33:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.033 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 390.60991421699998, 2023-12-06T18:33:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:33:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:33:36.034 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:33:36.035 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:33:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:33:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:33:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:33:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:33:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:33:36.036 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:33:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:33:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:33:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:33:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:33:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:33:36.037 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:33:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:33:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:33:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:33:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:33:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:33:36.038 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:33:36.039 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.039 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.039 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:36.039 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:33:36.039 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:33:36.039 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.039 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:36.039 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.039 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.039 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:36.039 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:36.041 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:33:36.041 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:36.037+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:36.108 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":44,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":39,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:33:36.141 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:36.138+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:36.181 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:36.181 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:36.181 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:36.191 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:33:36.262 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-12-06T18:33:36.270 INFO:teuthology.orchestra.run.smithi061.stderr:set osd.7 primary-affinity to 0 (802) 2023-12-06T18:33:36.278 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:36.274+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:36.344 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:36.340+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:36.389 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:36.389 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:36.389 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:36.389 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:36.389 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:36.399 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:33:36.445 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:36.442+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:36.546 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:36.543+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:36.582 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:36.582 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:36.582 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:36.582 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:36.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.582 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:33:36.582 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 374.76412216400001, 2023-12-06T18:33:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:33:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:33:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:36.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:33:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:33:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:33:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:33:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:33:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:33:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:33:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:33:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:33:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:33:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.585 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:36.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:33:36.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:33:36.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:36.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:36.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:36.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:33:36.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:33:36.586 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.586 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:36.586 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.586 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:36.587 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:36.587 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:36.592 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:33:36.647 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:36.644+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:36.695 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:33:36.749 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:36.745+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:36.820 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:36.820 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:36.821 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:36.821 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:36.832 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:33:36.877 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:36.873+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:36.950 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:36.946+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:36.996 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:36.997 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:36.997 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:36.997 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:36.997 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:37.009 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:33:37.039 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:33:37.039 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:33:37.084 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:37.080+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:37.147 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:33:37.147 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:33:37.157 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is set 2023-12-06T18:33:37.176 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:37.172+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:37.196 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:33:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 256.09530029000001, 2023-12-06T18:33:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:33:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:37.197 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:33:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.198 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:33:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:33:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:33:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:33:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:33:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:33:37.199 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:33:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:33:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:33:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:33:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:33:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:33:37.200 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:33:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 196.07704298799999, 2023-12-06T18:33:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:33:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:33:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.201 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:33:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:33:37.202 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:33:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:33:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:33:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:33:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:33:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:33:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.203 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:33:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:33:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:33:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:33:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:11 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2023-12-06T18:33:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:37.204 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 76.052621870999999, 2023-12-06T18:33:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0086075449999999994, 2023-12-06T18:33:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 11 2023-12-06T18:33:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.205 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135441+0000", 2023-12-06T18:33:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.8695999999999999e-05 2023-12-06T18:33:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135650+0000", 2023-12-06T18:33:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020883699999999999 2023-12-06T18:33:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.206 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135863+0000", 2023-12-06T18:33:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000213931 2023-12-06T18:33:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.136054+0000", 2023-12-06T18:33:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000190416 2023-12-06T18:33:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.142056+0000", 2023-12-06T18:33:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0060019330000000001 2023-12-06T18:33:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:37.207 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.144030+0000", 2023-12-06T18:33:37.208 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0019737320000000002 2023-12-06T18:33:37.208 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:37.208 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:37.208 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:37.208 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:37.208 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:37.208 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:37.209 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:33:37.254 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:37.250+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:37.276 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:33:37.276 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:33:37.340 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:37.341 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:37.341 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:37.341 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:37.349 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:33:37.375 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:33:37.376 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:33:37.403 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:37.400+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:37.454 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:37.451+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:37.465 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:33:37.465 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:33:37.520 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:37.520 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:37.530 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:33:37.553 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:33:37.554 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:33:37.572 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:37.568+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:37.644 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:33:37.645 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:33:37.657 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:37.653+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:37.701 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:37.701 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:37.710 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:33:37.732 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:33:37.733 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:33:37.758 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:37.754+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:37.844 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:37.844 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:37.854 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:33:37.896 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:37.892+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:37.960 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:37.956+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:38.026 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:38.027 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:38.035 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:33:38.060 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:38.057+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:38.162 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:38.158+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:38.203 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:38.203 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:38.212 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:33:38.262 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:38.258+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:38.363 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:38.360+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:38.384 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:38.384 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:38.394 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:33:38.464 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:38.461+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:38.523 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:38.523 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:38.533 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:33:38.574 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:38.571+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:38.687 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:38.683+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:38.701 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:38.701 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:38.710 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:33:38.767 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:38.764+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:38.868 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:38.865+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:38.880 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:38.880 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:38.890 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:33:38.969 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:38.966+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:39.019 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:39.020 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:39.029 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:33:39.074 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:39.070+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:39.122 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:33:39.158 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:39.158 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:39.167 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:33:39.175 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-12-06T18:33:39.223 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:39.212+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:39.275 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:39.271+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:39.339 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:39.339 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:39.348 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:33:39.378 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:39.372+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:39.476 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:39.473+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:39.528 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:39.528 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:39.538 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:33:39.577 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:39.574+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:39.678 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:39.675+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:39.717 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:39.717 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:39.726 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:33:39.745 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:33:39.779 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:39.775+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:39.800 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":44,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":42,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:33:39.880 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:39.876+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:39.936 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:39.936 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:39.945 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:33:39.988 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:39.983+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:40.082 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:40.079+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:40.117 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:33:40.117 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:33:40.159 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:40.159 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:40.169 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:33:40.183 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:40.180+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:40.224 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:33:40.224 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:33:40.284 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:40.280+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:40.356 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:33:40.356 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:33:40.373 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:40.373 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:40.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:40.373 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:40.373 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.373 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:33:40.373 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:40.373 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 404.96023615500002, 2023-12-06T18:33:40.373 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:33:40.373 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.373 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.374 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.374 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.374 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.374 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:33:40.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.374 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:40.374 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.374 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.374 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.374 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.374 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:40.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:33:40.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:33:40.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:33:40.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:33:40.375 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.375 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.375 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.375 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:33:40.375 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:33:40.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:33:40.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:33:40.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:33:40.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:33:40.376 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.376 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.376 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:33:40.376 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:33:40.376 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:33:40.377 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.377 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.377 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:33:40.377 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:33:40.377 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.377 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.377 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:33:40.377 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:33:40.377 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.377 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.377 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.377 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:33:40.377 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:33:40.378 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.378 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.378 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.378 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:33:40.378 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:33:40.378 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.378 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.378 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.378 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:33:40.378 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:33:40.378 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.378 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.378 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.379 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:33:40.379 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:33:40.379 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.379 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.379 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.379 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:33:40.379 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:33:40.379 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.379 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.379 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.379 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.379 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.379 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:40.379 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:40.379 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 402.85166291600001, 2023-12-06T18:33:40.379 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:33:40.379 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.380 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.380 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.380 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.380 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.380 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:33:40.380 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.380 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.380 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.380 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.380 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:40.380 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.380 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.380 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.380 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.380 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:40.381 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.381 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.381 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.381 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.381 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:33:40.381 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:33:40.381 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.381 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.381 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.381 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:33:40.381 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:33:40.381 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.381 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.381 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.381 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:33:40.381 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:33:40.381 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.382 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.382 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.382 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:33:40.382 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:33:40.382 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.382 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.382 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.382 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:33:40.382 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:33:40.382 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.382 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.382 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.382 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:33:40.382 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:33:40.382 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.383 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.383 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.383 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:33:40.383 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:33:40.383 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.383 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.383 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.383 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:33:40.383 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:33:40.383 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.383 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.383 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.383 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:33:40.383 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:33:40.383 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.383 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.383 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:33:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:33:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:33:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:33:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 402.85091768400002, 2023-12-06T18:33:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:33:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.384 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.385 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.385 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.385 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.385 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:33:40.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.385 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.385 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.385 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.385 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:40.385 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.385 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.385 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.385 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.385 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:40.386 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.386 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.386 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.386 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.386 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:33:40.386 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:33:40.386 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.386 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.386 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.386 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:33:40.386 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:33:40.386 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.386 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.386 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.387 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:33:40.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:33:40.387 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.387 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.387 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.387 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:33:40.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:33:40.387 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.387 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.387 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.387 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:33:40.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:33:40.387 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.387 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.387 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.387 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:33:40.387 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:33:40.388 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.388 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.388 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.388 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:33:40.388 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:33:40.388 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.388 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.388 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.388 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:33:40.388 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:33:40.388 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.388 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.388 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.388 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:33:40.388 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:33:40.388 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.389 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.389 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.389 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:33:40.389 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:33:40.389 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.389 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.389 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.389 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:33:40.389 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:33:40.389 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.389 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.389 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.389 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.389 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.389 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:40.390 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:40.390 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 402.85023392900001, 2023-12-06T18:33:40.390 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:33:40.390 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.390 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.390 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.390 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.390 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.390 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:33:40.390 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.390 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.390 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.390 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.390 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:40.390 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.391 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.391 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.391 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.391 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:40.391 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.391 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.391 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.391 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.391 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:33:40.391 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:33:40.391 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.391 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.391 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.391 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:33:40.391 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:33:40.391 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.391 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.392 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.392 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:33:40.392 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:33:40.392 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.392 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.392 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.392 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:33:40.392 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:33:40.392 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.392 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.392 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.392 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:33:40.392 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:33:40.392 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.392 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.392 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.393 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:33:40.393 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:33:40.393 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.393 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.393 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.393 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:33:40.393 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:33:40.393 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.393 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.393 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.393 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:33:40.393 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:33:40.393 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.393 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.393 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.393 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:33:40.393 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:33:40.393 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.394 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.394 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.394 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:33:40.394 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:33:40.394 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.394 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.394 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.394 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:33:40.394 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:33:40.394 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.394 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.394 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.394 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.394 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.394 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:40.395 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:40.395 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 402.84466067400001, 2023-12-06T18:33:40.395 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:33:40.395 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.395 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.395 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.395 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.395 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.395 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:33:40.395 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.395 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.395 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.395 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.395 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:40.395 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.395 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.395 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.396 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.396 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:40.396 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.396 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.396 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.396 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.396 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:33:40.396 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:33:40.396 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.396 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.396 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.396 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:33:40.396 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:33:40.396 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.396 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.397 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.397 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:33:40.397 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:33:40.397 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.397 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.397 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.397 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:33:40.397 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:33:40.397 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.397 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.397 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.397 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:33:40.397 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:33:40.397 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.397 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.397 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.397 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:33:40.398 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:33:40.398 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.398 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.398 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.398 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:33:40.398 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:33:40.398 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.398 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.398 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.398 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:33:40.398 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:33:40.398 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.398 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.398 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.399 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:33:40.399 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:33:40.399 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.399 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.399 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.399 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:33:40.399 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:33:40.399 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.399 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.399 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.399 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:33:40.399 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:33:40.399 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.399 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.399 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.399 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.400 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.400 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:40.400 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:40.400 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 402.84398703400001, 2023-12-06T18:33:40.400 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:33:40.400 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.400 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.400 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.400 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.400 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.400 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:33:40.400 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.400 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.400 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.400 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.401 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:40.401 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.401 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.401 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.401 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.401 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:40.401 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.401 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.401 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.401 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.401 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:33:40.401 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:33:40.401 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.401 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.402 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.402 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:33:40.402 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:33:40.402 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.402 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.402 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.402 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:33:40.402 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:33:40.402 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.402 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.402 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.402 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:33:40.402 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:33:40.402 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.403 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.403 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:33:40.403 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:33:40.403 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.403 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.403 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:33:40.403 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:33:40.403 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.403 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.403 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:33:40.403 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:33:40.403 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.403 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.404 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:33:40.404 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:33:40.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.404 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.404 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:33:40.404 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:33:40.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.404 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.404 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:33:40.404 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:33:40.404 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.404 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.404 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.404 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:33:40.405 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:33:40.405 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.405 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.405 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.405 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.405 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.405 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:40.405 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:40.405 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 402.84332483499998, 2023-12-06T18:33:40.405 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:33:40.405 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.405 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.405 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.405 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.405 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.405 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:33:40.405 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.406 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.406 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.406 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.406 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:40.406 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.406 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.406 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.406 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.406 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:40.406 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.406 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.406 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.406 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.406 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:33:40.406 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:33:40.406 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.407 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:33:40.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:33:40.407 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.407 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:33:40.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:33:40.407 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.407 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.407 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.407 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:33:40.407 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:33:40.407 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.408 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.408 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:33:40.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:33:40.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.408 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.408 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:33:40.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:33:40.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.408 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.408 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:33:40.408 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:33:40.408 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.408 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.408 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.409 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:33:40.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:33:40.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.409 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.409 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.409 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:33:40.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:33:40.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.409 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.409 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.409 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:33:40.409 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:33:40.409 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.409 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.409 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.409 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:33:40.410 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:33:40.410 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.410 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.410 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.410 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.410 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.410 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:40.410 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:40.410 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 402.7706154, 2023-12-06T18:33:40.410 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:33:40.410 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.410 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.410 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.411 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.411 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.411 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:33:40.411 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.411 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.411 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.411 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.411 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:40.411 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.411 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.411 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.411 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.411 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:40.411 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.411 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:33:40.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:33:40.412 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:33:40.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:33:40.412 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.412 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.412 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:33:40.412 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:33:40.412 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.412 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.413 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.413 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:33:40.413 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:33:40.413 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.413 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.413 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.413 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:33:40.413 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:33:40.413 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.413 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.413 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.413 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:33:40.413 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:33:40.413 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.413 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.414 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.414 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:33:40.414 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:33:40.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.414 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.414 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.414 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:33:40.414 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:33:40.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.414 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.414 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.414 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:33:40.414 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:33:40.414 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.414 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.415 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.415 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:33:40.415 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:33:40.415 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.415 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.415 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.415 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:33:40.415 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:33:40.415 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.415 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.415 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.415 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.415 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.415 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:40.415 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:40.416 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 402.76993369100001, 2023-12-06T18:33:40.416 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:33:40.416 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.416 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.416 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.416 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.416 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.416 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:33:40.416 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.416 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.416 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.416 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.416 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:40.416 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.416 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.417 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.417 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.417 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:40.417 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.417 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.417 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.417 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.417 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:33:40.417 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:33:40.417 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.417 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.417 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.417 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:33:40.417 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:33:40.417 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.418 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.418 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.418 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:33:40.418 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:33:40.418 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.418 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.418 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.418 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:33:40.418 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:33:40.418 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.418 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.418 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.418 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:33:40.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:33:40.419 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.419 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.419 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.419 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:33:40.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:33:40.419 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.419 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.419 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.419 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:33:40.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:33:40.419 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.419 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.419 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.419 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:33:40.419 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:33:40.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:33:40.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:33:40.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:33:40.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:33:40.420 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.420 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.420 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.420 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:33:40.420 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:33:40.421 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.421 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.421 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.421 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.421 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:40.421 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:40.421 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 401.69112960299998, 2023-12-06T18:33:40.421 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:33:40.421 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.421 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.421 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.421 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.421 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.421 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:33:40.421 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.422 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.422 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.422 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.422 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:40.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.422 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.422 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.422 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.422 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:40.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.422 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.422 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.422 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.422 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:33:40.422 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:33:40.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.423 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.423 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:33:40.423 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:33:40.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.423 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.423 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:33:40.423 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:33:40.423 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.423 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.423 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.423 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:33:40.423 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:33:40.424 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.424 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.424 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:33:40.424 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:33:40.424 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.424 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.424 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:33:40.424 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:33:40.424 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.424 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.424 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.424 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:33:40.424 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:33:40.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:33:40.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:33:40.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:33:40.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:33:40.425 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.425 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.425 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.425 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:33:40.425 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:33:40.426 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.426 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.426 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.426 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:33:40.426 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:33:40.426 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.426 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.426 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.426 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.426 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.426 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:40.426 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:40.426 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 401.68971302900002, 2023-12-06T18:33:40.427 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:33:40.427 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.427 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.427 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.427 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.427 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.427 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:33:40.427 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.427 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.427 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.427 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.427 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:40.427 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.427 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.427 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.427 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.428 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:40.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.428 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.428 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.428 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:33:40.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:33:40.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.428 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.428 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.428 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:33:40.428 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:33:40.428 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.428 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.429 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.429 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:33:40.429 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:33:40.429 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.429 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.429 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.429 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:33:40.429 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:33:40.429 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.429 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.429 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.429 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:33:40.429 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:33:40.429 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.430 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.430 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.430 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:33:40.430 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:33:40.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.430 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.430 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.430 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:33:40.430 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:33:40.430 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.430 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.430 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.430 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:33:40.430 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:33:40.431 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.431 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.431 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.431 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:33:40.431 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:33:40.431 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.431 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.431 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.431 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:33:40.431 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:33:40.431 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.431 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.431 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.431 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:33:40.431 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:33:40.431 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.432 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.432 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.432 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.432 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.432 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:40.432 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:40.432 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 401.68862790399999, 2023-12-06T18:33:40.432 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:33:40.432 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.432 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.432 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.432 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.432 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.432 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:33:40.432 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.432 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.433 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.433 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.433 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:40.433 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.433 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.433 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.433 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.433 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:40.433 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.433 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.433 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.433 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.433 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:33:40.433 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:33:40.433 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.433 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.434 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.434 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:33:40.434 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:33:40.434 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.434 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.434 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.434 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:33:40.434 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:33:40.434 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.434 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.434 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.434 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:33:40.434 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:33:40.434 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.434 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.435 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.435 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:33:40.435 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:33:40.435 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.435 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.435 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.435 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:33:40.435 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:33:40.435 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.435 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.435 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.435 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:33:40.435 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:33:40.435 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.435 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.436 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.436 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:33:40.436 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:33:40.436 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.436 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.436 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:33:40.436 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:33:40.436 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.436 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.436 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:33:40.436 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:33:40.436 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.436 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.437 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.437 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:33:40.437 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:33:40.437 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.437 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.437 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.437 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.437 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.437 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:40.437 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:40.437 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 401.653478802, 2023-12-06T18:33:40.437 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:33:40.437 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.437 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.437 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.437 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.438 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.438 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:33:40.438 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.438 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.438 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.438 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.438 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:40.438 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.438 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.438 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.438 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.438 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:40.438 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.438 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.438 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.439 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.439 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:33:40.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:33:40.439 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.439 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.439 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.439 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:33:40.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:33:40.439 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.439 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.439 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.439 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:33:40.439 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:33:40.439 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.439 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.439 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.440 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:33:40.440 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:33:40.440 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.440 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.440 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.440 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:33:40.440 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:33:40.440 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.440 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.440 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.440 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:33:40.440 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:33:40.440 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.440 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.440 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:33:40.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:33:40.441 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:33:40.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:33:40.441 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.441 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:33:40.441 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:33:40.441 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.441 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.441 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.442 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:33:40.442 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:33:40.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.442 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.442 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.442 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:33:40.442 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:33:40.442 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.442 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.442 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.442 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.442 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.442 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:40.442 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:40.443 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 395.10374827700002, 2023-12-06T18:33:40.443 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:33:40.443 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.443 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.443 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.443 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.443 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.443 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:33:40.443 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.443 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.443 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.443 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.443 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:40.443 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.444 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.444 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:40.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.444 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.444 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:33:40.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:33:40.444 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.444 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.444 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.444 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:33:40.444 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:33:40.445 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.445 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.445 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.445 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:33:40.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:33:40.445 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.445 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.445 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.445 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:33:40.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:33:40.445 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.445 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.445 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.445 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:33:40.445 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:33:40.446 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.446 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.446 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:33:40.446 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:33:40.446 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.446 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.446 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:33:40.446 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:33:40.446 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.446 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.446 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.446 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:33:40.447 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:33:40.447 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.447 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.447 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.447 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:33:40.447 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:33:40.447 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.447 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.447 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.447 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:33:40.447 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:33:40.447 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.447 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.447 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.447 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:33:40.448 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:33:40.448 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.448 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.448 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.448 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.448 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.448 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:40.448 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:40.448 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 395.10294266900002, 2023-12-06T18:33:40.448 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:33:40.448 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.448 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.448 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.448 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.448 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.449 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:33:40.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.449 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:40.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:40.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:33:40.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:33:40.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.450 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.450 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:33:40.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:33:40.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.450 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.450 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:33:40.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:33:40.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.451 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.451 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:33:40.451 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:33:40.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.451 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.451 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:33:40.451 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:33:40.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.451 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.451 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:33:40.451 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:33:40.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:33:40.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:33:40.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:33:40.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:33:40.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:33:40.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:33:40.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:33:40.453 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:33:40.453 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.453 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.453 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.453 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:33:40.453 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:33:40.453 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.453 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.453 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.453 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.453 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.453 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:40.453 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:40.453 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 395.10225186000002, 2023-12-06T18:33:40.453 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:33:40.453 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.453 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.454 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.454 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.454 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.454 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:33:40.454 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.454 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.454 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.454 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.454 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:40.454 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.454 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.454 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.454 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.454 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:40.454 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:33:40.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:33:40.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:33:40.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:33:40.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:33:40.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:33:40.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.456 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:33:40.456 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:33:40.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.456 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:33:40.456 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:33:40.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.456 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:33:40.456 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:33:40.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:33:40.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:33:40.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:33:40.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:33:40.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:33:40.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:33:40.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.458 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.458 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.458 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:33:40.458 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:33:40.458 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.458 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.458 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.458 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:33:40.458 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:33:40.458 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.458 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.458 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.458 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.458 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.458 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:40.459 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:40.459 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 395.09795435199999, 2023-12-06T18:33:40.459 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:33:40.459 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.459 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.459 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.459 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.459 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.459 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:33:40.459 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.459 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.459 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.459 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.459 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:40.459 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:40.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:33:40.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:33:40.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:33:40.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:33:40.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:33:40.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:33:40.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:33:40.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:33:40.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:33:40.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:33:40.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:33:40.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:33:40.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:33:40.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:33:40.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:33:40.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:33:40.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:33:40.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:33:40.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:33:40.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:33:40.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:33:40.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:33:40.464 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.464 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.464 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.464 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.464 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:40.464 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:40.464 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 395.097286677, 2023-12-06T18:33:40.464 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:33:40.464 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.464 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.464 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.464 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.464 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.464 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:33:40.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.465 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:40.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:40.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:33:40.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:33:40.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:33:40.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:33:40.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:33:40.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:33:40.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:33:40.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:33:40.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:33:40.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:33:40.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:33:40.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:33:40.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:33:40.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:33:40.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:33:40.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:33:40.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:33:40.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:33:40.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:33:40.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:33:40.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:33:40.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:33:40.469 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.469 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.469 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.469 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:40.469 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:40.469 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 395.09663124899998, 2023-12-06T18:33:40.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:33:40.469 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.470 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.470 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.470 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.470 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.470 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:33:40.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.470 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:40.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:40.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:33:40.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:33:40.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:33:40.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:33:40.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:33:40.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:33:40.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:33:40.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:33:40.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:33:40.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:33:40.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:33:40.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:33:40.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:33:40.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:33:40.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:33:40.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:33:40.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:33:40.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:33:40.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:33:40.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:33:40.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:33:40.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:33:40.474 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.474 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.474 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.475 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:40.475 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:40.475 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 395.05597670200001, 2023-12-06T18:33:40.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:33:40.475 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.475 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:40.475 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:40.475 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:40.475 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:40.475 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:33:40.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.475 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:40.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:40.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:33:40.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:33:40.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:33:40.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:33:40.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:33:40.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:33:40.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:33:40.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:33:40.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.477 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:33:40.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:33:40.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.478 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.478 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:33:40.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:33:40.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.478 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:40.478 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:33:40.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:33:40.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.478 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:40.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:33:40.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:33:40.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:40.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:33:40.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:33:40.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:40.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:33:40.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:33:40.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:33:40.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:33:40.480 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.480 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.480 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.480 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.480 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.480 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:40.481 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:33:40.482 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:40.479+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:40.486 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:33:40.486 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:33:40.528 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:08.778 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:33:40.576 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:33:40.576 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:33:40.582 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:40.578+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:40.619 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:40.619 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:40.619 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:40.620 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:40.628 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:33:40.662 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:33:40.663 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:33:40.683 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:40.680+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:40.748 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:33:40.749 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:33:40.769 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:09.019 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:33:40.784 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:40.781+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:40.809 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:40.809 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:40.809 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:40.809 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:40.809 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:40.818 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:33:40.843 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:33:40.843 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:33:40.899 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:40.895+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:40.986 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:40.982+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:40.994 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:40.995 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:40.995 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:40.995 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:40.995 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.995 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:33:40.995 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:40.995 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 379.17889346200002, 2023-12-06T18:33:40.995 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:33:40.995 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:40.995 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:33:40.995 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:40.995 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.995 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:40.996 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:40.996 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.996 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.996 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.996 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:40.996 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:40.996 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:40.996 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.996 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.996 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:40.996 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:33:40.996 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:33:40.996 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.996 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.996 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:40.997 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:33:40.997 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:33:40.997 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.997 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.997 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:40.997 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:33:40.997 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:33:40.997 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.997 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.997 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:40.997 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:33:40.997 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:33:40.997 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.997 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.997 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:40.998 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:33:40.998 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:33:40.998 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.998 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.998 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:40.998 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:33:40.998 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:33:40.998 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:40.998 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:40.998 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:40.998 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:33:40.998 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:33:40.998 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.998 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.999 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.999 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:40.999 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:40.999 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:41.004 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:33:41.087 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:41.083+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:41.107 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is set 2023-12-06T18:33:41.188 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:41.184+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:41.191 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:41.191 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:41.191 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:41.191 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:41.200 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:33:41.289 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:41.286+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:41.289 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 7, 5, 4] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:33:41.290 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:33:41.290 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2023-12-06T18:33:41.290 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2023-12-06T18:33:41.390 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:41.387+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:41.398 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:41.398 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:41.398 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:41.398 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:41.398 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:41.407 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:33:41.491 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:41.487+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:41.575 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:41.575 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:41.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:41.576 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:41.576 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.576 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:33:41.576 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:41.576 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 260.469318864, 2023-12-06T18:33:41.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:33:41.576 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:41.576 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:41.576 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:41.576 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:41.576 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:41.576 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:33:41.576 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.577 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:41.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.577 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:41.577 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:41.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:41.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.577 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:41.577 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:41.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:41.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.577 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:41.577 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:33:41.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:33:41.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:41.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:33:41.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:33:41.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:41.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:33:41.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:33:41.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:41.578 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:33:41.578 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:33:41.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.579 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.579 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:41.579 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:33:41.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:33:41.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.579 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.579 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:41.579 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:33:41.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:33:41.579 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.579 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.579 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.579 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.580 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:33:41.580 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:41.580 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 200.45106156200001, 2023-12-06T18:33:41.580 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:33:41.580 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:41.580 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:41.580 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:41.580 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:41.580 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:41.580 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:33:41.580 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.580 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:41.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.580 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:41.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:41.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:41.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.581 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.581 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:41.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:41.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:41.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.581 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.581 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:41.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:33:41.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:33:41.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.581 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.582 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:41.582 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:33:41.582 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:33:41.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.582 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:41.582 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:33:41.582 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:33:41.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.582 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:41.582 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:33:41.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:33:41.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:41.583 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:33:41.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:33:41.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:41.583 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:33:41.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:33:41.583 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.583 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.583 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.584 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:11 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2023-12-06T18:33:41.584 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:41.584 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 80.426640445000004, 2023-12-06T18:33:41.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0086075449999999994, 2023-12-06T18:33:41.584 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:41.584 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:41.584 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:41.584 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:41.584 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:41.584 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 11 2023-12-06T18:33:41.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.584 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:41.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.584 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:41.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:41.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:41.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.585 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.585 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:41.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:41.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:41.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.585 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.585 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:41.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135441+0000", 2023-12-06T18:33:41.585 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.8695999999999999e-05 2023-12-06T18:33:41.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.585 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.585 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:41.585 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135650+0000", 2023-12-06T18:33:41.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020883699999999999 2023-12-06T18:33:41.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:41.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135863+0000", 2023-12-06T18:33:41.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000213931 2023-12-06T18:33:41.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:41.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.136054+0000", 2023-12-06T18:33:41.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000190416 2023-12-06T18:33:41.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:41.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.142056+0000", 2023-12-06T18:33:41.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0060019330000000001 2023-12-06T18:33:41.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:41.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.144030+0000", 2023-12-06T18:33:41.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0019737320000000002 2023-12-06T18:33:41.587 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.587 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.587 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.587 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.587 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.587 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:41.588 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:33:41.630 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:41.626+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:41.693 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:41.689+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:41.717 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:41.717 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:41.717 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:41.717 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:41.726 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:33:41.810 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:41.807+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:41.894 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:41.891+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:41.899 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:33:41.900 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:41.900 INFO:teuthology.orchestra.run.smithi061.stdout: "epoch": 119, 2023-12-06T18:33:41.900 INFO:teuthology.orchestra.run.smithi061.stdout: "fsid": "497e3283-cc16-40ff-a72c-a7487474fb4b", 2023-12-06T18:33:41.900 INFO:teuthology.orchestra.run.smithi061.stdout: "created": "2023-12-06T18:26:07.122862+0000", 2023-12-06T18:33:41.900 INFO:teuthology.orchestra.run.smithi061.stdout: "modified": "2023-12-06T18:33:40.903828+0000", 2023-12-06T18:33:41.900 INFO:teuthology.orchestra.run.smithi061.stdout: "last_up_change": "2023-12-06T18:30:04.956811+0000", 2023-12-06T18:33:41.900 INFO:teuthology.orchestra.run.smithi061.stdout: "last_in_change": "2023-12-06T18:32:58.826891+0000", 2023-12-06T18:33:41.900 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-06T18:33:41.900 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_num": 5806080, 2023-12-06T18:33:41.900 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_set": [ 2023-12-06T18:33:41.900 INFO:teuthology.orchestra.run.smithi061.stdout: "nodeep-scrub", 2023-12-06T18:33:41.900 INFO:teuthology.orchestra.run.smithi061.stdout: "noscrub", 2023-12-06T18:33:41.900 INFO:teuthology.orchestra.run.smithi061.stdout: "pglog_hardlimit", 2023-12-06T18:33:41.900 INFO:teuthology.orchestra.run.smithi061.stdout: "purged_snapdirs", 2023-12-06T18:33:41.901 INFO:teuthology.orchestra.run.smithi061.stdout: "recovery_deletes", 2023-12-06T18:33:41.901 INFO:teuthology.orchestra.run.smithi061.stdout: "sortbitwise" 2023-12-06T18:33:41.901 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:33:41.901 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_version": 8, 2023-12-06T18:33:41.901 INFO:teuthology.orchestra.run.smithi061.stdout: "full_ratio": 0.94999998807907104, 2023-12-06T18:33:41.901 INFO:teuthology.orchestra.run.smithi061.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-12-06T18:33:41.901 INFO:teuthology.orchestra.run.smithi061.stdout: "nearfull_ratio": 0.85000002384185791, 2023-12-06T18:33:41.901 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_snapshot": "", 2023-12-06T18:33:41.901 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_max": 2, 2023-12-06T18:33:41.901 INFO:teuthology.orchestra.run.smithi061.stdout: "max_osd": 8, 2023-12-06T18:33:41.901 INFO:teuthology.orchestra.run.smithi061.stdout: "require_min_compat_client": "luminous", 2023-12-06T18:33:41.901 INFO:teuthology.orchestra.run.smithi061.stdout: "min_compat_client": "luminous", 2023-12-06T18:33:41.901 INFO:teuthology.orchestra.run.smithi061.stdout: "require_osd_release": "reef", 2023-12-06T18:33:41.901 INFO:teuthology.orchestra.run.smithi061.stdout: "allow_crimson": false, 2023-12-06T18:33:41.902 INFO:teuthology.orchestra.run.smithi061.stdout: "pools": [ 2023-12-06T18:33:41.902 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.902 INFO:teuthology.orchestra.run.smithi061.stdout: "pool": 1, 2023-12-06T18:33:41.902 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_name": ".mgr", 2023-12-06T18:33:41.902 INFO:teuthology.orchestra.run.smithi061.stdout: "create_time": "2023-12-06T18:26:52.648377+0000", 2023-12-06T18:33:41.902 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-12-06T18:33:41.902 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_names": "hashpspool", 2023-12-06T18:33:41.902 INFO:teuthology.orchestra.run.smithi061.stdout: "type": 1, 2023-12-06T18:33:41.902 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 2, 2023-12-06T18:33:41.902 INFO:teuthology.orchestra.run.smithi061.stdout: "min_size": 1, 2023-12-06T18:33:41.902 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_rule": 0, 2023-12-06T18:33:41.902 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_count": 0, 2023-12-06T18:33:41.902 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_target": 0, 2023-12-06T18:33:41.902 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_barrier": 0, 2023-12-06T18:33:41.903 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-06T18:33:41.903 INFO:teuthology.orchestra.run.smithi061.stdout: "object_hash": 2, 2023-12-06T18:33:41.903 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscale_mode": "off", 2023-12-06T18:33:41.903 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num": 1, 2023-12-06T18:33:41.903 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num": 1, 2023-12-06T18:33:41.903 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num_target": 1, 2023-12-06T18:33:41.903 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_target": 1, 2023-12-06T18:33:41.903 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_pending": 1, 2023-12-06T18:33:41.903 INFO:teuthology.orchestra.run.smithi061.stdout: "last_pg_merge_meta": { 2023-12-06T18:33:41.903 INFO:teuthology.orchestra.run.smithi061.stdout: "source_pgid": "0.0", 2023-12-06T18:33:41.903 INFO:teuthology.orchestra.run.smithi061.stdout: "ready_epoch": 0, 2023-12-06T18:33:41.903 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_started": 0, 2023-12-06T18:33:41.903 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_clean": 0, 2023-12-06T18:33:41.903 INFO:teuthology.orchestra.run.smithi061.stdout: "source_version": "0'0", 2023-12-06T18:33:41.904 INFO:teuthology.orchestra.run.smithi061.stdout: "target_version": "0'0" 2023-12-06T18:33:41.904 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.904 INFO:teuthology.orchestra.run.smithi061.stdout: "last_change": "19", 2023-12-06T18:33:41.904 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend": "0", 2023-12-06T18:33:41.904 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-06T18:33:41.904 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_preluminous": "0", 2023-12-06T18:33:41.904 INFO:teuthology.orchestra.run.smithi061.stdout: "auid": 0, 2023-12-06T18:33:41.904 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_mode": "selfmanaged", 2023-12-06T18:33:41.904 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_seq": 0, 2023-12-06T18:33:41.904 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_epoch": 0, 2023-12-06T18:33:41.904 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_snaps": [], 2023-12-06T18:33:41.904 INFO:teuthology.orchestra.run.smithi061.stdout: "removed_snaps": "[]", 2023-12-06T18:33:41.904 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_bytes": 0, 2023-12-06T18:33:41.904 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_objects": 0, 2023-12-06T18:33:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "tiers": [], 2023-12-06T18:33:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "tier_of": -1, 2023-12-06T18:33:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "read_tier": -1, 2023-12-06T18:33:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "write_tier": -1, 2023-12-06T18:33:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_mode": "none", 2023-12-06T18:33:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_bytes": 0, 2023-12-06T18:33:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_objects": 0, 2023-12-06T18:33:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-06T18:33:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-06T18:33:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-06T18:33:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_flush_age": 0, 2023-12-06T18:33:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_evict_age": 0, 2023-12-06T18:33:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "erasure_code_profile": "", 2023-12-06T18:33:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_params": { 2023-12-06T18:33:41.905 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "none" 2023-12-06T18:33:41.906 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.906 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_period": 0, 2023-12-06T18:33:41.906 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_count": 0, 2023-12-06T18:33:41.906 INFO:teuthology.orchestra.run.smithi061.stdout: "use_gmt_hitset": true, 2023-12-06T18:33:41.906 INFO:teuthology.orchestra.run.smithi061.stdout: "min_read_recency_for_promote": 0, 2023-12-06T18:33:41.906 INFO:teuthology.orchestra.run.smithi061.stdout: "min_write_recency_for_promote": 0, 2023-12-06T18:33:41.906 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_grade_decay_rate": 0, 2023-12-06T18:33:41.906 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_search_last_n": 0, 2023-12-06T18:33:41.906 INFO:teuthology.orchestra.run.smithi061.stdout: "grade_table": [], 2023-12-06T18:33:41.906 INFO:teuthology.orchestra.run.smithi061.stdout: "stripe_width": 0, 2023-12-06T18:33:41.906 INFO:teuthology.orchestra.run.smithi061.stdout: "expected_num_objects": 0, 2023-12-06T18:33:41.906 INFO:teuthology.orchestra.run.smithi061.stdout: "fast_read": false, 2023-12-06T18:33:41.906 INFO:teuthology.orchestra.run.smithi061.stdout: "options": { 2023-12-06T18:33:41.906 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_max": 32, 2023-12-06T18:33:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_min": 1 2023-12-06T18:33:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: "application_metadata": { 2023-12-06T18:33:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: "mgr": {} 2023-12-06T18:33:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: "read_balance": { 2023-12-06T18:33:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: "score_acting": 6.059999942779541, 2023-12-06T18:33:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: "score_stable": 6.059999942779541, 2023-12-06T18:33:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: "optimal_score": 0.33000001311302185, 2023-12-06T18:33:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_acting": 2, 2023-12-06T18:33:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_stable": 2, 2023-12-06T18:33:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity_weighted": 0.64166069030761719, 2023-12-06T18:33:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity": 0.86000001430511475, 2023-12-06T18:33:41.907 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity_weighted": 0.63999998569488525 2023-12-06T18:33:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: "pool": 2, 2023-12-06T18:33:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_name": "rbd", 2023-12-06T18:33:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: "create_time": "2023-12-06T18:27:01.712581+0000", 2023-12-06T18:33:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: "flags": 1, 2023-12-06T18:33:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: "flags_names": "hashpspool", 2023-12-06T18:33:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: "type": 1, 2023-12-06T18:33:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 2, 2023-12-06T18:33:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: "min_size": 1, 2023-12-06T18:33:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_rule": 0, 2023-12-06T18:33:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_count": 0, 2023-12-06T18:33:41.908 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_target": 0, 2023-12-06T18:33:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_barrier": 0, 2023-12-06T18:33:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-06T18:33:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: "object_hash": 2, 2023-12-06T18:33:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_autoscale_mode": "off", 2023-12-06T18:33:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num": 8, 2023-12-06T18:33:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num": 8, 2023-12-06T18:33:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_placement_num_target": 8, 2023-12-06T18:33:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_target": 8, 2023-12-06T18:33:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_num_pending": 8, 2023-12-06T18:33:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: "last_pg_merge_meta": { 2023-12-06T18:33:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: "source_pgid": "0.0", 2023-12-06T18:33:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: "ready_epoch": 0, 2023-12-06T18:33:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_started": 0, 2023-12-06T18:33:41.909 INFO:teuthology.orchestra.run.smithi061.stdout: "last_epoch_clean": 0, 2023-12-06T18:33:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: "source_version": "0'0", 2023-12-06T18:33:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: "target_version": "0'0" 2023-12-06T18:33:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: "last_change": "24", 2023-12-06T18:33:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend": "0", 2023-12-06T18:33:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-06T18:33:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: "last_force_op_resend_preluminous": "0", 2023-12-06T18:33:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: "auid": 0, 2023-12-06T18:33:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_mode": "selfmanaged", 2023-12-06T18:33:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_seq": 0, 2023-12-06T18:33:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: "snap_epoch": 0, 2023-12-06T18:33:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: "pool_snaps": [], 2023-12-06T18:33:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: "removed_snaps": "[]", 2023-12-06T18:33:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_bytes": 0, 2023-12-06T18:33:41.910 INFO:teuthology.orchestra.run.smithi061.stdout: "quota_max_objects": 0, 2023-12-06T18:33:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "tiers": [], 2023-12-06T18:33:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "tier_of": -1, 2023-12-06T18:33:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "read_tier": -1, 2023-12-06T18:33:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "write_tier": -1, 2023-12-06T18:33:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_mode": "none", 2023-12-06T18:33:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_bytes": 0, 2023-12-06T18:33:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "target_max_objects": 0, 2023-12-06T18:33:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-06T18:33:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-06T18:33:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-06T18:33:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_flush_age": 0, 2023-12-06T18:33:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "cache_min_evict_age": 0, 2023-12-06T18:33:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "erasure_code_profile": "", 2023-12-06T18:33:41.911 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_params": { 2023-12-06T18:33:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "none" 2023-12-06T18:33:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_period": 0, 2023-12-06T18:33:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_count": 0, 2023-12-06T18:33:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "use_gmt_hitset": true, 2023-12-06T18:33:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "min_read_recency_for_promote": 0, 2023-12-06T18:33:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "min_write_recency_for_promote": 0, 2023-12-06T18:33:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_grade_decay_rate": 0, 2023-12-06T18:33:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "hit_set_search_last_n": 0, 2023-12-06T18:33:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "grade_table": [], 2023-12-06T18:33:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "stripe_width": 0, 2023-12-06T18:33:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "expected_num_objects": 0, 2023-12-06T18:33:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "fast_read": false, 2023-12-06T18:33:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "options": {}, 2023-12-06T18:33:41.912 INFO:teuthology.orchestra.run.smithi061.stdout: "application_metadata": { 2023-12-06T18:33:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: "rbd": {} 2023-12-06T18:33:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: "read_balance": { 2023-12-06T18:33:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: "score_acting": 2.25, 2023-12-06T18:33:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: "score_stable": 2.25, 2023-12-06T18:33:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: "optimal_score": 1.1699999570846558, 2023-12-06T18:33:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_acting": 2.630000114440918, 2023-12-06T18:33:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: "raw_score_stable": 2.630000114440918, 2023-12-06T18:33:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity_weighted": 0.64166069030761719, 2023-12-06T18:33:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity": 0.86000001430511475, 2023-12-06T18:33:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: "average_primary_affinity_weighted": 0.63999998569488525 2023-12-06T18:33:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:33:41.913 INFO:teuthology.orchestra.run.smithi061.stdout: "osds": [ 2023-12-06T18:33:41.914 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.914 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 0, 2023-12-06T18:33:41.914 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "5e6e17b9-e72e-4e34-a531-0830d659bfe0", 2023-12-06T18:33:41.914 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:33:41.914 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:33:41.914 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:33:41.914 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 0.4124908447265625, 2023-12-06T18:33:41.914 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:33:41.914 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:33:41.914 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 16, 2023-12-06T18:33:41.914 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 109, 2023-12-06T18:33:41.914 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:33:41.914 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:33:41.914 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:33:41.914 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.915 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.915 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.915 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6805", 2023-12-06T18:33:41.915 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:33:41.915 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.915 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.915 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.915 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:33:41.915 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.915 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.915 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.915 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6806", 2023-12-06T18:33:41.915 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:33:41.915 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.915 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.916 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.916 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:33:41.916 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.916 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.916 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.916 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6808", 2023-12-06T18:33:41.916 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:33:41.916 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.916 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.916 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.916 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:33:41.916 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.916 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.916 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.917 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6807", 2023-12-06T18:33:41.917 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 281956997 2023-12-06T18:33:41.917 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.917 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.917 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.917 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6805/281956997", 2023-12-06T18:33:41.917 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6806/281956997", 2023-12-06T18:33:41.917 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6808/281956997", 2023-12-06T18:33:41.917 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6807/281956997", 2023-12-06T18:33:41.917 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:33:41.917 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:33:41.917 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:33:41.917 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.918 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.918 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.918 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 1, 2023-12-06T18:33:41.918 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "b78c49b0-360d-4351-b44e-65ccaaeb768f", 2023-12-06T18:33:41.918 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:33:41.918 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:33:41.918 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:33:41.918 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:33:41.918 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:33:41.918 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:33:41.918 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 16, 2023-12-06T18:33:41.918 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 79, 2023-12-06T18:33:41.918 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:33:41.919 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:33:41.919 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:33:41.919 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.919 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.919 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.919 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6809", 2023-12-06T18:33:41.919 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:33:41.919 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.919 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.919 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.919 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:33:41.919 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.919 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.919 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.919 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6810", 2023-12-06T18:33:41.920 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:33:41.920 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.920 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.920 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.920 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:33:41.920 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.920 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.920 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.920 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6812", 2023-12-06T18:33:41.920 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:33:41.920 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.920 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.920 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.920 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:33:41.920 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.921 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.921 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.921 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6811", 2023-12-06T18:33:41.921 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3306625622 2023-12-06T18:33:41.921 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.921 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.921 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.921 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6809/3306625622", 2023-12-06T18:33:41.921 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6810/3306625622", 2023-12-06T18:33:41.921 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6812/3306625622", 2023-12-06T18:33:41.921 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6811/3306625622", 2023-12-06T18:33:41.921 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:33:41.921 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:33:41.921 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:33:41.921 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.922 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.922 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.922 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 2, 2023-12-06T18:33:41.922 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "a2af3151-a4da-4c34-8fb6-78abf035785b", 2023-12-06T18:33:41.922 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:33:41.922 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:33:41.922 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:33:41.922 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:33:41.922 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:33:41.922 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:33:41.922 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 16, 2023-12-06T18:33:41.922 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 35, 2023-12-06T18:33:41.922 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:33:41.922 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:33:41.923 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:33:41.923 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.923 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.923 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.923 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6801", 2023-12-06T18:33:41.923 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:33:41.923 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.923 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.923 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.923 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:33:41.923 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.923 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.923 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.923 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6802", 2023-12-06T18:33:41.924 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:33:41.924 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.924 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.924 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.924 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:33:41.924 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.924 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.924 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.924 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6804", 2023-12-06T18:33:41.924 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:33:41.924 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.924 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.924 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.924 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:33:41.924 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.925 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.925 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.925 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6803", 2023-12-06T18:33:41.925 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3589270120 2023-12-06T18:33:41.925 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.925 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.925 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.925 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6801/3589270120", 2023-12-06T18:33:41.925 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6802/3589270120", 2023-12-06T18:33:41.925 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6804/3589270120", 2023-12-06T18:33:41.925 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6803/3589270120", 2023-12-06T18:33:41.925 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:33:41.925 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:33:41.925 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:33:41.926 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.926 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.926 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.926 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 3, 2023-12-06T18:33:41.926 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "1c754ed6-c535-454f-b35b-00f62cd6ce5a", 2023-12-06T18:33:41.926 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:33:41.926 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:33:41.926 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:33:41.926 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:33:41.926 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:33:41.926 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:33:41.926 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 18, 2023-12-06T18:33:41.926 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 100, 2023-12-06T18:33:41.926 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:33:41.927 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:33:41.927 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:33:41.927 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.927 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.927 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.927 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6813", 2023-12-06T18:33:41.927 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:33:41.927 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.927 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.927 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.927 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:33:41.927 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.927 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.927 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6814", 2023-12-06T18:33:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:33:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:33:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6816", 2023-12-06T18:33:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:33:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.928 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:33:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.61:6815", 2023-12-06T18:33:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2849920842 2023-12-06T18:33:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.61:6813/2849920842", 2023-12-06T18:33:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.61:6814/2849920842", 2023-12-06T18:33:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.61:6816/2849920842", 2023-12-06T18:33:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.61:6815/2849920842", 2023-12-06T18:33:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:33:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:33:41.929 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:33:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 4, 2023-12-06T18:33:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "766572af-b9ca-4f38-927d-36f26101ba51", 2023-12-06T18:33:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:33:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:33:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:33:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 0.720794677734375, 2023-12-06T18:33:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:33:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:33:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 15, 2023-12-06T18:33:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 100, 2023-12-06T18:33:41.930 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:33:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:33:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:33:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6800", 2023-12-06T18:33:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:33:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:33:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.931 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6801", 2023-12-06T18:33:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:33:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:33:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6803", 2023-12-06T18:33:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:33:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:33:41.932 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6802", 2023-12-06T18:33:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 2778770069 2023-12-06T18:33:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6800/2778770069", 2023-12-06T18:33:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6801/2778770069", 2023-12-06T18:33:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6803/2778770069", 2023-12-06T18:33:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6802/2778770069", 2023-12-06T18:33:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:33:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:33:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:33:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.933 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 5, 2023-12-06T18:33:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "7a566b60-1543-41d7-894c-e2fbb63e48dc", 2023-12-06T18:33:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:33:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:33:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 1, 2023-12-06T18:33:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 1, 2023-12-06T18:33:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:33:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:33:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 15, 2023-12-06T18:33:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 117, 2023-12-06T18:33:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:33:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:33:41.934 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:33:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6809", 2023-12-06T18:33:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:33:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:33:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6810", 2023-12-06T18:33:41.935 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:33:41.936 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.936 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.936 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:33:41.936 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.936 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.936 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6814", 2023-12-06T18:33:41.936 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:33:41.936 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.936 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.936 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.936 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:33:41.936 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.936 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.937 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.937 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6812", 2023-12-06T18:33:41.937 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 263671335 2023-12-06T18:33:41.937 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.937 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.937 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.937 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6809/263671335", 2023-12-06T18:33:41.937 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6810/263671335", 2023-12-06T18:33:41.937 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6814/263671335", 2023-12-06T18:33:41.937 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6812/263671335", 2023-12-06T18:33:41.937 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:33:41.937 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:33:41.937 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:33:41.937 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.938 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.938 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.938 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 6, 2023-12-06T18:33:41.938 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "28cc2358-4ac4-40b2-91a4-20edfe625a73", 2023-12-06T18:33:41.938 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:33:41.938 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 0, 2023-12-06T18:33:41.938 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 0, 2023-12-06T18:33:41.938 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 0, 2023-12-06T18:33:41.938 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 0, 2023-12-06T18:33:41.938 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 0, 2023-12-06T18:33:41.938 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 14, 2023-12-06T18:33:41.938 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 25, 2023-12-06T18:33:41.938 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 0, 2023-12-06T18:33:41.938 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:33:41.938 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:33:41.939 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.939 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.939 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.939 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6808", 2023-12-06T18:33:41.939 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:33:41.939 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.939 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.939 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.939 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:33:41.939 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.939 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.939 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.939 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6811", 2023-12-06T18:33:41.939 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:33:41.940 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.940 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.940 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.940 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:33:41.940 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.940 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.940 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.940 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6815", 2023-12-06T18:33:41.940 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:33:41.940 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.940 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.940 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.940 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:33:41.940 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.941 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.941 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.941 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6813", 2023-12-06T18:33:41.941 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 4245639079 2023-12-06T18:33:41.941 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.941 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.941 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.941 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6808/4245639079", 2023-12-06T18:33:41.941 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6811/4245639079", 2023-12-06T18:33:41.941 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6815/4245639079", 2023-12-06T18:33:41.941 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6813/4245639079", 2023-12-06T18:33:41.941 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:33:41.941 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:33:41.941 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:33:41.942 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.942 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.942 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.942 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 7, 2023-12-06T18:33:41.942 INFO:teuthology.orchestra.run.smithi061.stdout: "uuid": "8f79279b-d2d0-438e-b3c2-88df1241ff30", 2023-12-06T18:33:41.942 INFO:teuthology.orchestra.run.smithi061.stdout: "up": 1, 2023-12-06T18:33:41.942 INFO:teuthology.orchestra.run.smithi061.stdout: "in": 1, 2023-12-06T18:33:41.942 INFO:teuthology.orchestra.run.smithi061.stdout: "weight": 0.747711181640625, 2023-12-06T18:33:41.942 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_affinity": 0, 2023-12-06T18:33:41.942 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_begin": 14, 2023-12-06T18:33:41.942 INFO:teuthology.orchestra.run.smithi061.stdout: "last_clean_end": 37, 2023-12-06T18:33:41.942 INFO:teuthology.orchestra.run.smithi061.stdout: "up_from": 61, 2023-12-06T18:33:41.942 INFO:teuthology.orchestra.run.smithi061.stdout: "up_thru": 62, 2023-12-06T18:33:41.942 INFO:teuthology.orchestra.run.smithi061.stdout: "down_at": 38, 2023-12-06T18:33:41.942 INFO:teuthology.orchestra.run.smithi061.stdout: "lost_at": 0, 2023-12-06T18:33:41.943 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addrs": { 2023-12-06T18:33:41.943 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.943 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.943 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6804", 2023-12-06T18:33:41.943 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3075390540 2023-12-06T18:33:41.943 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.943 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.943 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.943 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addrs": { 2023-12-06T18:33:41.943 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.943 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.943 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.943 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6805", 2023-12-06T18:33:41.944 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3075390540 2023-12-06T18:33:41.944 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.944 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.944 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addrs": { 2023-12-06T18:33:41.944 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.944 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.944 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.944 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6807", 2023-12-06T18:33:41.944 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3075390540 2023-12-06T18:33:41.944 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.944 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.944 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.944 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addrs": { 2023-12-06T18:33:41.944 INFO:teuthology.orchestra.run.smithi061.stdout: "addrvec": [ 2023-12-06T18:33:41.945 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.945 INFO:teuthology.orchestra.run.smithi061.stdout: "type": "v2", 2023-12-06T18:33:41.945 INFO:teuthology.orchestra.run.smithi061.stdout: "addr": "172.21.15.152:6806", 2023-12-06T18:33:41.945 INFO:teuthology.orchestra.run.smithi061.stdout: "nonce": 3075390540 2023-12-06T18:33:41.945 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.945 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.945 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.945 INFO:teuthology.orchestra.run.smithi061.stdout: "public_addr": "172.21.15.152:6804/3075390540", 2023-12-06T18:33:41.945 INFO:teuthology.orchestra.run.smithi061.stdout: "cluster_addr": "172.21.15.152:6805/3075390540", 2023-12-06T18:33:41.945 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_back_addr": "172.21.15.152:6807/3075390540", 2023-12-06T18:33:41.945 INFO:teuthology.orchestra.run.smithi061.stdout: "heartbeat_front_addr": "172.21.15.152:6806/3075390540", 2023-12-06T18:33:41.945 INFO:teuthology.orchestra.run.smithi061.stdout: "state": [ 2023-12-06T18:33:41.945 INFO:teuthology.orchestra.run.smithi061.stdout: "exists", 2023-12-06T18:33:41.945 INFO:teuthology.orchestra.run.smithi061.stdout: "up" 2023-12-06T18:33:41.945 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.946 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.946 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:33:41.946 INFO:teuthology.orchestra.run.smithi061.stdout: "osd_xinfo": [ 2023-12-06T18:33:41.946 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.946 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 0, 2023-12-06T18:33:41.946 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:33:41.946 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:33:41.946 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:33:41.946 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:33:41.946 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:33:41.946 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:52.171904+0000", 2023-12-06T18:33:41.946 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:33:41.946 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.946 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.947 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 1, 2023-12-06T18:33:41.947 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:33:41.947 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:33:41.947 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:33:41.947 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:33:41.947 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:33:41.947 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:52.428840+0000", 2023-12-06T18:33:41.947 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:33:41.947 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.947 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.947 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 2, 2023-12-06T18:33:41.947 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:33:41.947 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:33:41.947 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:33:41.948 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:33:41.948 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:33:41.948 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:51.681954+0000", 2023-12-06T18:33:41.948 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:33:41.948 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.948 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.948 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 3, 2023-12-06T18:33:41.948 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:33:41.948 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:33:41.948 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:33:41.948 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:33:41.948 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:33:41.948 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:53.687417+0000", 2023-12-06T18:33:41.948 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:33:41.948 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.949 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.949 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 4, 2023-12-06T18:33:41.949 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:33:41.949 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:33:41.949 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:33:41.949 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:33:41.949 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:33:41.949 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:50.158682+0000", 2023-12-06T18:33:41.949 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:33:41.949 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.949 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.949 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 5, 2023-12-06T18:33:41.949 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:33:41.949 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:33:41.949 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:33:41.949 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:33:41.950 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:33:41.950 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:50.443807+0000", 2023-12-06T18:33:41.950 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:33:41.950 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.950 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.950 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 6, 2023-12-06T18:33:41.950 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:33:41.950 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:33:41.950 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:33:41.950 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:33:41.950 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 65536, 2023-12-06T18:33:41.950 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:49.750276+0000", 2023-12-06T18:33:41.950 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 0 2023-12-06T18:33:41.951 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.951 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.951 INFO:teuthology.orchestra.run.smithi061.stdout: "osd": 7, 2023-12-06T18:33:41.951 INFO:teuthology.orchestra.run.smithi061.stdout: "down_stamp": "0.000000", 2023-12-06T18:33:41.951 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_probability": 0, 2023-12-06T18:33:41.951 INFO:teuthology.orchestra.run.smithi061.stdout: "laggy_interval": 0, 2023-12-06T18:33:41.951 INFO:teuthology.orchestra.run.smithi061.stdout: "features": 4540138322906710015, 2023-12-06T18:33:41.951 INFO:teuthology.orchestra.run.smithi061.stdout: "old_weight": 0, 2023-12-06T18:33:41.951 INFO:teuthology.orchestra.run.smithi061.stdout: "last_purged_snaps_scrub": "2023-12-06T18:26:49.770987+0000", 2023-12-06T18:33:41.951 INFO:teuthology.orchestra.run.smithi061.stdout: "dead_epoch": 38 2023-12-06T18:33:41.951 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.951 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:33:41.951 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_upmap": [ 2023-12-06T18:33:41.951 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.952 INFO:teuthology.orchestra.run.smithi061.stdout: "pgid": "2.7", 2023-12-06T18:33:41.952 INFO:teuthology.orchestra.run.smithi061.stdout: "osds": [ 2023-12-06T18:33:41.952 INFO:teuthology.orchestra.run.smithi061.stdout: 5, 2023-12-06T18:33:41.952 INFO:teuthology.orchestra.run.smithi061.stdout: 3 2023-12-06T18:33:41.952 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.952 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.952 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:33:41.952 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_upmap_items": [ 2023-12-06T18:33:41.952 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.952 INFO:teuthology.orchestra.run.smithi061.stdout: "pgid": "1.0", 2023-12-06T18:33:41.952 INFO:teuthology.orchestra.run.smithi061.stdout: "mappings": [ 2023-12-06T18:33:41.952 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.952 INFO:teuthology.orchestra.run.smithi061.stdout: "from": 1, 2023-12-06T18:33:41.952 INFO:teuthology.orchestra.run.smithi061.stdout: "to": 0 2023-12-06T18:33:41.953 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.953 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.953 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.953 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.953 INFO:teuthology.orchestra.run.smithi061.stdout: "pgid": "2.3", 2023-12-06T18:33:41.953 INFO:teuthology.orchestra.run.smithi061.stdout: "mappings": [ 2023-12-06T18:33:41.953 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:41.953 INFO:teuthology.orchestra.run.smithi061.stdout: "from": 3, 2023-12-06T18:33:41.953 INFO:teuthology.orchestra.run.smithi061.stdout: "to": 1 2023-12-06T18:33:41.953 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.953 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:41.953 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.953 INFO:teuthology.orchestra.run.smithi061.stdout: ], 2023-12-06T18:33:41.953 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_upmap_primaries": [], 2023-12-06T18:33:41.953 INFO:teuthology.orchestra.run.smithi061.stdout: "pg_temp": [], 2023-12-06T18:33:41.953 INFO:teuthology.orchestra.run.smithi061.stdout: "primary_temp": [], 2023-12-06T18:33:41.954 INFO:teuthology.orchestra.run.smithi061.stdout: "blocklist": {}, 2023-12-06T18:33:41.954 INFO:teuthology.orchestra.run.smithi061.stdout: "range_blocklist": {}, 2023-12-06T18:33:41.954 INFO:teuthology.orchestra.run.smithi061.stdout: "erasure_code_profiles": { 2023-12-06T18:33:41.954 INFO:teuthology.orchestra.run.smithi061.stdout: "default": { 2023-12-06T18:33:41.954 INFO:teuthology.orchestra.run.smithi061.stdout: "crush-failure-domain": "osd", 2023-12-06T18:33:41.954 INFO:teuthology.orchestra.run.smithi061.stdout: "k": "2", 2023-12-06T18:33:41.954 INFO:teuthology.orchestra.run.smithi061.stdout: "m": "1", 2023-12-06T18:33:41.954 INFO:teuthology.orchestra.run.smithi061.stdout: "plugin": "jerasure", 2023-12-06T18:33:41.954 INFO:teuthology.orchestra.run.smithi061.stdout: "technique": "reed_sol_van" 2023-12-06T18:33:41.954 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.954 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:41.954 INFO:teuthology.orchestra.run.smithi061.stdout: "removed_snaps_queue": [], 2023-12-06T18:33:41.954 INFO:teuthology.orchestra.run.smithi061.stdout: "new_removed_snaps": [], 2023-12-06T18:33:41.955 INFO:teuthology.orchestra.run.smithi061.stdout: "new_purged_snaps": [], 2023-12-06T18:33:41.955 INFO:teuthology.orchestra.run.smithi061.stdout: "crush_node_flags": {}, 2023-12-06T18:33:41.955 INFO:teuthology.orchestra.run.smithi061.stdout: "device_class_flags": {}, 2023-12-06T18:33:41.955 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode": { 2023-12-06T18:33:41.955 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode_enabled": false, 2023-12-06T18:33:41.955 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_bucket_count": 0, 2023-12-06T18:33:41.955 INFO:teuthology.orchestra.run.smithi061.stdout: "degraded_stretch_mode": 0, 2023-12-06T18:33:41.955 INFO:teuthology.orchestra.run.smithi061.stdout: "recovering_stretch_mode": 0, 2023-12-06T18:33:41.955 INFO:teuthology.orchestra.run.smithi061.stdout: "stretch_mode_bucket": 0 2023-12-06T18:33:41.955 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:41.955 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:41.957 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:41.957 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:41.957 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:41.957 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:41.957 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:41.958 INFO:tasks.thrashosds.thrasher:j is {'epoch': 119, 'fsid': '497e3283-cc16-40ff-a72c-a7487474fb4b', 'created': '2023-12-06T18:26:07.122862+0000', 'modified': '2023-12-06T18:33:40.903828+0000', 'last_up_change': '2023-12-06T18:30:04.956811+0000', 'last_in_change': '2023-12-06T18:32:58.826891+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 8, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-06T18:26:52.648377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.6416606903076172, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.6399999856948853}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-12-06T18:27:01.712581+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1.1699999570846558, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.6416606903076172, 'average_primary_affinity': 0.8600000143051147, 'average_primary_affinity_weighted': 0.6399999856948853}}], 'osds': [{'osd': 0, 'uuid': '5e6e17b9-e72e-4e34-a531-0830d659bfe0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.4124908447265625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 109, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6805', 'nonce': 281956997}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6806', 'nonce': 281956997}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6808', 'nonce': 281956997}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6807', 'nonce': 281956997}]}, 'public_addr': '172.21.15.61:6805/281956997', 'cluster_addr': '172.21.15.61:6806/281956997', 'heartbeat_back_addr': '172.21.15.61:6808/281956997', 'heartbeat_front_addr': '172.21.15.61:6807/281956997', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'b78c49b0-360d-4351-b44e-65ccaaeb768f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 79, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6809', 'nonce': 3306625622}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6810', 'nonce': 3306625622}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6812', 'nonce': 3306625622}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6811', 'nonce': 3306625622}]}, 'public_addr': '172.21.15.61:6809/3306625622', 'cluster_addr': '172.21.15.61:6810/3306625622', 'heartbeat_back_addr': '172.21.15.61:6812/3306625622', 'heartbeat_front_addr': '172.21.15.61:6811/3306625622', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'a2af3151-a4da-4c34-8fb6-78abf035785b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 35, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6801', 'nonce': 3589270120}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6802', 'nonce': 3589270120}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6804', 'nonce': 3589270120}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6803', 'nonce': 3589270120}]}, 'public_addr': '172.21.15.61:6801/3589270120', 'cluster_addr': '172.21.15.61:6802/3589270120', 'heartbeat_back_addr': '172.21.15.61:6804/3589270120', 'heartbeat_front_addr': '172.21.15.61:6803/3589270120', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '1c754ed6-c535-454f-b35b-00f62cd6ce5a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 100, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6813', 'nonce': 2849920842}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6814', 'nonce': 2849920842}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6816', 'nonce': 2849920842}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.61:6815', 'nonce': 2849920842}]}, 'public_addr': '172.21.15.61:6813/2849920842', 'cluster_addr': '172.21.15.61:6814/2849920842', 'heartbeat_back_addr': '172.21.15.61:6816/2849920842', 'heartbeat_front_addr': '172.21.15.61:6815/2849920842', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '766572af-b9ca-4f38-927d-36f26101ba51', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.720794677734375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 100, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6800', 'nonce': 2778770069}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6801', 'nonce': 2778770069}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6803', 'nonce': 2778770069}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6802', 'nonce': 2778770069}]}, 'public_addr': '172.21.15.152:6800/2778770069', 'cluster_addr': '172.21.15.152:6801/2778770069', 'heartbeat_back_addr': '172.21.15.152:6803/2778770069', 'heartbeat_front_addr': '172.21.15.152:6802/2778770069', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '7a566b60-1543-41d7-894c-e2fbb63e48dc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 117, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6809', 'nonce': 263671335}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6810', 'nonce': 263671335}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6814', 'nonce': 263671335}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6812', 'nonce': 263671335}]}, 'public_addr': '172.21.15.152:6809/263671335', 'cluster_addr': '172.21.15.152:6810/263671335', 'heartbeat_back_addr': '172.21.15.152:6814/263671335', 'heartbeat_front_addr': '172.21.15.152:6812/263671335', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '28cc2358-4ac4-40b2-91a4-20edfe625a73', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6808', 'nonce': 4245639079}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6811', 'nonce': 4245639079}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6815', 'nonce': 4245639079}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6813', 'nonce': 4245639079}]}, 'public_addr': '172.21.15.152:6808/4245639079', 'cluster_addr': '172.21.15.152:6811/4245639079', 'heartbeat_back_addr': '172.21.15.152:6815/4245639079', 'heartbeat_front_addr': '172.21.15.152:6813/4245639079', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '8f79279b-d2d0-438e-b3c2-88df1241ff30', 'up': 1, 'in': 1, 'weight': 0.747711181640625, 'primary_affinity': 0, 'last_clean_begin': 14, 'last_clean_end': 37, 'up_from': 61, 'up_thru': 62, 'down_at': 38, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6804', 'nonce': 3075390540}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6805', 'nonce': 3075390540}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6807', 'nonce': 3075390540}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6806', 'nonce': 3075390540}]}, 'public_addr': '172.21.15.152:6804/3075390540', 'cluster_addr': '172.21.15.152:6805/3075390540', 'heartbeat_back_addr': '172.21.15.152:6807/3075390540', 'heartbeat_front_addr': '172.21.15.152:6806/3075390540', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:52.171904+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:52.428840+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:51.681954+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:53.687417+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:50.158682+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:50.443807+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2023-12-06T18:26:49.750276+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2023-12-06T18:26:49.770987+0000', 'dead_epoch': 38}], 'pg_upmap': [{'pgid': '2.7', 'osds': [5, 3]}], 'pg_upmap_items': [{'pgid': '1.0', 'mappings': [{'from': 1, 'to': 0}]}, {'pgid': '2.3', 'mappings': [{'from': 3, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2023-12-06T18:33:41.958 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:33:41.958 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:33:41.998 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:41.995+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:42.096 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:42.093+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:42.136 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:42.136 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:42.136 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:42.136 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:33:42.136 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.136 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:33:42.136 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:33:42.136 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 320.64278022299999, 2023-12-06T18:33:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:33:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:33:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:33:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.137 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:33:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:33:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:33:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:33:42.138 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:33:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:33:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:33:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:33:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:33:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:33:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:33:42.139 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:33:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:33:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.140 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.140 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:42.147 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:33:42.232 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:42.228+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:42.297 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:42.294+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:42.321 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:42.321 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:42.321 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:42.321 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:42.330 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:33:42.415 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:42.411+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:42.499 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:42.496+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:42.503 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:42.503 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:42.503 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:42.503 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:42.503 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:42.512 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:33:42.528 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:02:21.973 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:33:42.534 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:02:21.979 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:33:42.583 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:01:16.175 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:33:42.585 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:01:16.177 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:33:42.594 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:02:22.040 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:33:42.600 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:42.596+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:42.622 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:33:42.623 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:33:42.636 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:01:16.227 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:33:42.638 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2023-12-06T18:33:42.284128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":201348,"kb_used_data":7000,"kb_used_omap":13,"kb_used_meta":190002,"kb_avail":660401532,"statfs":{"total":676457349120,"available":676251168768,"internally_reserved":0,"allocated":7168000,"data_stored":3070266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13963,"internal_metadata":194562421},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":59,"apply_latency_ms":59,"commit_latency_ns":59000000,"apply_latency_ns":59000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001022"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":72,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-12-06T18:33:36.661456+0000","last_change":"2023-12-06T18:29:09.051293+0000","last_active":"2023-12-06T18:33:36.661456+0000","last_peered":"2023-12-06T18:33:36.661456+0000","last_clean":"2023-12-06T18:33:36.661456+0000","last_became_active":"2023-12-06T18:29:09.045331+0000","last_became_peered":"2023-12-06T18:29:09.045331+0000","last_unstale":"2023-12-06T18:33:36.661456+0000","last_undegraded":"2023-12-06T18:33:36.661456+0000","last_fullsized":"2023-12-06T18:33:36.661456+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T02:46:01.854508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":27,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-12-06T18:33:36.761398+0000","last_change":"2023-12-06T18:33:00.623558+0000","last_active":"2023-12-06T18:33:36.761398+0000","last_peered":"2023-12-06T18:33:36.761398+0000","last_clean":"2023-12-06T18:33:36.761398+0000","last_became_active":"2023-12-06T18:33:00.616353+0000","last_became_peered":"2023-12-06T18:33:00.616353+0000","last_unstale":"2023-12-06T18:33:36.761398+0000","last_undegraded":"2023-12-06T18:33:36.761398+0000","last_fullsized":"2023-12-06T18:33:36.761398+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T18:33:31.069235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":118,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-06T18:33:34.395554+0000","last_change":"2023-12-06T18:33:00.592340+0000","last_active":"2023-12-06T18:33:34.395554+0000","last_peered":"2023-12-06T18:33:34.395554+0000","last_clean":"2023-12-06T18:33:34.395554+0000","last_became_active":"2023-12-06T18:33:00.581037+0000","last_became_peered":"2023-12-06T18:33:00.581037+0000","last_unstale":"2023-12-06T18:33:34.395554+0000","last_undegraded":"2023-12-06T18:33:34.395554+0000","last_fullsized":"2023-12-06T18:33:34.395554+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T06:13:54.498147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":87,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-12-06T18:33:36.655752+0000","last_change":"2023-12-06T18:32:23.586404+0000","last_active":"2023-12-06T18:33:36.655752+0000","last_peered":"2023-12-06T18:33:36.655752+0000","last_clean":"2023-12-06T18:33:36.655752+0000","last_became_active":"2023-12-06T18:32:23.580588+0000","last_became_peered":"2023-12-06T18:32:23.580588+0000","last_unstale":"2023-12-06T18:33:36.655752+0000","last_undegraded":"2023-12-06T18:33:36.655752+0000","last_fullsized":"2023-12-06T18:33:36.655752+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:31:58.277721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":111,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-06T18:33:34.389136+0000","last_change":"2023-12-06T18:32:23.632179+0000","last_active":"2023-12-06T18:33:34.389136+0000","last_peered":"2023-12-06T18:33:34.389136+0000","last_clean":"2023-12-06T18:33:34.389136+0000","last_became_active":"2023-12-06T18:32:23.623784+0000","last_became_peered":"2023-12-06T18:32:23.623784+0000","last_unstale":"2023-12-06T18:33:34.389136+0000","last_undegraded":"2023-12-06T18:33:34.389136+0000","last_fullsized":"2023-12-06T18:33:34.389136+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T21:18:05.348471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-12-06T18:33:37.903656+0000","last_change":"2023-12-06T18:33:37.896702+0000","last_active":"2023-12-06T18:33:37.903656+0000","last_peered":"2023-12-06T18:33:37.903656+0000","last_clean":"2023-12-06T18:33:37.903656+0000","last_became_active":"2023-12-06T18:33:37.890795+0000","last_became_peered":"2023-12-06T18:33:37.890795+0000","last_unstale":"2023-12-06T18:33:37.903656+0000","last_undegraded":"2023-12-06T18:33:37.903656+0000","last_fullsized":"2023-12-06T18:33:37.903656+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T22:57:30.740378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":118,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-06T18:33:34.391046+0000","last_change":"2023-12-06T18:28:47.876556+0000","last_active":"2023-12-06T18:33:34.391046+0000","last_peered":"2023-12-06T18:33:34.391046+0000","last_clean":"2023-12-06T18:33:34.391046+0000","last_became_active":"2023-12-06T18:28:47.865169+0000","last_became_peered":"2023-12-06T18:28:47.865169+0000","last_unstale":"2023-12-06T18:33:34.391046+0000","last_undegraded":"2023-12-06T18:33:34.391046+0000","last_fullsized":"2023-12-06T18:33:34.391046+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:16:40.095542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":48,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-12-06T18:33:36.781854+0000","last_change":"2023-12-06T18:31:57.187068+0000","last_active":"2023-12-06T18:33:36.781854+0000","last_peered":"2023-12-06T18:33:36.781854+0000","last_clean":"2023-12-06T18:33:36.781854+0000","last_became_active":"2023-12-06T18:31:57.175444+0000","last_became_peered":"2023-12-06T18:31:57.175444+0000","last_unstale":"2023-12-06T18:33:36.781854+0000","last_undegraded":"2023-12-06T18:33:36.781854+0000","last_fullsized":"2023-12-06T18:33:36.781854+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:25:09.547812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":24,"reported_epoch":117,"state":"active+clean","last_fresh":"2023-12-06T18:33:36.288725+0000","last_change":"2023-12-06T18:33:22.203137+0000","last_active":"2023-12-06T18:33:36.288725+0000","last_peered":"2023-12-06T18:33:36.288725+0000","last_clean":"2023-12-06T18:33:36.288725+0000","last_became_active":"2023-12-06T18:33:21.485790+0000","last_became_peered":"2023-12-06T18:33:21.485790+0000","last_unstale":"2023-12-06T18:33:36.288725+0000","last_undegraded":"2023-12-06T18:33:36.288725+0000","last_fullsized":"2023-12-06T18:33:36.288725+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T01:25:32.645799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1605632,"data_stored":1591360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411411,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":901120,"data_stored":321826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":10.098000000000001,"5min":10.752000000000001,"15min":10.554},"min":{"1min":6.3730000000000002,"5min":3.835,"15min":3.835},"max":{"1min":25.571000000000002,"5min":108.57599999999999,"15min":108.57599999999999},"last":8.2059999999999995},{"interface":"front","average":{"1min":11.791,"5min":11.002000000000001,"15min":12.173999999999999},"min":{"1min":4.8639999999999999,"5min":3.7200000000000002,"15min":3.7200000000000002},"max":{"1min":20.649999999999999,"5min":128.09399999999999,"15min":128.09399999999999},"last":9.3889999999999993}]},{"osd":1,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":13.631,"5min":12.202,"15min":12.771000000000001},"min":{"1min":7.6390000000000002,"5min":6.2539999999999996,"15min":6.2539999999999996},"max":{"1min":26.565999999999999,"5min":70.072000000000003,"15min":70.072000000000003},"last":11.765000000000001},{"interface":"front","average":{"1min":14.135999999999999,"5min":12.827999999999999,"15min":15.066000000000001},"min":{"1min":6.8280000000000003,"5min":6.444,"15min":6.444},"max":{"1min":26.420000000000002,"5min":27.934000000000001,"15min":32.920000000000002},"last":10.382}]},{"osd":2,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":13.890000000000001,"5min":14.91,"15min":14.795},"min":{"1min":8.8390000000000004,"5min":7.7690000000000001,"15min":7.5149999999999997},"max":{"1min":24.039000000000001,"5min":75.125,"15min":75.125},"last":14.458},{"interface":"front","average":{"1min":15.166,"5min":15.824999999999999,"15min":17.459},"min":{"1min":7.5389999999999997,"5min":7.5389999999999997,"15min":7.5389999999999997},"max":{"1min":24.375,"5min":69.403999999999996,"15min":69.403999999999996},"last":13.789}]},{"osd":4,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":16.661000000000001,"5min":13.574,"15min":12.426},"min":{"1min":10.935,"5min":5.8879999999999999,"15min":5.8879999999999999},"max":{"1min":27.975000000000001,"5min":27.975000000000001,"15min":27.975000000000001},"last":56.255000000000003},{"interface":"front","average":{"1min":10.597,"5min":13.162000000000001,"15min":15.118},"min":{"1min":7.29,"5min":6.3099999999999996,"15min":6.3099999999999996},"max":{"1min":23.651,"5min":71.718000000000004,"15min":71.718000000000004},"last":54.616}]},{"osd":5,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":18.983000000000001,"5min":18.795999999999999,"15min":18.07},"min":{"1min":13.872,"5min":9.798,"15min":8.3179999999999996},"max":{"1min":28.015999999999998,"5min":75.751999999999995,"15min":75.751999999999995},"last":12.678000000000001},{"interface":"front","average":{"1min":18.129000000000001,"5min":17.302,"15min":17.986999999999998},"min":{"1min":10.148,"5min":7.9269999999999996,"15min":7.9269999999999996},"max":{"1min":28.039999999999999,"5min":70.933000000000007,"15min":70.933000000000007},"last":17.216999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":13.880000000000001,"5min":14.038,"15min":15.648999999999999},"min":{"1min":10.093,"5min":7.0919999999999996,"15min":7.0919999999999996},"max":{"1min":22.777000000000001,"5min":28.469999999999999,"15min":29.245000000000001},"last":25.798999999999999},{"interface":"front","average":{"1min":19.372,"5min":19.238,"15min":20.812000000000001},"min":{"1min":14.287000000000001,"5min":10.973000000000001,"15min":10.973000000000001},"max":{"1min":29.497,"5min":74.224000000000004,"15min":74.224000000000004},"last":16.817}]},{"osd":7,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":17.841000000000001,"5min":19.835999999999999,"15min":21.25},"min":{"1min":9.1859999999999999,"5min":9.1859999999999999,"15min":9.1859999999999999},"max":{"1min":33.747,"5min":122.93000000000001,"15min":122.93000000000001},"last":18.033000000000001},{"interface":"front","average":{"1min":22.260000000000002,"5min":22.786999999999999,"15min":23.417999999999999},"min":{"1min":15.458,"5min":13.196,"15min":13.196},"max":{"1min":31.978999999999999,"5min":124.473,"15min":124.473},"last":15.08}]}]},{"osd":2,"up_from":16,"seq":68719476819,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":913408,"data_stored":326243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:33:08 2023","interfaces":[{"interface":"back","average":{"1min":12.861000000000001,"5min":10.439,"15min":11.157999999999999},"min":{"1min":6.0350000000000001,"5min":4.1639999999999997,"15min":4.1639999999999997},"max":{"1min":29.417000000000002,"5min":39.220999999999997,"15min":39.220999999999997},"last":47.884999999999998},{"interface":"front","average":{"1min":14.871,"5min":11.584,"15min":11.446},"min":{"1min":7.1440000000000001,"5min":4.3280000000000003,"15min":4.3280000000000003},"max":{"1min":31.372,"5min":44.691000000000003,"15min":44.691000000000003},"last":9.4199999999999999}]},{"osd":1,"last update":"Wed Dec 6 18:33:08 2023","interfaces":[{"interface":"back","average":{"1min":9.8409999999999993,"5min":11.188000000000001,"15min":11.483000000000001},"min":{"1min":7.2530000000000001,"5min":5.9139999999999997,"15min":5.9139999999999997},"max":{"1min":15.18,"5min":42.920999999999999,"15min":42.920999999999999},"last":22.960999999999999},{"interface":"front","average":{"1min":13.111000000000001,"5min":13.525,"15min":13.849},"min":{"1min":7.1230000000000002,"5min":6.1479999999999997,"15min":6.1479999999999997},"max":{"1min":18.948,"5min":34.607999999999997,"15min":34.607999999999997},"last":14.382}]},{"osd":3,"last update":"Wed Dec 6 18:33:08 2023","interfaces":[{"interface":"back","average":{"1min":15.618,"5min":14.375999999999999,"15min":18.050000000000001},"min":{"1min":9.5869999999999997,"5min":6.1260000000000003,"15min":6.1260000000000003},"max":{"1min":26.550999999999998,"5min":49.609000000000002,"15min":112.129},"last":25.952999999999999},{"interface":"front","average":{"1min":16.542999999999999,"5min":16.094000000000001,"15min":22.181999999999999},"min":{"1min":10.18,"5min":7.5510000000000002,"15min":7.5510000000000002},"max":{"1min":22.042999999999999,"5min":61.243000000000002,"15min":181.601},"last":36.770000000000003}]},{"osd":4,"last update":"Wed Dec 6 18:33:08 2023","interfaces":[{"interface":"back","average":{"1min":14.792999999999999,"5min":14.91,"15min":17.263999999999999},"min":{"1min":10.734,"5min":6.609,"15min":6.609},"max":{"1min":21.219999999999999,"5min":60.332000000000001,"15min":97.088999999999999},"last":35.396999999999998},{"interface":"front","average":{"1min":11.157,"5min":13.459,"15min":19.145},"min":{"1min":7.2990000000000004,"5min":6.0720000000000001,"15min":6.0720000000000001},"max":{"1min":16.617999999999999,"5min":43.198,"15min":98.069000000000003},"last":24.061}]},{"osd":5,"last update":"Wed Dec 6 18:33:08 2023","interfaces":[{"interface":"back","average":{"1min":17.873999999999999,"5min":17.538,"15min":18.042000000000002},"min":{"1min":10.359,"5min":7.8170000000000002,"15min":7.8170000000000002},"max":{"1min":39.686,"5min":57.628999999999998,"15min":57.628999999999998},"last":10.743},{"interface":"front","average":{"1min":15.845000000000001,"5min":15.821999999999999,"15min":15.834},"min":{"1min":9.8049999999999997,"5min":8.5990000000000002,"15min":8.5990000000000002},"max":{"1min":23.103000000000002,"5min":47.749000000000002,"15min":47.749000000000002},"last":39.875999999999998}]},{"osd":6,"last update":"Wed Dec 6 18:33:08 2023","interfaces":[{"interface":"back","average":{"1min":18.725000000000001,"5min":18.283000000000001,"15min":37.774999999999999},"min":{"1min":11.263,"5min":8.7289999999999992,"15min":8.7289999999999992},"max":{"1min":31.329000000000001,"5min":64.034999999999997,"15min":498.38499999999999},"last":38.799999999999997},{"interface":"front","average":{"1min":20.073,"5min":19.332000000000001,"15min":39.524999999999999},"min":{"1min":10.803000000000001,"5min":9.7010000000000005,"15min":9.7010000000000005},"max":{"1min":30.225000000000001,"5min":54.920000000000002,"15min":496.17399999999998},"last":12.461}]},{"osd":7,"last update":"Wed Dec 6 18:33:14 2023","interfaces":[{"interface":"back","average":{"1min":19.437999999999999,"5min":18.989999999999998,"15min":18.945},"min":{"1min":11.891999999999999,"5min":10.064,"15min":10.064},"max":{"1min":29.789000000000001,"5min":49.456000000000003,"15min":49.456000000000003},"last":35.945},{"interface":"front","average":{"1min":20.567,"5min":20.460000000000001,"15min":20.257000000000001},"min":{"1min":14.108000000000001,"5min":12.288,"15min":12.288},"max":{"1min":32.514000000000003,"5min":40.656999999999996,"15min":40.656999999999996},"last":41.965000000000003}]}]},{"osd":1,"up_from":16,"seq":68719476820,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":913408,"data_stored":326243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:33:06 2023","interfaces":[{"interface":"back","average":{"1min":11.112,"5min":10.651,"15min":19.792000000000002},"min":{"1min":7.1459999999999999,"5min":5.0890000000000004,"15min":5.0890000000000004},"max":{"1min":25.581,"5min":34.286000000000001,"15min":238.87100000000001},"last":7.851},{"interface":"front","average":{"1min":13.031000000000001,"5min":10.532,"15min":37.002000000000002},"min":{"1min":6.452,"5min":4.4269999999999996,"15min":4.4269999999999996},"max":{"1min":61.262,"5min":61.262,"15min":661.33600000000001},"last":9.7279999999999998}]},{"osd":2,"last update":"Wed Dec 6 18:33:06 2023","interfaces":[{"interface":"back","average":{"1min":11.746,"5min":12.837,"15min":28.960999999999999},"min":{"1min":7.9390000000000001,"5min":6.6340000000000003,"15min":6.6340000000000003},"max":{"1min":21.129000000000001,"5min":93.168000000000006,"15min":384.315},"last":10.981999999999999},{"interface":"front","average":{"1min":17.257999999999999,"5min":17.582000000000001,"15min":41.338999999999999},"min":{"1min":12.568,"5min":8.109,"15min":5.7009999999999996},"max":{"1min":27.864999999999998,"5min":94.230000000000004,"15min":656.98699999999997},"last":18.059000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:33:06 2023","interfaces":[{"interface":"back","average":{"1min":12.555999999999999,"5min":11.630000000000001,"15min":12.943},"min":{"1min":7.8159999999999998,"5min":5.548,"15min":5.548},"max":{"1min":28.707000000000001,"5min":31.16,"15min":34.057000000000002},"last":13.35},{"interface":"front","average":{"1min":16.106000000000002,"5min":13.483000000000001,"15min":15.641999999999999},"min":{"1min":6.9160000000000004,"5min":6.5730000000000004,"15min":6.5730000000000004},"max":{"1min":65.537000000000006,"5min":65.537000000000006,"15min":65.537000000000006},"last":16.526}]},{"osd":4,"last update":"Wed Dec 6 18:33:06 2023","interfaces":[{"interface":"back","average":{"1min":14.526,"5min":12.695,"15min":35.174999999999997},"min":{"1min":8.3130000000000006,"5min":6.2930000000000001,"15min":6.2930000000000001},"max":{"1min":33.201999999999998,"5min":33.201999999999998,"15min":447.67200000000003},"last":16.045999999999999},{"interface":"front","average":{"1min":17.960999999999999,"5min":16.978000000000002,"15min":42.109999999999999},"min":{"1min":10.378,"5min":9.6129999999999995,"15min":7.0919999999999996},"max":{"1min":32.469999999999999,"5min":44.043999999999997,"15min":630.572},"last":21.506}]},{"osd":5,"last update":"Wed Dec 6 18:33:06 2023","interfaces":[{"interface":"back","average":{"1min":16.893999999999998,"5min":16.215,"15min":37.536999999999999},"min":{"1min":9.5530000000000008,"5min":7.0540000000000003,"15min":7.0540000000000003},"max":{"1min":68.013999999999996,"5min":68.013999999999996,"15min":389.55000000000001},"last":20.422999999999998},{"interface":"front","average":{"1min":19.353000000000002,"5min":16.658999999999999,"15min":48.807000000000002},"min":{"1min":11.243,"5min":7.0410000000000004,"15min":7.0410000000000004},"max":{"1min":27.831,"5min":37.744,"15min":658.48599999999999},"last":23.879999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:33:06 2023","interfaces":[{"interface":"back","average":{"1min":17.175000000000001,"5min":14.875,"15min":31.696999999999999},"min":{"1min":8.3640000000000008,"5min":7.5899999999999999,"15min":7.5899999999999999},"max":{"1min":35.575000000000003,"5min":54.383000000000003,"15min":439.56400000000002},"last":18.82},{"interface":"front","average":{"1min":20.062000000000001,"5min":19.577000000000002,"15min":33.789000000000001},"min":{"1min":11.862,"5min":10.641999999999999,"15min":10.641999999999999},"max":{"1min":44.154000000000003,"5min":55.423999999999999,"15min":386.71499999999997},"last":24.683}]},{"osd":7,"last update":"Wed Dec 6 18:33:11 2023","interfaces":[{"interface":"back","average":{"1min":19.498999999999999,"5min":18.276,"15min":18.617999999999999},"min":{"1min":9.3680000000000003,"5min":9.3680000000000003,"15min":9.3680000000000003},"max":{"1min":62.582999999999998,"5min":97.561999999999998,"15min":97.561999999999998},"last":22.273},{"interface":"front","average":{"1min":26.725000000000001,"5min":19.928999999999998,"15min":18.407},"min":{"1min":14.022,"5min":8.7219999999999995,"15min":8.7219999999999995},"max":{"1min":101.762,"5min":101.762,"15min":101.762},"last":25.097000000000001}]}]},{"osd":0,"up_from":16,"seq":68719476821,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28304,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343536,"statfs":{"total":96636764160,"available":96607780864,"internally_reserved":0,"allocated":1314816,"data_stored":724083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":11.467000000000001,"5min":10.736000000000001,"15min":15.943},"min":{"1min":5.6749999999999998,"5min":3.2360000000000002,"15min":3.2360000000000002},"max":{"1min":40.502000000000002,"5min":47.661000000000001,"15min":116.593},"last":8.5199999999999996},{"interface":"front","average":{"1min":14.481,"5min":12.929,"15min":16.911999999999999},"min":{"1min":5.5709999999999997,"5min":4.9039999999999999,"15min":4.9039999999999999},"max":{"1min":42.502000000000002,"5min":50.127000000000002,"15min":117.896},"last":12.555}]},{"osd":2,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":15.561,"5min":14.813000000000001,"15min":13.863},"min":{"1min":6.726,"5min":5.694,"15min":5.694},"max":{"1min":25.344000000000001,"5min":43.752000000000002,"15min":43.752000000000002},"last":28.181999999999999},{"interface":"front","average":{"1min":14.154999999999999,"5min":13.552,"15min":11.832000000000001},"min":{"1min":6.5220000000000002,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":27.962,"5min":49.670000000000002,"15min":49.670000000000002},"last":20.120000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:33:11 2023","interfaces":[{"interface":"back","average":{"1min":18.148,"5min":14.635999999999999,"15min":16.108000000000001},"min":{"1min":11.849,"5min":6.1379999999999999,"15min":6.1379999999999999},"max":{"1min":38.896000000000001,"5min":38.896000000000001,"15min":46.689999999999998},"last":22.085000000000001},{"interface":"front","average":{"1min":14.398999999999999,"5min":15.494,"15min":16.018000000000001},"min":{"1min":7.5510000000000002,"5min":7.5510000000000002,"15min":7.5510000000000002},"max":{"1min":26.088999999999999,"5min":50.545999999999999,"15min":50.545999999999999},"last":14.92}]},{"osd":4,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":12.005000000000001,"5min":12.326000000000001,"15min":18.361999999999998},"min":{"1min":6.9699999999999998,"5min":5.9349999999999996,"15min":5.9349999999999996},"max":{"1min":16.777000000000001,"5min":43.920999999999999,"15min":82.432000000000002},"last":23.260000000000002},{"interface":"front","average":{"1min":18.808,"5min":18.149999999999999,"15min":19.920000000000002},"min":{"1min":12.162000000000001,"5min":9.5239999999999991,"15min":6.2629999999999999},"max":{"1min":28.553999999999998,"5min":61.402999999999999,"15min":81.239999999999995},"last":26.135000000000002}]},{"osd":5,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":18.818999999999999,"5min":16.928999999999998,"15min":17.396999999999998},"min":{"1min":8.8119999999999994,"5min":8.0099999999999998,"15min":8.0099999999999998},"max":{"1min":36.323,"5min":71.367000000000004,"15min":71.367000000000004},"last":15.864000000000001},{"interface":"front","average":{"1min":17.681000000000001,"5min":16.157,"15min":17.521000000000001},"min":{"1min":9.2650000000000006,"5min":7.1779999999999999,"15min":7.1779999999999999},"max":{"1min":36.862000000000002,"5min":72.307000000000002,"15min":72.307000000000002},"last":24.350000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":20.559999999999999,"5min":19.556999999999999,"15min":19.190999999999999},"min":{"1min":13.153,"5min":8.9179999999999993,"15min":7.7560000000000002},"max":{"1min":44.655999999999999,"5min":62.932000000000002,"15min":62.932000000000002},"last":25.498000000000001},{"interface":"front","average":{"1min":17.888000000000002,"5min":17.213000000000001,"15min":18.768000000000001},"min":{"1min":10.164,"5min":9.3520000000000003,"15min":9.3520000000000003},"max":{"1min":29.501000000000001,"5min":43.088000000000001,"15min":52.627000000000002},"last":16.783000000000001}]},{"osd":7,"last update":"Wed Dec 6 18:33:11 2023","interfaces":[{"interface":"back","average":{"1min":21.228999999999999,"5min":20.321000000000002,"15min":19.521000000000001},"min":{"1min":16.484999999999999,"5min":12.009,"15min":12.009},"max":{"1min":32.023000000000003,"5min":58.021000000000001,"15min":58.021000000000001},"last":27.498000000000001},{"interface":"front","average":{"1min":19.852,"5min":18.920999999999999,"15min":18.238},"min":{"1min":11.643000000000001,"5min":11.515000000000001,"15min":11.515000000000001},"max":{"1min":36.466000000000001,"5min":40.978999999999999,"15min":40.978999999999999},"last":13.775}]}]},{"osd":5,"up_from":15,"seq":64424509525,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":1314816,"data_stored":724083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":8.9949999999999992,"5min":8.4920000000000009,"15min":13.872},"min":{"1min":4.1669999999999998,"5min":3.3660000000000001,"15min":3.3660000000000001},"max":{"1min":23.98,"5min":23.98,"15min":208.291},"last":18.751000000000001},{"interface":"front","average":{"1min":11.83,"5min":10.465999999999999,"15min":17.478000000000002},"min":{"1min":6.3940000000000001,"5min":4.4299999999999997,"15min":4.4299999999999997},"max":{"1min":24.289999999999999,"5min":24.289999999999999,"15min":199.94900000000001},"last":8.4109999999999996}]},{"osd":1,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":14.759,"5min":13.478999999999999,"15min":48.890999999999998},"min":{"1min":7.0339999999999998,"5min":5.085,"15min":5.085},"max":{"1min":22.251000000000001,"5min":41.234999999999999,"15min":1211.6010000000001},"last":13.789},{"interface":"front","average":{"1min":10.929,"5min":10.045999999999999,"15min":44.293999999999997},"min":{"1min":7.0060000000000002,"5min":4.7199999999999998,"15min":4.7199999999999998},"max":{"1min":17.417999999999999,"5min":39.802999999999997,"15min":1189.3869999999999},"last":14.815}]},{"osd":2,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":16.795000000000002,"5min":14.260999999999999,"15min":22.454999999999998},"min":{"1min":7.9980000000000002,"5min":6.5129999999999999,"15min":6.5129999999999999},"max":{"1min":68.617999999999995,"5min":68.617999999999995,"15min":283.46800000000002},"last":9.0820000000000007},{"interface":"front","average":{"1min":21.155999999999999,"5min":17.585000000000001,"15min":26.276},"min":{"1min":8.173,"5min":8.173,"15min":8.173},"max":{"1min":69.513999999999996,"5min":69.513999999999996,"15min":296.47800000000001},"last":13.176}]},{"osd":3,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":12.999000000000001,"5min":14.625999999999999,"15min":17.956},"min":{"1min":6.0300000000000002,"5min":6.0300000000000002,"15min":6.0300000000000002},"max":{"1min":36.156999999999996,"5min":36.156999999999996,"15min":47.926000000000002},"last":10.51},{"interface":"front","average":{"1min":17.372,"5min":13.675000000000001,"15min":15.026999999999999},"min":{"1min":11.651,"5min":5.3979999999999997,"15min":5.3979999999999997},"max":{"1min":34.609000000000002,"5min":34.609000000000002,"15min":66.176000000000002},"last":19.518000000000001}]},{"osd":4,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":19.289999999999999,"5min":17.210000000000001,"15min":19.231999999999999},"min":{"1min":11.885,"5min":9.7210000000000001,"15min":9.7210000000000001},"max":{"1min":41.923999999999999,"5min":41.923999999999999,"15min":67.248000000000005},"last":11.897},{"interface":"front","average":{"1min":11.738,"5min":10.41,"15min":12.5},"min":{"1min":6.4450000000000003,"5min":5.3300000000000001,"15min":5.3300000000000001},"max":{"1min":43.101999999999997,"5min":43.101999999999997,"15min":77.266999999999996},"last":6.5279999999999996}]},{"osd":6,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":14.673,"5min":12.750999999999999,"15min":15.19},"min":{"1min":7.4160000000000004,"5min":5.915,"15min":5.915},"max":{"1min":44.587000000000003,"5min":44.587000000000003,"15min":76.751999999999995},"last":17.959},{"interface":"front","average":{"1min":20.553000000000001,"5min":18.672000000000001,"15min":20.558},"min":{"1min":12.913,"5min":8.8390000000000004,"15min":8.8390000000000004},"max":{"1min":45.366,"5min":45.366,"15min":57.101999999999997},"last":12.577}]},{"osd":7,"last update":"Wed Dec 6 18:33:12 2023","interfaces":[{"interface":"back","average":{"1min":17.077999999999999,"5min":15.398999999999999,"15min":14.606999999999999},"min":{"1min":10.875,"5min":7.5519999999999996,"15min":7.5519999999999996},"max":{"1min":29.731000000000002,"5min":29.965,"15min":29.965},"last":20.346},{"interface":"front","average":{"1min":13.878,"5min":12.369999999999999,"15min":12.048},"min":{"1min":9.3049999999999997,"5min":8.2829999999999995,"15min":8.2829999999999995},"max":{"1min":27.423999999999999,"5min":27.423999999999999,"15min":27.423999999999999},"last":11.307}]}]},{"osd":4,"up_from":15,"seq":64424509526,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":909312,"data_stored":325962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":9.5589999999999993,"5min":9.7349999999999994,"15min":12.707000000000001},"min":{"1min":6.0220000000000002,"5min":3.7429999999999999,"15min":3.7429999999999999},"max":{"1min":17.300999999999998,"5min":41.076000000000001,"15min":53.298999999999999},"last":6.1520000000000001},{"interface":"front","average":{"1min":10.759,"5min":9.9280000000000008,"15min":10.686},"min":{"1min":4.9930000000000003,"5min":4.8140000000000001,"15min":3.9670000000000001},"max":{"1min":29.567,"5min":35.863,"15min":52.567},"last":8.6010000000000009}]},{"osd":1,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":11.622,"5min":11.404999999999999,"15min":13.449999999999999},"min":{"1min":8.0660000000000007,"5min":5.8390000000000004,"15min":5.835},"max":{"1min":18.09,"5min":27.251999999999999,"15min":49.183},"last":7.0549999999999997},{"interface":"front","average":{"1min":12.454000000000001,"5min":12.068,"15min":14.85},"min":{"1min":6.6849999999999996,"5min":5.96,"15min":5.96},"max":{"1min":19.509,"5min":26.495999999999999,"15min":45.412999999999997},"last":13.031000000000001}]},{"osd":2,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":15.634,"5min":12.238,"15min":20.167000000000002},"min":{"1min":7.4009999999999998,"5min":5.6390000000000002,"15min":5.6390000000000002},"max":{"1min":35.258000000000003,"5min":44.649000000000001,"15min":85.394000000000005},"last":18.821000000000002},{"interface":"front","average":{"1min":17.91,"5min":14.865,"15min":19.396999999999998},"min":{"1min":7.5209999999999999,"5min":6.3639999999999999,"15min":6.3639999999999999},"max":{"1min":34.097999999999999,"5min":42.500999999999998,"15min":69.530000000000001},"last":7.9269999999999996}]},{"osd":3,"last update":"Wed Dec 6 18:33:14 2023","interfaces":[{"interface":"back","average":{"1min":17.870999999999999,"5min":15.256,"15min":14.686},"min":{"1min":8.0250000000000004,"5min":8.0039999999999996,"15min":7.008},"max":{"1min":37.561999999999998,"5min":37.561999999999998,"15min":37.561999999999998},"last":9.8439999999999994},{"interface":"front","average":{"1min":15.468,"5min":13.134,"15min":14.765000000000001},"min":{"1min":7.3650000000000002,"5min":5.5549999999999997,"15min":5.5549999999999997},"max":{"1min":36.103999999999999,"5min":36.103999999999999,"15min":36.103999999999999},"last":12.433}]},{"osd":5,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":13.163,"5min":13.558,"15min":16.303000000000001},"min":{"1min":9.2739999999999991,"5min":5.4969999999999999,"15min":5.4969999999999999},"max":{"1min":20.664000000000001,"5min":53.840000000000003,"15min":78.798000000000002},"last":17.768999999999998},{"interface":"front","average":{"1min":18.699999999999999,"5min":17.393000000000001,"15min":18.565999999999999},"min":{"1min":11.209,"5min":8.0519999999999996,"15min":6.4450000000000003},"max":{"1min":27.582000000000001,"5min":52.781999999999996,"15min":55.814999999999998},"last":10.425000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:33:04 2023","interfaces":[{"interface":"back","average":{"1min":19.576000000000001,"5min":17.699000000000002,"15min":17.986999999999998},"min":{"1min":9.2100000000000009,"5min":8.2300000000000004,"15min":7.7850000000000001},"max":{"1min":41.723999999999997,"5min":51.149000000000001,"15min":51.149000000000001},"last":9.3170000000000002},{"interface":"front","average":{"1min":15.221,"5min":14.712999999999999,"15min":16.190999999999999},"min":{"1min":9.6809999999999992,"5min":8.5139999999999993,"15min":7.8700000000000001},"max":{"1min":27.405000000000001,"5min":40.426000000000002,"15min":40.426000000000002},"last":11.760999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:33:14 2023","interfaces":[{"interface":"back","average":{"1min":15.782999999999999,"5min":17.422999999999998,"15min":18.308},"min":{"1min":8.093,"5min":8.093,"15min":8.093},"max":{"1min":30.408000000000001,"5min":60.390000000000001,"15min":60.390000000000001},"last":13.724},{"interface":"front","average":{"1min":20.154,"5min":21.510000000000002,"15min":22.376000000000001},"min":{"1min":15.909000000000001,"5min":13.69,"15min":13.69},"max":{"1min":29.472000000000001,"5min":57.606999999999999,"15min":57.606999999999999},"last":11.083}]}]},{"osd":7,"up_from":61,"seq":261993005101,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33084,"kb_used_data":880,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94338756,"statfs":{"total":96636764160,"available":96602886144,"internally_reserved":0,"allocated":901120,"data_stored":321826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4395,"internal_metadata":28634837},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":9.8659999999999997,"5min":10.045999999999999,"15min":10.109},"min":{"1min":4.4359999999999999,"5min":4.1059999999999999,"15min":4.1059999999999999},"max":{"1min":23.795000000000002,"5min":37.481999999999999,"15min":37.481999999999999},"last":22.285},{"interface":"front","average":{"1min":10.321999999999999,"5min":10.752000000000001,"15min":10.927},"min":{"1min":5.4909999999999997,"5min":4.2530000000000001,"15min":4.2530000000000001},"max":{"1min":29.928999999999998,"5min":38.475999999999999,"15min":38.475999999999999},"last":11.013}]},{"osd":1,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":12.515000000000001,"5min":14.898999999999999,"15min":16.606000000000002},"min":{"1min":4.6929999999999996,"5min":4.6929999999999996,"15min":4.6929999999999996},"max":{"1min":30.350000000000001,"5min":64.765000000000001,"15min":64.765000000000001},"last":6.8879999999999999},{"interface":"front","average":{"1min":15.795,"5min":16.404,"15min":17.541},"min":{"1min":8.0419999999999998,"5min":7.0739999999999998,"15min":7.0739999999999998},"max":{"1min":34.390000000000001,"5min":65.994,"15min":65.994},"last":11.638}]},{"osd":2,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":16.215,"5min":16.803000000000001,"15min":17.611999999999998},"min":{"1min":8.734,"5min":7.9909999999999997,"15min":7.9909999999999997},"max":{"1min":36.622,"5min":46.478000000000002,"15min":46.478000000000002},"last":24.219000000000001},{"interface":"front","average":{"1min":14.967000000000001,"5min":14.003,"15min":14.08},"min":{"1min":6.5910000000000002,"5min":6.5910000000000002,"15min":6.5910000000000002},"max":{"1min":37.026000000000003,"5min":41.603999999999999,"15min":41.603999999999999},"last":21.701000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":13.579000000000001,"5min":15.653,"15min":16.725000000000001},"min":{"1min":7.4950000000000001,"5min":6.8319999999999999,"15min":6.8319999999999999},"max":{"1min":45.731000000000002,"5min":60.463999999999999,"15min":60.463999999999999},"last":11.949},{"interface":"front","average":{"1min":18.077000000000002,"5min":18.809000000000001,"15min":19.835000000000001},"min":{"1min":10.327,"5min":9.7370000000000001,"15min":9.7370000000000001},"max":{"1min":47.265999999999998,"5min":56.286000000000001,"15min":56.286000000000001},"last":23.565999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":19.510999999999999,"5min":20.824999999999999,"15min":21.161000000000001},"min":{"1min":9.6180000000000003,"5min":8.4960000000000004,"15min":8.4960000000000004},"max":{"1min":50.741999999999997,"5min":57.716999999999999,"15min":57.716999999999999},"last":25.693000000000001},{"interface":"front","average":{"1min":15.906000000000001,"5min":18.274000000000001,"15min":18.507000000000001},"min":{"1min":7.3840000000000003,"5min":7.3840000000000003,"15min":7.3840000000000003},"max":{"1min":46.180999999999997,"5min":67.159000000000006,"15min":67.159000000000006},"last":9.0180000000000007}]},{"osd":5,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":14.403,"5min":14.342000000000001,"15min":14.606},"min":{"1min":7.7510000000000003,"5min":6.3399999999999999,"15min":6.3399999999999999},"max":{"1min":44.884,"5min":44.884,"15min":44.884},"last":7.7640000000000002},{"interface":"front","average":{"1min":16.992999999999999,"5min":18.634,"15min":19.498000000000001},"min":{"1min":10.260999999999999,"5min":10.111000000000001,"15min":10.111000000000001},"max":{"1min":31.847999999999999,"5min":70.900000000000006,"15min":70.900000000000006},"last":14.43}]},{"osd":6,"last update":"Wed Dec 6 18:33:10 2023","interfaces":[{"interface":"back","average":{"1min":17.672000000000001,"5min":16.728999999999999,"15min":16.760000000000002},"min":{"1min":11.443,"5min":8.7170000000000005,"15min":8.7170000000000005},"max":{"1min":41.091000000000001,"5min":54.393000000000001,"15min":54.393000000000001},"last":14.724},{"interface":"front","average":{"1min":18.923999999999999,"5min":20.111000000000001,"15min":20.655999999999999},"min":{"1min":10.144,"5min":10.144,"15min":10.144},"max":{"1min":42.084000000000003,"5min":69.263000000000005,"15min":69.263000000000005},"last":24.937999999999999}]}]},{"osd":6,"up_from":0,"seq":60129542227,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:33:42.640 INFO:tasks.thrashosds.thrasher:Setting 2.1 to [1, 5, 4, 7] 2023-12-06T18:33:42.640 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.1', '1', '5', '4', '7'] 2023-12-06T18:33:42.640 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.1 1 5 4 7 2023-12-06T18:33:42.729 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:42.729 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:42.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:42.729 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:33:42.729 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.729 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:33:42.729 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:33:42.729 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 406.11171624999997, 2023-12-06T18:33:42.729 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:33:42.729 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.729 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.730 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:33:42.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:33:42.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.730 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.730 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.730 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.730 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:33:42.730 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:33:42.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:33:42.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:33:42.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:33:42.731 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:33:42.731 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.731 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.731 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.731 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:33:42.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:33:42.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:33:42.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:33:42.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.732 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:33:42.732 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:33:42.732 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.732 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.732 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:33:42.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:33:42.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.733 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.733 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:33:42.733 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:33:42.733 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.733 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.733 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.733 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.733 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.733 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:33:42.733 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:33:42.733 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 405.088889241, 2023-12-06T18:33:42.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:33:42.734 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.734 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.734 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:33:42.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.734 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.734 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.734 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:33:42.734 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.734 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:33:42.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:33:42.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:33:42.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:33:42.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.735 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.735 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:33:42.735 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:33:42.735 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.735 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:33:42.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:33:42.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:33:42.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:33:42.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.736 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.736 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:33:42.736 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:33:42.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.736 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:33:42.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:33:42.737 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.737 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.737 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.737 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:33:42.737 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:33:42.737 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.738 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.738 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.738 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.738 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:33:42.738 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:33:42.738 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 405.07684260299999, 2023-12-06T18:33:42.738 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:33:42.738 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.738 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.738 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.738 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.738 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.738 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:33:42.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:33:42.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:33:42.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:33:42.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.739 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.739 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.739 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:33:42.739 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:33:42.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:33:42.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:33:42.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:33:42.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:33:42.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.740 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.740 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:33:42.740 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:33:42.740 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.740 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:33:42.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:33:42.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:33:42.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:33:42.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.741 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.741 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.741 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:33:42.741 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:33:42.741 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.741 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.741 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.742 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:33:42.742 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:33:42.742 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 405.05544199600001, 2023-12-06T18:33:42.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:33:42.742 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.742 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.742 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:33:42.742 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.742 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.742 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.742 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:33:42.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:33:42.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:33:42.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:33:42.743 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:33:42.743 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.743 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.743 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.743 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:33:42.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:33:42.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:33:42.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:33:42.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.744 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:33:42.744 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:33:42.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.744 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.744 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:33:42.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:33:42.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:33:42.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:33:42.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.745 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.745 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.745 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:33:42.745 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:33:42.745 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.745 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.745 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.746 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:33:42.746 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:33:42.746 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 405.05426267299998, 2023-12-06T18:33:42.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:33:42.746 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.746 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.746 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.746 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.746 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.746 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:33:42.746 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:33:42.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:33:42.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:33:42.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.747 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.747 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.747 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:33:42.747 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:33:42.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:33:42.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:33:42.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:33:42.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:33:42.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.748 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.748 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.748 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:33:42.748 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:33:42.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:33:42.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:33:42.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:33:42.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:33:42.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.749 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.749 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.749 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:33:42.749 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:33:42.750 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.750 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.750 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.750 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:33:42.750 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:33:42.750 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 405.05369069199998, 2023-12-06T18:33:42.750 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:33:42.750 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.750 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.750 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.750 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:33:42.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:33:42.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.751 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.751 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:33:42.751 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:33:42.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.751 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:33:42.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:33:42.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:33:42.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:33:42.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:33:42.752 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:33:42.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.752 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.752 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.752 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:33:42.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:33:42.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:33:42.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:33:42.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:33:42.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:33:42.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.753 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.753 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.753 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:33:42.753 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:33:42.753 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.754 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.754 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.754 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:33:42.754 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:33:42.754 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 404.60785522200001, 2023-12-06T18:33:42.754 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:33:42.754 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.754 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.754 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.754 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.754 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.754 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:33:42.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:33:42.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:33:42.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:33:42.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.755 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.755 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:33:42.755 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:33:42.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.755 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:33:42.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:33:42.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:33:42.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:33:42.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.756 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:33:42.756 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:33:42.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.756 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.756 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:33:42.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:33:42.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:33:42.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:33:42.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.757 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.757 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.757 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:33:42.757 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:33:42.757 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.757 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.757 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.758 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:33:42.758 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:33:42.758 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 404.60625536600003, 2023-12-06T18:33:42.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:33:42.758 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.758 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.758 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.758 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:33:42.758 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.758 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.758 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:33:42.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:33:42.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:33:42.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.759 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.759 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:33:42.759 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:33:42.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.759 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.760 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:33:42.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:33:42.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.760 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:33:42.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:33:42.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.760 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.760 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:33:42.760 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:33:42.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.760 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:33:42.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:33:42.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:33:42.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:33:42.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.761 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.761 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.761 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:33:42.761 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:33:42.761 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.761 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.762 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.762 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:33:42.762 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:33:42.762 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 404.22704974999999, 2023-12-06T18:33:42.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:33:42.762 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.762 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.762 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.762 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.762 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.762 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:33:42.762 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:33:42.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:33:42.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:33:42.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.763 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:33:42.763 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:33:42.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.763 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.763 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:33:42.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:33:42.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:33:42.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:33:42.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:33:42.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:33:42.764 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.764 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.764 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.764 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:33:42.764 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:33:42.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:33:42.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:33:42.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.765 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.765 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.765 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:33:42.765 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:33:42.765 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.765 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.765 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.765 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.766 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:33:42.766 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:33:42.766 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 403.98954756799998, 2023-12-06T18:33:42.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:33:42.766 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.766 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.766 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.766 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:33:42.766 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.766 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.766 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.766 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:33:42.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:33:42.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:33:42.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.767 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:33:42.767 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:33:42.767 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.767 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.767 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.768 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:33:42.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:33:42.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.768 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.768 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:33:42.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:33:42.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.768 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.768 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:33:42.768 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:33:42.768 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.768 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:33:42.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:33:42.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:33:42.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:33:42.769 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.769 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.769 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.769 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:33:42.769 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:33:42.769 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.769 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.769 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.770 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:33:42.770 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:33:42.770 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 403.98881956899999, 2023-12-06T18:33:42.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:33:42.770 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.770 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.770 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.770 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.770 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.770 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:33:42.770 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:33:42.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:33:42.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:33:42.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.771 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.771 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:33:42.771 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:33:42.771 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.771 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:33:42.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:33:42.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:33:42.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:33:42.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.772 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.772 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.772 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:33:42.772 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:33:42.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:33:42.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:33:42.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:33:42.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:33:42.773 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.773 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.773 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.773 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:33:42.773 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:33:42.773 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.774 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.774 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.774 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.774 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:33:42.774 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:33:42.774 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 403.98380715899998, 2023-12-06T18:33:42.774 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:33:42.774 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.774 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.774 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.774 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.774 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.774 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:33:42.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:33:42.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:33:42.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:33:42.775 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.775 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.775 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.775 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:33:42.775 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:33:42.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:33:42.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:33:42.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:33:42.776 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:33:42.776 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.776 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.776 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.776 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:33:42.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:33:42.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:33:42.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:33:42.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:33:42.777 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:33:42.777 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.777 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.777 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.777 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:33:42.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:33:42.778 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.778 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.778 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.778 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.778 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:33:42.778 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:33:42.778 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 403.97828007999999, 2023-12-06T18:33:42.778 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:33:42.778 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.778 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.778 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.778 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.778 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:33:42.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:33:42.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:33:42.779 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:33:42.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.779 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.779 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.779 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:33:42.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:33:42.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:33:42.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:33:42.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:33:42.780 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:33:42.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.780 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.780 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.780 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:33:42.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:33:42.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:33:42.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:33:42.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:33:42.781 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:33:42.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.781 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.781 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.781 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:33:42.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:33:42.782 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.782 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.782 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.782 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.782 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.782 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:33:42.782 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:33:42.782 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 403.97776924300001, 2023-12-06T18:33:42.782 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:33:42.782 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.782 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.782 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:33:42.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:33:42.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.783 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.783 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:33:42.783 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:33:42.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.783 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:33:42.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:33:42.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:33:42.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:33:42.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.784 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:33:42.784 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:33:42.784 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.784 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.784 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:33:42.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:33:42.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:33:42.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:33:42.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.785 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.785 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:33:42.785 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:33:42.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.785 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.786 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.786 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:33:42.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:33:42.786 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.786 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.786 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.786 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.786 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.786 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:33:42.786 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:33:42.786 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 403.97730998200001, 2023-12-06T18:33:42.786 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:33:42.786 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.787 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.787 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:33:42.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.787 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:33:42.787 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.787 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.787 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:33:42.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:33:42.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:33:42.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:33:42.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.788 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.788 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.788 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:33:42.788 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:33:42.788 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:33:42.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:33:42.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:33:42.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:33:42.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.789 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:33:42.789 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:33:42.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.789 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.789 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:33:42.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:33:42.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.790 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.790 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:33:42.790 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:33:42.790 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.790 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.790 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.790 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.790 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.790 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:33:42.790 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:33:42.790 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 397.39739417499999, 2023-12-06T18:33:42.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:33:42.791 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.791 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.791 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:33:42.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.791 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.791 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:33:42.791 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.791 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.791 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:33:42.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:33:42.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:33:42.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:33:42.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.792 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:33:42.792 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:33:42.792 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.792 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.792 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:33:42.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:33:42.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:33:42.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:33:42.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:33:42.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:33:42.793 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.793 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.793 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.793 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:33:42.793 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:33:42.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.794 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.794 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:33:42.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:33:42.794 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.794 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.794 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.794 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.794 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.794 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:33:42.794 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:33:42.794 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 397.39115636999998, 2023-12-06T18:33:42.794 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:33:42.794 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.795 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.795 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:33:42.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:33:42.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.795 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.795 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.795 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.795 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:33:42.795 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:33:42.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:33:42.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:33:42.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:33:42.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:33:42.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.796 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.796 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.796 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:33:42.796 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:33:42.796 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:33:42.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:33:42.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:33:42.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:33:42.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.797 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.797 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.797 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:33:42.797 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:33:42.797 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.798 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.798 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:33:42.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:33:42.798 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.798 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.798 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.798 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.798 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.798 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:33:42.798 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:33:42.798 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 397.38509329099998, 2023-12-06T18:33:42.798 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:33:42.798 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.798 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.798 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:33:42.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:33:42.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.799 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.799 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:33:42.799 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:33:42.799 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.799 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:33:42.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:33:42.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:33:42.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:33:42.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:33:42.800 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:33:42.800 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.800 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.800 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.800 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:33:42.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:33:42.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:33:42.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:33:42.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:33:42.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:33:42.801 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.801 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.801 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.801 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:33:42.801 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:33:42.802 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.802 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.802 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.802 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.802 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:33:42.802 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:33:42.802 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 397.24770620599998, 2023-12-06T18:33:42.802 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:33:42.802 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.802 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.802 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.802 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.802 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.802 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:33:42.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:33:42.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:33:42.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:33:42.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.803 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.803 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:33:42.803 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:33:42.803 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.803 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:33:42.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:33:42.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:33:42.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:33:42.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:33:42.804 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:33:42.804 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.804 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.804 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.804 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:33:42.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:33:42.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:33:42.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:33:42.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.805 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.805 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:33:42.805 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:33:42.805 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.805 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.805 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.805 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.805 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.805 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:33:42.805 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:33:42.806 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 397.241035479, 2023-12-06T18:33:42.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:33:42.806 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:42.806 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:42.806 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:42.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:42.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:33:42.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.806 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.806 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.806 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:42.806 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:33:42.806 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:42.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:42.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:33:42.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:33:42.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:42.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:33:42.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:33:42.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.807 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.807 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:42.807 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:33:42.807 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:33:42.807 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:42.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:33:42.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:33:42.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:42.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:33:42.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:33:42.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:42.808 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:33:42.808 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:33:42.808 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.808 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.808 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:42.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:33:42.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:33:42.809 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:42.809 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:42.809 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:42.809 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:33:42.809 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:33:42.809 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.809 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.809 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.809 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:42.809 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:42.809 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:42.810 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:42.724+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:42.810 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:33:42.812 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:42.808+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:42.819 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:33:42.858 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:33:42.913 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:42.910+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:42.983 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:42.983 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:42.983 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:42.983 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:42.992 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:33:43.014 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:43.010+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:43.115 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:43.111+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:43.122 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-12-06T18:33:43.123 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:43.123 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:43.123 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:43.123 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:43.123 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:43.132 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:33:43.216 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:43.212+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:43.318 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:43.314+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:43.319 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:43.319 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:43.319 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:43.319 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:33:43.319 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:43.328 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:33:43.418 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:43.415+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:43.515 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:43.515 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:43.515 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:43.519 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:43.515+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:43.524 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:33:43.620 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:43.617+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:43.652 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":44,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":46,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:33:43.718 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:43.718 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:43.718 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:43.718 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:43.718 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:43.728 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:33:43.742 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:33:43.743 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:33:43.743 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:43.739+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:43.822 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:43.818+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:43.867 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:33:43.868 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:33:43.898 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:43.898 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:43.898 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:43.899 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:33:43.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:43.899 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:33:43.899 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:43.899 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 202.23524810999999, 2023-12-06T18:33:43.899 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:33:43.899 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:43.899 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:43.899 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:43.899 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:43.899 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:43.899 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:33:43.899 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:43.899 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:43.899 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:43.899 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:43.899 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:43.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:43.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:43.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:43.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:43.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:43.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:43.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:43.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:43.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:43.900 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:33:43.900 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:33:43.900 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:43.900 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:43.900 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:43.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:33:43.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:33:43.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:43.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:43.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:43.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:33:43.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:33:43.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:43.901 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:43.901 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:43.901 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:33:43.901 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:33:43.901 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:43.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:43.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:43.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:33:43.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:33:43.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:43.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:43.902 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:43.902 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:33:43.902 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:33:43.902 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:43.902 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:43.902 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:43.902 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:43.902 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:43.902 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:12 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2023-12-06T18:33:43.902 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:43.903 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 82.085301485000002, 2023-12-06T18:33:43.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0032177460000000001, 2023-12-06T18:33:43.903 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:43.903 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:43.903 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:43.903 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:43.903 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:43.903 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 12 2023-12-06T18:33:43.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:43.903 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:43.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:43.903 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:43.903 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:43.903 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:43.903 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:43.903 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:43.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:43.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:43.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:43.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:43.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:43.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:43.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807030+0000", 2023-12-06T18:33:43.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6030000000000001e-05 2023-12-06T18:33:43.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:43.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:43.904 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:43.904 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807210+0000", 2023-12-06T18:33:43.904 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017936800000000001 2023-12-06T18:33:43.904 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:43.904 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:43.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:43.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807443+0000", 2023-12-06T18:33:43.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00023339699999999999 2023-12-06T18:33:43.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:43.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:43.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:43.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807614+0000", 2023-12-06T18:33:43.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000171032 2023-12-06T18:33:43.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:43.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:43.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:43.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.808696+0000", 2023-12-06T18:33:43.905 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010817190000000001 2023-12-06T18:33:43.905 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:43.905 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:43.905 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:43.905 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.810232+0000", 2023-12-06T18:33:43.906 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0015361999999999999 2023-12-06T18:33:43.906 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:43.906 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:43.906 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:43.906 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:43.906 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:43.906 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:43.907 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:33:43.949 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:43.945+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:43.960 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:33:43.960 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:33:44.024 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:44.020+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:44.057 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:44.058 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:44.068 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:33:44.074 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:33:44.075 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:33:44.125 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:44.122+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:44.171 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:33:44.172 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:33:44.207 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:44.207 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:44.216 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:33:44.256 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:33:44.256 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:33:44.257 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:44.253+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:44.327 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:44.323+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:44.362 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:33:44.363 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:33:44.426 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:44.426 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:44.427 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:44.424+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:44.434 INFO:teuthology.orchestra.run.smithi061.stderr:set 2.1 pg_upmap_items mapping to [1->5,4->7] 2023-12-06T18:33:44.435 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:33:44.453 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:33:44.453 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:33:44.492 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is unset 2023-12-06T18:33:44.529 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:44.525+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:44.596 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:44.596 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:44.605 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:33:44.650 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:44.646+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:44.731 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:44.727+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:44.742 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:44.743 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:44.752 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:33:44.837 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:44.833+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:44.929 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:44.930 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:44.932 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:44.929+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:44.938 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:33:45.033 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:45.030+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:45.116 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:45.116 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:45.125 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:33:45.168 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:45.163+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:45.236 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:45.232+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:45.303 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:45.304 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:45.314 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:33:45.355 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:45.351+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:45.438 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:45.435+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:45.493 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:45.493 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:45.503 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:33:45.548 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:45.544+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:45.640 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:45.636+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:45.641 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:45.641 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:45.650 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:33:45.740 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:45.737+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:45.841 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:45.838+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:45.843 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:45.843 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:45.852 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:33:45.942 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:45.939+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:46.031 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:46.031 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:46.039 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:33:46.043 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:46.040+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:46.144 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:46.141+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:46.170 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:46.170 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:46.180 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:33:46.264 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:46.261+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:46.346 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:46.343+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:46.350 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:46.351 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:46.362 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:33:46.447 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:46.443+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:46.469 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:33:46.507 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-12-06T18:33:46.531 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:46.531 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:46.541 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:33:46.583 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:46.579+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:46.648 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:46.645+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:46.668 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:33:46.712 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:46.712 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:46.721 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:33:46.749 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:46.746+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:46.774 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:33:46.850 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:46.846+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:46.852 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:46.852 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:46.862 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:33:46.867 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:33:46.867 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:33:46.951 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:46.948+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:46.977 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:33:46.978 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:33:47.000 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:47.000 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:47.009 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:33:47.052 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:47.048+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:47.070 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:33:47.070 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:33:47.142 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:47.142 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:47.151 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:33:47.152 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:47.149+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:47.161 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:33:47.161 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:33:47.254 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:47.251+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:47.256 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:33:47.257 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:33:47.296 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:47.296 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:47.305 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:33:47.360 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:33:47.360 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:33:47.364 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:47.360+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:47.455 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:33:47.455 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:33:47.457 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:47.454+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:33:47.480 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:47.489 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:33:47.554 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":44,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":50,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:33:47.556 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:47.553+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:47.621 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:47.621 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:47.621 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:47.621 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:47.630 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:33:47.657 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:47.654+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:47.758 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:47.754+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:47.763 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:47.763 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:47.763 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:47.763 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:47.763 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:47.773 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:33:47.859 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:47.856+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:47.943 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:47.944 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:47.944 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:47.944 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:33:47.944 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:47.944 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:33:47.944 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:47.944 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 206.28032060300001, 2023-12-06T18:33:47.944 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:33:47.944 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:47.944 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:47.944 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:47.944 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:47.944 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:47.944 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:33:47.944 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:47.944 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:47.945 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:47.945 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:47.945 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:47.945 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:47.945 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:47.945 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:47.945 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:47.945 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:47.945 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:47.945 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:47.945 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:47.945 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:47.945 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:33:47.945 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:33:47.945 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:47.945 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:47.946 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:47.946 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:33:47.946 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:33:47.946 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:47.946 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:47.946 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:47.946 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:33:47.946 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:33:47.946 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:47.946 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:47.946 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:47.946 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:33:47.946 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:33:47.946 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:47.946 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:47.947 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:47.947 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:33:47.947 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:33:47.947 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:47.947 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:47.948 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:47.948 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:33:47.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:33:47.948 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:47.948 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:47.948 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:47.948 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:47.948 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:47.948 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:12 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2023-12-06T18:33:47.948 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:47.948 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 86.130373977999994, 2023-12-06T18:33:47.948 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0032177460000000001, 2023-12-06T18:33:47.948 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:47.948 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:47.949 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:47.949 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:47.949 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:47.949 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 12 2023-12-06T18:33:47.949 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:47.949 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:47.949 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:47.949 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:47.949 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:47.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:47.949 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:47.949 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:47.949 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:47.949 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:47.949 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:47.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:47.950 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:47.950 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:47.950 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807030+0000", 2023-12-06T18:33:47.950 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6030000000000001e-05 2023-12-06T18:33:47.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:47.950 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:47.950 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:47.950 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807210+0000", 2023-12-06T18:33:47.950 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017936800000000001 2023-12-06T18:33:47.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:47.950 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:47.950 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:47.950 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807443+0000", 2023-12-06T18:33:47.950 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00023339699999999999 2023-12-06T18:33:47.950 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:47.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:47.951 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:47.951 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807614+0000", 2023-12-06T18:33:47.951 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000171032 2023-12-06T18:33:47.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:47.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:47.951 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:47.951 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.808696+0000", 2023-12-06T18:33:47.951 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010817190000000001 2023-12-06T18:33:47.951 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:47.951 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:47.951 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:47.951 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.810232+0000", 2023-12-06T18:33:47.951 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0015361999999999999 2023-12-06T18:33:47.951 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:47.951 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:47.952 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:47.952 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:47.952 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:47.952 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:47.953 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:33:47.998 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:47.994+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:48.060 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:48.057+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:48.093 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:48.093 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:48.094 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:48.094 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:48.102 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:33:48.187 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:48.183+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:48.261 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:48.258+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:48.280 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:48.281 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:48.281 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:48.281 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:48.281 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:48.290 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:33:48.327 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is unset 2023-12-06T18:33:48.363 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:48.359+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:48.463 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:48.459+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:48.469 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:48.469 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:48.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:48.469 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:48.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.469 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 110/109 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 23'75, data_included: [0~4096,65536~66064], data_size: 70160, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@23'75, size: 131600, copy_subset: [(0, 131600)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 131600, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:33:48.469 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:33:21.824095+0000", 2023-12-06T18:33:48.470 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 26.638130901, 2023-12-06T18:33:48.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.22522905200000001, 2023-12-06T18:33:48.470 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:48.470 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:33:48.470 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:48.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:48.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.824095+0000", 2023-12-06T18:33:48.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:48.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:48.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:48.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.824095+0000", 2023-12-06T18:33:48.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:48.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:48.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:48.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.824111+0000", 2023-12-06T18:33:48.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5596000000000001e-05 2023-12-06T18:33:48.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:48.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:48.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.866577+0000", 2023-12-06T18:33:48.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.042466773999999999 2023-12-06T18:33:48.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:48.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:48.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.871496+0000", 2023-12-06T18:33:48.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049188699999999997 2023-12-06T18:33:48.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:48.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:48.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.886281+0000", 2023-12-06T18:33:48.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.014784535 2023-12-06T18:33:48.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:48.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:48.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.903661+0000", 2023-12-06T18:33:48.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.017379950000000002 2023-12-06T18:33:48.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:48.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:48.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.908970+0000", 2023-12-06T18:33:48.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.005309322 2023-12-06T18:33:48.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:48.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:48.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.049324+0000", 2023-12-06T18:33:48.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.140354005 2023-12-06T18:33:48.473 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:48.473 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:48.473 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:48.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:48.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.473 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 110/109 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:33:48.473 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:33:22.129048+0000", 2023-12-06T18:33:48.473 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 26.333177715000001, 2023-12-06T18:33:48.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.029478353999999998, 2023-12-06T18:33:48.473 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:48.474 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:33:48.474 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:48.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:48.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.129048+0000", 2023-12-06T18:33:48.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:48.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:48.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:48.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.129048+0000", 2023-12-06T18:33:48.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:48.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:48.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:48.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.129058+0000", 2023-12-06T18:33:48.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 9.8630000000000003e-06 2023-12-06T18:33:48.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:48.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:48.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.133519+0000", 2023-12-06T18:33:48.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0044613229999999997 2023-12-06T18:33:48.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:48.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:48.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.133935+0000", 2023-12-06T18:33:48.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00041585999999999999 2023-12-06T18:33:48.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:48.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:48.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.134363+0000", 2023-12-06T18:33:48.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00042782200000000001 2023-12-06T18:33:48.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:48.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:48.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.136423+0000", 2023-12-06T18:33:48.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002059759 2023-12-06T18:33:48.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:48.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:48.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.136759+0000", 2023-12-06T18:33:48.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000335898 2023-12-06T18:33:48.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:48.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:48.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:48.477 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.158527+0000", 2023-12-06T18:33:48.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021767828999999999 2023-12-06T18:33:48.477 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:48.477 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:48.477 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:48.477 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:48.477 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:48.477 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:48.479 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:33:48.563 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:48.560+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:48.656 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:48.656 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:48.657 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:48.657 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:48.664 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:48.661+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:48.665 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:33:48.770 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:48.767+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:48.842 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:48.842 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:48.842 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:48.842 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:48.842 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:48.852 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:33:48.867 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:48.864+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:48.968 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:48.965+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:49.048 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:49.048 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:49.048 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:49.048 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:49.048 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.049 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:33:49.049 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:49.049 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 413.63741138799998, 2023-12-06T18:33:49.049 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:33:49.049 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.049 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.049 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.049 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.049 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.049 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:33:49.049 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.049 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.049 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.049 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:49.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:49.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.050 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.050 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.050 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:33:49.050 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:33:49.050 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:33:49.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:33:49.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:33:49.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:33:49.051 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.051 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.051 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.051 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:33:49.051 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:33:49.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:33:49.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:33:49.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:33:49.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:33:49.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:33:49.052 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.052 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.052 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.052 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:33:49.052 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:33:49.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:33:49.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:33:49.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:33:49.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:33:49.053 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.053 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.053 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.053 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:33:49.053 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:33:49.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.054 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:33:49.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:33:49.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.054 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:33:49.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:33:49.054 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.054 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.054 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.054 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:33:49.054 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:33:49.055 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.055 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.055 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.055 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.055 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.055 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:49.055 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:49.055 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 411.52883814900002, 2023-12-06T18:33:49.055 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:33:49.055 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.055 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.055 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.055 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.055 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.056 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:33:49.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.056 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:49.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.056 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:49.056 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.056 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.056 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.056 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:33:49.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:33:49.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:33:49.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:33:49.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.057 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:33:49.057 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:33:49.057 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.057 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.057 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:33:49.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:33:49.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:33:49.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:33:49.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.058 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.058 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:33:49.058 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:33:49.058 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.058 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:33:49.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:33:49.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:33:49.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:33:49.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.059 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.059 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:33:49.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:33:49.059 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.060 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:33:49.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:33:49.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.060 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:33:49.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:33:49.060 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.060 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.060 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.060 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:49.060 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:49.061 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 411.52809291699998, 2023-12-06T18:33:49.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:33:49.061 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.061 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.061 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.061 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.061 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.061 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:33:49.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.061 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.061 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.061 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:49.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:49.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:33:49.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:33:49.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:33:49.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:33:49.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:33:49.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:33:49.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:33:49.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:33:49.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.063 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.063 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:33:49.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:33:49.063 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:33:49.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:33:49.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:33:49.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:33:49.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.064 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.064 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.064 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:33:49.064 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:33:49.064 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:33:49.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:33:49.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:33:49.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:33:49.065 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.065 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.065 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.065 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:33:49.065 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:33:49.065 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.065 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.066 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.066 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.066 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:49.066 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:49.066 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 411.52740916200003, 2023-12-06T18:33:49.066 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:33:49.066 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.066 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.066 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.066 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.066 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.066 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:33:49.066 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.067 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.067 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.067 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.067 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:49.067 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.067 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.067 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.067 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.067 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:49.067 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.067 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.067 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.067 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.067 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:33:49.067 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:33:49.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:33:49.068 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:33:49.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:33:49.068 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:33:49.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.068 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.068 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:33:49.068 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:33:49.068 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.068 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:33:49.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:33:49.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:33:49.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:33:49.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.069 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.069 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.069 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:33:49.069 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:33:49.069 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.070 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.070 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:33:49.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:33:49.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.070 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.070 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:33:49.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:33:49.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.070 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.070 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.070 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:33:49.070 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:33:49.070 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.071 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.071 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:33:49.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:33:49.071 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.071 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.071 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.071 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.071 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.071 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:49.071 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:49.071 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 411.52183590700002, 2023-12-06T18:33:49.071 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:33:49.071 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.071 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.071 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.072 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.072 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.072 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:33:49.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.072 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.072 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.072 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.072 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:49.072 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.072 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.072 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.072 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:49.072 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.072 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:33:49.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:33:49.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:33:49.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:33:49.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.073 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.073 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.073 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:33:49.073 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:33:49.073 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.074 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:33:49.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:33:49.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.074 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:33:49.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:33:49.074 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.074 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.074 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.074 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:33:49.074 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:33:49.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.075 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:33:49.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:33:49.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.075 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:33:49.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:33:49.075 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.075 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.075 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.075 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:33:49.075 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:33:49.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.076 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:33:49.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:33:49.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.076 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.076 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:33:49.076 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:33:49.076 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.076 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.076 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.076 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.076 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.077 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:49.077 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:49.077 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 411.52116226700002, 2023-12-06T18:33:49.077 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:33:49.077 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.077 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.077 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.077 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.077 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.077 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:33:49.077 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.077 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.077 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.077 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.077 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:49.077 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.078 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.078 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:49.078 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.078 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.078 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:33:49.078 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:33:49.078 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.078 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.078 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.078 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:33:49.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:33:49.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:33:49.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:33:49.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:33:49.079 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:33:49.079 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.079 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.079 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.079 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:33:49.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:33:49.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:33:49.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:33:49.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:33:49.080 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:33:49.080 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.080 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.080 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.080 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:33:49.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:33:49.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.081 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:33:49.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:33:49.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.081 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:33:49.081 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:33:49.081 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.081 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.081 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.081 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:33:49.082 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:33:49.082 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.082 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.082 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.082 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.082 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.082 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:49.082 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:49.082 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 411.52050006799999, 2023-12-06T18:33:49.082 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:33:49.082 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.082 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.082 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.082 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.083 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.083 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:33:49.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.083 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.083 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.083 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.083 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:49.083 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.083 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.083 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.083 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:49.083 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.083 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.083 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.083 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:33:49.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:33:49.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:33:49.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:33:49.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.084 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:33:49.084 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:33:49.084 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.084 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.084 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:33:49.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:33:49.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:33:49.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:33:49.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:33:49.085 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:33:49.085 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.085 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.085 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.085 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:33:49.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:33:49.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:33:49.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:33:49.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.086 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:33:49.086 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:33:49.086 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.086 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.086 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:33:49.087 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:33:49.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.087 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.087 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:33:49.087 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:33:49.087 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.087 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.087 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.087 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.087 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.088 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:49.088 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:49.088 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 411.44779063300001, 2023-12-06T18:33:49.088 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:33:49.088 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.088 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.088 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.088 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.088 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.088 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:33:49.088 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.088 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.088 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:49.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:49.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.089 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:33:49.089 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:33:49.089 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.089 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.089 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:33:49.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:33:49.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:33:49.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:33:49.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.090 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.090 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.090 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:33:49.090 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:33:49.090 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:33:49.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:33:49.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:33:49.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:33:49.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.091 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.091 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:33:49.091 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:33:49.091 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.091 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:33:49.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:33:49.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:33:49.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:33:49.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.092 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.092 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.092 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:33:49.092 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:33:49.092 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.093 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.093 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.093 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:33:49.093 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:33:49.093 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.093 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.093 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.093 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.093 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.093 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:49.093 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:49.093 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 411.44710892400002, 2023-12-06T18:33:49.093 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:33:49.093 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.093 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.093 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.094 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.094 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.094 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:33:49.094 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.094 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.094 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.094 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.094 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:49.094 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.094 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.094 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.094 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.094 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:49.094 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.095 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.095 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.095 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:33:49.095 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:33:49.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.095 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.095 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.095 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:33:49.095 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:33:49.095 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.095 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.095 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.095 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:33:49.095 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:33:49.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.096 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.096 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:33:49.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:33:49.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.096 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.096 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:33:49.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:33:49.096 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.096 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.096 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.096 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:33:49.096 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:33:49.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.097 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.097 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:33:49.097 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:33:49.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.097 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.097 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:33:49.097 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:33:49.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.097 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.097 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.097 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:33:49.097 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:33:49.097 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.098 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.098 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.098 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:33:49.098 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:33:49.098 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.098 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.098 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.098 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:33:49.098 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:33:49.098 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.098 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.098 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.098 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.098 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.098 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:49.099 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:49.099 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 410.36830483599999, 2023-12-06T18:33:49.099 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:33:49.099 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.099 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.099 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.099 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.099 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.099 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:33:49.099 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.099 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.099 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.099 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.099 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:49.099 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.100 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:49.100 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.100 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:33:49.100 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:33:49.100 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.100 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.100 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.100 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:33:49.100 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:33:49.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:33:49.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:33:49.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:33:49.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:33:49.101 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.101 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.101 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.101 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:33:49.101 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:33:49.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:33:49.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:33:49.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:33:49.102 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:33:49.102 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.102 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.102 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.102 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:33:49.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:33:49.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.103 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.103 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.103 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:33:49.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:33:49.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.103 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.103 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.103 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:33:49.103 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:33:49.103 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.103 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.103 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.103 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:33:49.104 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:33:49.104 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.104 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.104 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.104 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.104 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.104 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:49.104 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:49.104 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 410.36688826199997, 2023-12-06T18:33:49.104 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:33:49.104 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.104 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.104 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.104 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.104 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.105 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:33:49.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.105 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.105 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.105 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:49.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.105 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.105 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:49.105 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.105 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.105 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.105 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.106 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:33:49.106 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:33:49.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.106 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.106 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:33:49.106 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:33:49.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.106 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.106 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:33:49.106 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:33:49.106 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.106 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.106 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:33:49.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:33:49.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.107 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:33:49.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:33:49.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.107 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:33:49.107 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:33:49.107 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.107 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.107 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.107 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:33:49.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:33:49.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.108 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.108 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:33:49.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:33:49.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.108 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.108 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:33:49.108 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:33:49.108 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.108 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.108 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.108 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:33:49.109 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:33:49.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.109 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.109 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.109 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:33:49.109 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:33:49.109 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.109 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.109 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.109 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.109 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.109 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:49.109 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:49.109 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 410.365803137, 2023-12-06T18:33:49.109 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:33:49.110 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.110 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.110 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.110 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.110 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.110 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:33:49.110 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.110 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.110 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.110 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.110 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:49.110 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.110 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.110 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.110 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:49.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:33:49.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:33:49.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:33:49.111 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:33:49.111 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.111 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.111 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.111 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:33:49.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:33:49.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:33:49.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:33:49.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:33:49.112 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:33:49.112 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.112 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.112 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.112 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:33:49.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:33:49.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:33:49.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:33:49.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:33:49.113 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:33:49.113 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.113 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.113 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.113 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:33:49.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:33:49.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.114 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:33:49.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:33:49.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.114 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.114 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.114 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:33:49.114 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:33:49.114 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.114 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.114 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.114 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.115 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:49.115 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:49.115 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 410.33065403500001, 2023-12-06T18:33:49.115 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:33:49.115 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.115 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.115 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.115 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.115 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.115 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:33:49.115 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.115 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.115 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.115 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:49.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:49.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.116 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:33:49.116 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:33:49.116 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.116 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.116 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:33:49.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:33:49.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:33:49.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:33:49.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.117 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:33:49.117 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:33:49.117 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.117 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.117 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:33:49.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:33:49.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:33:49.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:33:49.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.118 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:33:49.118 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:33:49.118 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.118 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.118 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:33:49.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:33:49.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:33:49.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:33:49.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.119 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.119 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:33:49.119 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:33:49.119 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.119 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.120 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.120 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:33:49.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:33:49.120 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.120 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.120 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.120 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.120 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.120 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:49.120 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:49.120 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 403.78092350999998, 2023-12-06T18:33:49.120 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:33:49.120 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.120 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.120 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.120 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.121 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.121 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:33:49.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.121 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:49.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.121 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:49.121 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.121 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.121 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.121 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:33:49.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:33:49.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:33:49.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:33:49.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.122 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.122 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:33:49.122 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:33:49.122 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.122 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:33:49.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:33:49.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:33:49.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:33:49.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.123 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.123 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:33:49.123 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:33:49.123 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.123 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.124 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.124 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:33:49.124 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:33:49.124 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.124 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.124 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.124 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:33:49.124 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:33:49.124 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.124 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.124 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.124 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:33:49.124 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:33:49.124 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.124 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.124 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.125 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:33:49.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:33:49.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.125 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.125 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:33:49.125 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:33:49.125 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.125 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.125 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.125 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.125 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.125 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:49.125 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:49.126 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 403.78011790199997, 2023-12-06T18:33:49.126 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:33:49.126 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.126 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.126 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.126 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.126 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.126 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:33:49.126 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.126 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.126 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.126 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.126 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:49.126 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.126 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.127 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.127 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:49.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.127 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.127 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:33:49.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:33:49.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.127 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.127 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.127 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:33:49.127 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:33:49.127 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.128 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:33:49.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:33:49.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.128 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:33:49.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:33:49.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.128 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.128 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.128 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:33:49.128 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:33:49.128 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.129 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.129 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.129 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:33:49.129 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:33:49.129 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.129 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.129 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.129 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:33:49.129 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:33:49.129 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.129 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.129 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.129 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:33:49.129 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:33:49.129 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.130 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.130 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.130 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:33:49.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:33:49.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.130 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.130 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.130 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:33:49.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:33:49.130 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.130 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.130 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.130 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:33:49.130 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:33:49.131 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.131 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.131 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.131 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.131 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:49.131 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:49.131 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 403.77942709299998, 2023-12-06T18:33:49.131 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:33:49.131 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.131 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.131 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.131 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.131 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.131 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:33:49.131 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.132 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:49.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:49.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.132 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.132 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.132 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.132 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:33:49.132 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:33:49.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:33:49.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:33:49.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:33:49.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:33:49.133 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.133 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.133 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.133 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:33:49.133 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:33:49.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:33:49.134 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:33:49.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:33:49.134 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:33:49.134 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.134 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.134 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.134 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:33:49.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:33:49.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.135 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.135 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:33:49.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:33:49.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.135 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.135 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:33:49.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:33:49.135 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.135 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.135 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.135 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:33:49.135 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:33:49.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.136 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.136 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:33:49.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:33:49.136 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.136 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.136 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.136 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.136 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.136 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:49.136 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:49.136 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 403.775129585, 2023-12-06T18:33:49.136 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:33:49.136 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.136 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.137 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.137 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.137 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.137 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:33:49.137 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.137 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.137 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:49.137 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.137 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.137 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.137 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.137 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:49.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.138 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:33:49.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:33:49.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.138 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:33:49.138 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:33:49.138 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.138 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.138 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:33:49.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:33:49.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:33:49.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:33:49.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.139 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.139 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.139 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:33:49.139 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:33:49.139 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:33:49.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:33:49.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:33:49.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:33:49.140 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.140 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.140 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.140 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:33:49.140 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:33:49.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:33:49.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:33:49.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:33:49.141 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:33:49.141 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.141 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.141 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.141 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:33:49.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:33:49.142 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.142 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.142 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.142 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.142 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.142 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:49.142 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:49.142 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 403.77446191000001, 2023-12-06T18:33:49.142 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:33:49.142 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.142 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.142 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.142 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.142 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.143 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:33:49.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.143 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.143 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:49.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.143 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:49.143 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.143 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.143 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.143 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:33:49.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:33:49.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:33:49.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:33:49.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.144 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.144 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.144 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:33:49.144 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:33:49.144 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:33:49.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:33:49.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:33:49.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:33:49.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.145 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.145 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:33:49.145 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:33:49.145 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.145 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:33:49.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:33:49.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:33:49.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:33:49.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.146 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.146 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:33:49.146 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:33:49.146 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.146 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:33:49.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:33:49.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.147 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.147 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:33:49.147 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:33:49.147 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.147 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.147 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.147 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.147 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.147 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:49.147 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:49.147 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 403.773806482, 2023-12-06T18:33:49.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:33:49.148 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.148 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.148 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.148 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.148 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.148 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:33:49.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.148 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.148 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.148 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:49.148 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.148 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.148 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:49.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:33:49.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:33:49.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.149 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:33:49.149 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:33:49.149 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.149 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.149 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:33:49.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:33:49.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:33:49.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:33:49.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.150 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:33:49.150 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:33:49.150 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.150 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.150 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:33:49.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:33:49.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.151 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:33:49.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:33:49.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.151 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.151 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:33:49.151 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:33:49.151 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.151 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:33:49.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:33:49.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:33:49.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:33:49.152 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.152 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.152 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.152 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:33:49.152 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:33:49.152 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.152 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.152 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.153 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:49.153 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:49.153 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 403.73315193500002, 2023-12-06T18:33:49.153 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:33:49.153 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.153 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:49.153 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:49.153 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:49.153 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:49.153 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:33:49.153 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.153 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.153 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:49.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:49.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.154 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.154 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.154 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:33:49.154 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:33:49.154 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:33:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:33:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:33:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:33:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:33:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:33:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.155 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:33:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:33:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:33:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:33:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:33:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:33:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.156 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:33:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:33:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:33:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:33:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:33:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:33:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.157 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:33:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:33:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.158 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.158 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:49.159 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:33:49.160 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:49.157+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:49.260 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:49.257+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:49.305 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:49.305 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:49.305 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:49.305 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:49.314 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:33:49.361 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:49.357+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:49.450 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:49.450 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:49.450 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:49.450 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:49.450 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:49.453 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 7, 5, 4] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2023-12-06T18:33:49.453 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2023-12-06T18:33:49.453 INFO:tasks.thrashosds.thrasher:Adding osd 6 2023-12-06T18:33:49.453 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 6 2023-12-06T18:33:49.459 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:33:49.461 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:49.458+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:49.468 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2023-12-06T18:33:49.562 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:49.559+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:49.656 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:49.656 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:49.657 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:49.657 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:49.657 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.657 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:33:49.657 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:49.657 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 387.84040057999999, 2023-12-06T18:33:49.657 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:33:49.657 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:49.657 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:33:49.657 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:49.657 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.657 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:49.657 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:49.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.658 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:49.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:49.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:49.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.658 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:49.658 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:33:49.658 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:33:49.658 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.658 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:49.659 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:33:49.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:33:49.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:49.659 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:33:49.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:33:49.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:49.659 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:33:49.659 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:33:49.659 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.659 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.659 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:49.660 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:33:49.660 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:33:49.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.660 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:49.660 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:33:49.660 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:33:49.660 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:49.660 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:49.660 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:49.660 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:33:49.660 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:33:49.660 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.660 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.660 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.660 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:49.660 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:49.661 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:49.663 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:49.660+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:49.667 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:33:49.764 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:33:49.764 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:33:49.764 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:49.760+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:49.864 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:49.864 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:49.864 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:49.865 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:49.865 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:49.861+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:49.873 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:33:49.874 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:33:49.874 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:33:49.966 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:49.963+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:49.968 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:33:49.968 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:33:50.067 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:50.063+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:50.073 INFO:tasks.ceph.mon.a.smithi061.stderr:2023-12-06T18:33:50.068+0000 ff5f640 -1 mon.a@0(leader).osd e123 definitely_dead 0 2023-12-06T18:33:50.080 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:33:50.080 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:33:50.089 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:50.089 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:50.098 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:33:50.169 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:50.165+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:50.169 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:33:50.170 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:33:50.255 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:33:50.255 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:33:50.268 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:50.264+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:50.278 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:50.278 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:50.288 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:33:50.342 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2023-12-06T18:33:50.346 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:33:50.346 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:33:50.389 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:50.385+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:50.434 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:50.434 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:50.443 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:33:50.446 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:33:50.446 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:33:50.502 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:50.498+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:50.567 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:33:50.570 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:50.567+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:50.573 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:50.573 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:50.584 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:33:50.672 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:50.668+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:50.712 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:50.712 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:50.722 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:33:50.772 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:50.769+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:50.882 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:50.873+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:50.907 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:50.907 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:50.916 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:33:50.974 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:50.971+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:51.046 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:51.046 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:51.056 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:33:51.097 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:51.094+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:51.176 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:51.172+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:51.230 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:51.230 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:51.234 INFO:teuthology.orchestra.run.smithi061.stderr:marked in osd.6. 2023-12-06T18:33:51.239 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:33:51.246 INFO:tasks.thrashosds.thrasher:Added osd 6 2023-12-06T18:33:51.277 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":44,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":54,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:33:51.278 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:51.274+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:51.369 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:51.369 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:51.379 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:33:51.379 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:51.375+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:51.479 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:51.476+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:51.547 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:51.547 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:51.556 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:33:51.601 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:51.597+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:51.681 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:51.677+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:51.699 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:51.699 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:51.709 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:33:51.782 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:51.778+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:51.878 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:51.879 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:51.882 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:51.879+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:51.888 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:33:51.983 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:51.979+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:52.018 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:52.018 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:52.027 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:33:52.084 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:52.081+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:52.156 INFO:teuthology.orchestra.run.smithi152.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:52.156 INFO:teuthology.orchestra.run.smithi152.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:52.165 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2023-12-06T18:33:52.210 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:52.206+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:52.294 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:52.290+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:52.308 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:52.308 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:52.318 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2023-12-06T18:33:52.388 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:52.385+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:52.460 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2023-12-06T18:33:52.469 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is set 2023-12-06T18:33:52.504 INFO:teuthology.orchestra.run.smithi061.stdout:ERROR: (22) Invalid argument 2023-12-06T18:33:52.504 INFO:teuthology.orchestra.run.smithi061.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2023-12-06T18:33:52.505 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:52.501+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:52.514 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2023-12-06T18:33:52.600 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:52.596+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:52.690 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:52.686+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:52.742 INFO:teuthology.orchestra.run.smithi061.stdout:osd.0: {} 2023-12-06T18:33:52.742 INFO:teuthology.orchestra.run.smithi061.stderr:osd.0: osd_enable_op_tracker = '' 2023-12-06T18:33:52.768 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:52.768 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:52.768 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:52.768 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:52.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.769 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 110/109 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 23'75, data_included: [0~4096,65536~66064], data_size: 70160, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@23'75, size: 131600, copy_subset: [(0, 131600)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 131600, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:33:52.769 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:33:21.824095+0000", 2023-12-06T18:33:52.769 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 30.936740728, 2023-12-06T18:33:52.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.22522905200000001, 2023-12-06T18:33:52.769 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:52.769 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:33:52.769 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:52.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.769 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:52.769 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.824095+0000", 2023-12-06T18:33:52.769 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:52.769 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:52.769 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:52.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.824095+0000", 2023-12-06T18:33:52.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:52.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:52.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:52.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.824111+0000", 2023-12-06T18:33:52.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5596000000000001e-05 2023-12-06T18:33:52.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:52.770 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.770 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:52.770 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.866577+0000", 2023-12-06T18:33:52.770 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.042466773999999999 2023-12-06T18:33:52.770 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:52.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:52.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.871496+0000", 2023-12-06T18:33:52.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049188699999999997 2023-12-06T18:33:52.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:52.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:52.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.886281+0000", 2023-12-06T18:33:52.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.014784535 2023-12-06T18:33:52.771 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:52.771 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.771 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:52.771 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.903661+0000", 2023-12-06T18:33:52.771 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.017379950000000002 2023-12-06T18:33:52.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:52.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.772 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:52.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:21.908970+0000", 2023-12-06T18:33:52.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.005309322 2023-12-06T18:33:52.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:52.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.772 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:52.772 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.049324+0000", 2023-12-06T18:33:52.772 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.140354005 2023-12-06T18:33:52.772 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:52.772 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:52.772 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:52.772 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:52.772 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.773 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 110/109 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:33:52.773 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:33:22.129048+0000", 2023-12-06T18:33:52.773 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 30.631787542000001, 2023-12-06T18:33:52.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.029478353999999998, 2023-12-06T18:33:52.773 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:52.773 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:33:52.773 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:52.773 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.773 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:52.773 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.129048+0000", 2023-12-06T18:33:52.773 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:52.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:52.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:52.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.129048+0000", 2023-12-06T18:33:52.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:52.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:52.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:52.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.129058+0000", 2023-12-06T18:33:52.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 9.8630000000000003e-06 2023-12-06T18:33:52.774 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:52.774 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.774 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:52.774 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.133519+0000", 2023-12-06T18:33:52.774 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0044613229999999997 2023-12-06T18:33:52.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:52.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:52.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.133935+0000", 2023-12-06T18:33:52.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00041585999999999999 2023-12-06T18:33:52.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:52.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:52.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.134363+0000", 2023-12-06T18:33:52.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00042782200000000001 2023-12-06T18:33:52.775 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:52.775 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.775 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:52.775 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.136423+0000", 2023-12-06T18:33:52.775 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002059759 2023-12-06T18:33:52.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:52.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.776 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:52.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.136759+0000", 2023-12-06T18:33:52.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000335898 2023-12-06T18:33:52.776 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:52.776 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:52.776 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:52.776 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:33:22.158527+0000", 2023-12-06T18:33:52.776 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021767828999999999 2023-12-06T18:33:52.776 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:52.776 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:52.777 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:52.777 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:52.777 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:52.777 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:52.778 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2023-12-06T18:33:52.791 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:52.787+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:52.840 INFO:teuthology.orchestra.run.smithi061.stdout:osd.1: {} 2023-12-06T18:33:52.840 INFO:teuthology.orchestra.run.smithi061.stderr:osd.1: osd_enable_op_tracker = '' 2023-12-06T18:33:52.892 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:52.888+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:52.975 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:52.975 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:52.975 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:52.975 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:52.976 INFO:teuthology.orchestra.run.smithi061.stdout:osd.2: {} 2023-12-06T18:33:52.976 INFO:teuthology.orchestra.run.smithi061.stderr:osd.2: osd_enable_op_tracker = '' 2023-12-06T18:33:52.985 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2023-12-06T18:33:52.993 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:52.990+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:53.076 INFO:teuthology.orchestra.run.smithi061.stdout:osd.3: {} 2023-12-06T18:33:53.077 INFO:teuthology.orchestra.run.smithi061.stderr:osd.3: osd_enable_op_tracker = '' 2023-12-06T18:33:53.093 INFO:tasks.ceph.osd.7.smithi152.stderr:2023-12-06T18:33:53.090+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 7 (PID: 99441) UID: 0 2023-12-06T18:33:53.173 INFO:teuthology.orchestra.run.smithi061.stdout:osd.4: {} 2023-12-06T18:33:53.173 INFO:teuthology.orchestra.run.smithi061.stderr:osd.4: osd_enable_op_tracker = '' 2023-12-06T18:33:53.189 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:53.190 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:53.190 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:53.190 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:53.190 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:53.194 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:53.191+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:53.201 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2023-12-06T18:33:53.260 INFO:teuthology.orchestra.run.smithi061.stdout:osd.5: {} 2023-12-06T18:33:53.261 INFO:teuthology.orchestra.run.smithi061.stderr:osd.5: osd_enable_op_tracker = '' 2023-12-06T18:33:53.296 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:53.292+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:53.349 INFO:teuthology.orchestra.run.smithi061.stdout:osd.6: {} 2023-12-06T18:33:53.349 INFO:teuthology.orchestra.run.smithi061.stderr:osd.6: osd_enable_op_tracker = '' 2023-12-06T18:33:53.397 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:53.393+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:53.438 INFO:teuthology.orchestra.run.smithi061.stdout:osd.7: {} 2023-12-06T18:33:53.439 INFO:teuthology.orchestra.run.smithi061.stderr:osd.7: osd_enable_op_tracker = '' 2023-12-06T18:33:53.447 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:53.447 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:53.447 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:53.448 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:53.448 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.448 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2023-12-06T18:33:53.448 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:53.448 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 418.00629523200001, 2023-12-06T18:33:53.448 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.435408504, 2023-12-06T18:33:53.448 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.448 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.448 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.448 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.448 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.448 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 1 2023-12-06T18:33:53.448 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.448 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:53.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.349526+0000", 2023-12-06T18:33:53.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.449 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.449 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.350206+0000", 2023-12-06T18:33:53.449 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00067953299999999996 2023-12-06T18:33:53.449 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.449 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.450 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.450 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.355118+0000", 2023-12-06T18:33:53.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0049124520000000003 2023-12-06T18:33:53.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.450 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.450 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.359752+0000", 2023-12-06T18:33:53.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046337959999999999 2023-12-06T18:33:53.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.450 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.450 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.450 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.368866+0000", 2023-12-06T18:33:53.450 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0091139370000000008 2023-12-06T18:33:53.450 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.451 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.451 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.399811+0000", 2023-12-06T18:33:53.451 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030944625999999999 2023-12-06T18:33:53.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.451 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for peered", 2023-12-06T18:33:53.451 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:55.407159+0000", 2023-12-06T18:33:53.451 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0073479110000000004 2023-12-06T18:33:53.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.451 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.451 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.451 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.235560+0000", 2023-12-06T18:33:53.451 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.82840102999999998 2023-12-06T18:33:53.451 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.323413+0000", 2023-12-06T18:33:53.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.087852934999999993 2023-12-06T18:33:53.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.415815+0000", 2023-12-06T18:33:53.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.092402478999999996 2023-12-06T18:33:53.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.452 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.452 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.452 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.578536+0000", 2023-12-06T18:33:53.452 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.162720899 2023-12-06T18:33:53.452 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.453 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.453 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.453 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.754507+0000", 2023-12-06T18:33:53.453 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.175970981 2023-12-06T18:33:53.453 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.453 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.453 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.453 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.768312+0000", 2023-12-06T18:33:53.453 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.013805158999999999 2023-12-06T18:33:53.453 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.453 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.453 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.453 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:56.784935+0000", 2023-12-06T18:33:53.453 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016622766000000001 2023-12-06T18:33:53.453 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.453 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.453 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.454 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.454 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.454 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:30 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:53.454 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:53.454 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 415.897721993, 2023-12-06T18:33:53.454 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.191999118, 2023-12-06T18:33:53.454 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.454 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.454 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.454 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.454 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.454 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 30 2023-12-06T18:33:53.454 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.454 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458100+0000", 2023-12-06T18:33:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458112+0000", 2023-12-06T18:33:53.455 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1990999999999999e-05 2023-12-06T18:33:53.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.456 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458282+0000", 2023-12-06T18:33:53.456 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000170218 2023-12-06T18:33:53.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.456 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458438+0000", 2023-12-06T18:33:53.456 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000155964 2023-12-06T18:33:53.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.456 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.456 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.456 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458589+0000", 2023-12-06T18:33:53.456 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015157199999999999 2023-12-06T18:33:53.456 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.467552+0000", 2023-12-06T18:33:53.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0089629140000000006 2023-12-06T18:33:53.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.469186+0000", 2023-12-06T18:33:53.457 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016341680000000001 2023-12-06T18:33:53.457 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.457 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.457 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.457 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.471811+0000", 2023-12-06T18:33:53.458 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026241989999999998 2023-12-06T18:33:53.458 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.458 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.458 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.458 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.624877+0000", 2023-12-06T18:33:53.458 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15306636300000001 2023-12-06T18:33:53.458 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.458 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.458 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.458 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.647994+0000", 2023-12-06T18:33:53.458 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023117038999999999 2023-12-06T18:33:53.458 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.458 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.458 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.459 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.649318+0000", 2023-12-06T18:33:53.459 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001324389 2023-12-06T18:33:53.459 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.459 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.459 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.459 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.650099+0000", 2023-12-06T18:33:53.459 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00078030099999999995 2023-12-06T18:33:53.459 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.459 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.459 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.459 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.459 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.459 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:31 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:53.459 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:53.459 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 415.89697676100002, 2023-12-06T18:33:53.459 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25178181599999999, 2023-12-06T18:33:53.460 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.460 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.460 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.460 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.460 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.460 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 31 2023-12-06T18:33:53.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.460 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.460 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.460 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:53.460 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.460 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.460 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458845+0000", 2023-12-06T18:33:53.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458856+0000", 2023-12-06T18:33:53.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1277999999999999e-05 2023-12-06T18:33:53.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.461 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.458989+0000", 2023-12-06T18:33:53.461 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013234600000000001 2023-12-06T18:33:53.461 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.461 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.461 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459130+0000", 2023-12-06T18:33:53.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014160900000000001 2023-12-06T18:33:53.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459286+0000", 2023-12-06T18:33:53.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015613000000000001 2023-12-06T18:33:53.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.462 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.475774+0000", 2023-12-06T18:33:53.462 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.016487340999999999 2023-12-06T18:33:53.462 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.462 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.462 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.477457+0000", 2023-12-06T18:33:53.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00168385 2023-12-06T18:33:53.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.480113+0000", 2023-12-06T18:33:53.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026551349999999999 2023-12-06T18:33:53.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625013+0000", 2023-12-06T18:33:53.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.144900483 2023-12-06T18:33:53.463 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.463 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.463 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.463 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.706608+0000", 2023-12-06T18:33:53.463 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081595091999999994 2023-12-06T18:33:53.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.464 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.464 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.464 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.707657+0000", 2023-12-06T18:33:53.464 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010488439999999999 2023-12-06T18:33:53.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.464 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.464 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.464 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.710627+0000", 2023-12-06T18:33:53.464 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029697080000000002 2023-12-06T18:33:53.464 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.464 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.464 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.464 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.464 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.464 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:32 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:53.465 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:53.465 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 415.89629300600001, 2023-12-06T18:33:53.465 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25386585099999998, 2023-12-06T18:33:53.465 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.465 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.465 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.465 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.465 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.465 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 32 2023-12-06T18:33:53.465 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.465 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.465 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.465 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.465 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:53.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459529+0000", 2023-12-06T18:33:53.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.466 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.466 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.466 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.466 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.459540+0000", 2023-12-06T18:33:53.466 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.111e-05 2023-12-06T18:33:53.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463658+0000", 2023-12-06T18:33:53.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0041186599999999997 2023-12-06T18:33:53.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463809+0000", 2023-12-06T18:33:53.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000151061 2023-12-06T18:33:53.467 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.467 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.467 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.467 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.463956+0000", 2023-12-06T18:33:53.467 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014693200000000001 2023-12-06T18:33:53.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.485896+0000", 2023-12-06T18:33:53.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021939152999999999 2023-12-06T18:33:53.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.487576+0000", 2023-12-06T18:33:53.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016809170000000001 2023-12-06T18:33:53.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.468 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.468 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.468 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.490960+0000", 2023-12-06T18:33:53.468 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003383774 2023-12-06T18:33:53.468 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.625182+0000", 2023-12-06T18:33:53.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13422207899999999 2023-12-06T18:33:53.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.711397+0000", 2023-12-06T18:33:53.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086214347999999996 2023-12-06T18:33:53.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.469 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.469 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.469 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.712430+0000", 2023-12-06T18:33:53.469 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001033618 2023-12-06T18:33:53.469 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.470 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.470 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.713395+0000", 2023-12-06T18:33:53.470 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000964199 2023-12-06T18:33:53.470 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.470 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.470 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.470 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.470 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.470 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:33 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:53.470 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:53.471 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 415.89071975100001, 2023-12-06T18:33:53.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25095713200000003, 2023-12-06T18:33:53.471 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.471 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.471 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.471 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.471 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.471 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 33 2023-12-06T18:33:53.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.471 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.471 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.471 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.471 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:53.471 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.471 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465102+0000", 2023-12-06T18:33:53.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465113+0000", 2023-12-06T18:33:53.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1482e-05 2023-12-06T18:33:53.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.472 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.472 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.472 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465256+0000", 2023-12-06T18:33:53.472 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000142175 2023-12-06T18:33:53.472 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465398+0000", 2023-12-06T18:33:53.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014285799999999999 2023-12-06T18:33:53.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465547+0000", 2023-12-06T18:33:53.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000148257 2023-12-06T18:33:53.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.473 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.473 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.504047+0000", 2023-12-06T18:33:53.473 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.038500283000000003 2023-12-06T18:33:53.473 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.473 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.505743+0000", 2023-12-06T18:33:53.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001695544 2023-12-06T18:33:53.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.508403+0000", 2023-12-06T18:33:53.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002660745 2023-12-06T18:33:53.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.474 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.474 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627814+0000", 2023-12-06T18:33:53.474 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.119410407 2023-12-06T18:33:53.474 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.474 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.714152+0000", 2023-12-06T18:33:53.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.086338371999999997 2023-12-06T18:33:53.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.475 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.475 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.475 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.715181+0000", 2023-12-06T18:33:53.475 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010293349999999999 2023-12-06T18:33:53.475 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.476 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.476 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.476 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716059+0000", 2023-12-06T18:33:53.476 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087767400000000005 2023-12-06T18:33:53.476 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.476 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.476 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.476 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.477 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:53.477 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:53.477 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 415.890046111, 2023-12-06T18:33:53.477 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25294423300000002, 2023-12-06T18:33:53.477 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.477 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.477 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.477 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.477 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.477 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 34 2023-12-06T18:33:53.477 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.477 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.477 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.478 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.478 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:53.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.478 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.478 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465776+0000", 2023-12-06T18:33:53.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.478 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.478 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.478 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465786+0000", 2023-12-06T18:33:53.478 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0745e-05 2023-12-06T18:33:53.478 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.465917+0000", 2023-12-06T18:33:53.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000130677 2023-12-06T18:33:53.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466056+0000", 2023-12-06T18:33:53.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013925100000000001 2023-12-06T18:33:53.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.479 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.479 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.479 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466211+0000", 2023-12-06T18:33:53.479 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015498700000000001 2023-12-06T18:33:53.479 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.515964+0000", 2023-12-06T18:33:53.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.049752480000000002 2023-12-06T18:33:53.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.517649+0000", 2023-12-06T18:33:53.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001684863 2023-12-06T18:33:53.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.480 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.480 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.480 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.521104+0000", 2023-12-06T18:33:53.480 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034551030000000002 2023-12-06T18:33:53.480 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.627931+0000", 2023-12-06T18:33:53.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.106827149 2023-12-06T18:33:53.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.716818+0000", 2023-12-06T18:33:53.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.088887306999999999 2023-12-06T18:33:53.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.481 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.481 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.481 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.717832+0000", 2023-12-06T18:33:53.481 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010136500000000001 2023-12-06T18:33:53.481 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.482 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.482 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.718720+0000", 2023-12-06T18:33:53.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088802100000000001 2023-12-06T18:33:53.482 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.482 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.482 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.482 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.482 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.482 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:53.482 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:53.482 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 415.88938391200003, 2023-12-06T18:33:53.482 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.25503690200000001, 2023-12-06T18:33:53.482 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.482 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.483 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.483 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.483 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.483 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 35 2023-12-06T18:33:53.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.483 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:53.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.483 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.483 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.483 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.483 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466438+0000", 2023-12-06T18:33:53.483 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.484 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.484 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.484 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.484 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.466449+0000", 2023-12-06T18:33:53.484 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0766e-05 2023-12-06T18:33:53.484 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.484 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.484 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.484 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538533+0000", 2023-12-06T18:33:53.484 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.072084125999999998 2023-12-06T18:33:53.484 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.484 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.484 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.484 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538694+0000", 2023-12-06T18:33:53.484 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000161631 2023-12-06T18:33:53.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.485 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.485 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.538857+0000", 2023-12-06T18:33:53.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016247000000000001 2023-12-06T18:33:53.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.485 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.485 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.541037+0000", 2023-12-06T18:33:53.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0021802789999999998 2023-12-06T18:33:53.485 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.485 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.485 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.485 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.542771+0000", 2023-12-06T18:33:53.485 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017343580000000001 2023-12-06T18:33:53.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.546234+0000", 2023-12-06T18:33:53.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0034630669999999998 2023-12-06T18:33:53.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628041+0000", 2023-12-06T18:33:53.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.081806833999999995 2023-12-06T18:33:53.486 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.486 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.486 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.486 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.719478+0000", 2023-12-06T18:33:53.486 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.091437216000000002 2023-12-06T18:33:53.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.720492+0000", 2023-12-06T18:33:53.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010138549999999999 2023-12-06T18:33:53.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.487 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.487 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.487 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.721475+0000", 2023-12-06T18:33:53.487 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00098229999999999997 2023-12-06T18:33:53.487 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.487 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.487 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.487 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.488 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:53.488 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:53.488 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 415.81667447699999, 2023-12-06T18:33:53.488 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.184979859, 2023-12-06T18:33:53.488 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.488 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.488 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.488 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.488 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.488 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 36 2023-12-06T18:33:53.488 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.488 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.488 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.488 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:53.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539147+0000", 2023-12-06T18:33:53.489 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.489 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.489 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.489 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.489 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539169+0000", 2023-12-06T18:33:53.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1917999999999999e-05 2023-12-06T18:33:53.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.490 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.490 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539308+0000", 2023-12-06T18:33:53.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013934200000000001 2023-12-06T18:33:53.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.490 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.490 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539449+0000", 2023-12-06T18:33:53.490 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014079199999999999 2023-12-06T18:33:53.490 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.490 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.490 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539597+0000", 2023-12-06T18:33:53.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014782 2023-12-06T18:33:53.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.552175+0000", 2023-12-06T18:33:53.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012578126 2023-12-06T18:33:53.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.491 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.553903+0000", 2023-12-06T18:33:53.491 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001728212 2023-12-06T18:33:53.491 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.491 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.491 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.556779+0000", 2023-12-06T18:33:53.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028755220000000001 2023-12-06T18:33:53.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628151+0000", 2023-12-06T18:33:53.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.071372347000000003 2023-12-06T18:33:53.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.722237+0000", 2023-12-06T18:33:53.492 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.094085428999999998 2023-12-06T18:33:53.492 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.492 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.492 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.492 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.723252+0000", 2023-12-06T18:33:53.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001015356 2023-12-06T18:33:53.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.493 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.493 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724127+0000", 2023-12-06T18:33:53.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00087499500000000005 2023-12-06T18:33:53.493 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.493 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.493 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.493 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.493 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.493 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2023-12-06T18:33:53.493 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:53.493 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 415.815992768, 2023-12-06T18:33:53.493 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.186985014, 2023-12-06T18:33:53.493 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.494 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.494 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.494 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.494 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.494 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 37 2023-12-06T18:33:53.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.494 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:53.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.494 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.494 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.494 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.494 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539829+0000", 2023-12-06T18:33:53.494 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539840+0000", 2023-12-06T18:33:53.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1212999999999999e-05 2023-12-06T18:33:53.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.539979+0000", 2023-12-06T18:33:53.495 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138796 2023-12-06T18:33:53.495 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.495 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.495 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.495 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540118+0000", 2023-12-06T18:33:53.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000138959 2023-12-06T18:33:53.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.496 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.540278+0000", 2023-12-06T18:33:53.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016009600000000001 2023-12-06T18:33:53.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.496 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.561540+0000", 2023-12-06T18:33:53.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021261604 2023-12-06T18:33:53.496 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.496 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.496 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.496 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.563870+0000", 2023-12-06T18:33:53.496 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023305309999999998 2023-12-06T18:33:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.566716+0000", 2023-12-06T18:33:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0028463109999999998 2023-12-06T18:33:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.628270+0000", 2023-12-06T18:33:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.061553692 2023-12-06T18:33:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.724896+0000", 2023-12-06T18:33:53.497 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.096625956999999998 2023-12-06T18:33:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.725925+0000", 2023-12-06T18:33:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001029416 2023-12-06T18:33:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:57.726814+0000", 2023-12-06T18:33:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00088843899999999998 2023-12-06T18:33:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.498 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:82 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 414.73718867999997, 2023-12-06T18:33:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.12787030799999999, 2023-12-06T18:33:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 82 2023-12-06T18:33:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.499 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:53.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618633+0000", 2023-12-06T18:33:53.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618654+0000", 2023-12-06T18:33:53.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.1007999999999999e-05 2023-12-06T18:33:53.500 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.500 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.500 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.500 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.618922+0000", 2023-12-06T18:33:53.500 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026790100000000002 2023-12-06T18:33:53.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.501 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.501 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619188+0000", 2023-12-06T18:33:53.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00026588300000000002 2023-12-06T18:33:53.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.501 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.501 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.619446+0000", 2023-12-06T18:33:53.501 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00025831900000000002 2023-12-06T18:33:53.501 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.501 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.501 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.501 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621982+0000", 2023-12-06T18:33:53.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0025360830000000002 2023-12-06T18:33:53.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.623687+0000", 2023-12-06T18:33:53.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017043659999999999 2023-12-06T18:33:53.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.626410+0000", 2023-12-06T18:33:53.502 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002723474 2023-12-06T18:33:53.502 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.502 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.502 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.502 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728747+0000", 2023-12-06T18:33:53.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.10233684699999999 2023-12-06T18:33:53.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.744678+0000", 2023-12-06T18:33:53.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015931059000000001 2023-12-06T18:33:53.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.745730+0000", 2023-12-06T18:33:53.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010515909999999999 2023-12-06T18:33:53.503 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.503 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.503 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.503 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.746503+0000", 2023-12-06T18:33:53.503 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00077377700000000002 2023-12-06T18:33:53.504 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.504 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.504 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.504 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.504 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.504 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:83 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:53.504 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:53.504 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 414.73577210600001, 2023-12-06T18:33:53.504 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13688794800000001, 2023-12-06T18:33:53.504 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.504 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.504 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.504 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.504 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.504 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 83 2023-12-06T18:33:53.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.505 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:53.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.505 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.505 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.505 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620050+0000", 2023-12-06T18:33:53.505 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.505 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620066+0000", 2023-12-06T18:33:53.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6646999999999998e-05 2023-12-06T18:33:53.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.506 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.506 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620278+0000", 2023-12-06T18:33:53.506 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021203199999999999 2023-12-06T18:33:53.506 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.506 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620525+0000", 2023-12-06T18:33:53.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00024645400000000001 2023-12-06T18:33:53.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.620767+0000", 2023-12-06T18:33:53.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000242797 2023-12-06T18:33:53.507 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.507 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.507 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.507 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.630811+0000", 2023-12-06T18:33:53.507 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.010043704000000001 2023-12-06T18:33:53.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.632507+0000", 2023-12-06T18:33:53.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00169535 2023-12-06T18:33:53.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.508 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.635302+0000", 2023-12-06T18:33:53.508 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027956869999999998 2023-12-06T18:33:53.508 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.508 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.508 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.728865+0000", 2023-12-06T18:33:53.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.093563188000000005 2023-12-06T18:33:53.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.755116+0000", 2023-12-06T18:33:53.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026250241000000001 2023-12-06T18:33:53.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756176+0000", 2023-12-06T18:33:53.509 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010605530000000001 2023-12-06T18:33:53.509 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.509 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.509 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.509 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.756938+0000", 2023-12-06T18:33:53.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00076129500000000005 2023-12-06T18:33:53.510 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.510 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.510 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.510 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.510 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.510 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:84 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:53.510 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:53.510 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 414.73468698099998, 2023-12-06T18:33:53.510 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.13833368700000001, 2023-12-06T18:33:53.510 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.510 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.510 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.510 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.511 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.511 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 84 2023-12-06T18:33:53.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.511 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:53.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.511 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.511 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621135+0000", 2023-12-06T18:33:53.511 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.511 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.511 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.621151+0000", 2023-12-06T18:33:53.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.6283e-05 2023-12-06T18:33:53.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.512 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.647966+0000", 2023-12-06T18:33:53.512 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026815249999999999 2023-12-06T18:33:53.512 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.512 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.512 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652644+0000", 2023-12-06T18:33:53.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0046775310000000004 2023-12-06T18:33:53.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.652922+0000", 2023-12-06T18:33:53.513 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027870400000000001 2023-12-06T18:33:53.513 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.513 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.513 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.513 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.657015+0000", 2023-12-06T18:33:53.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040923929999999997 2023-12-06T18:33:53.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.659259+0000", 2023-12-06T18:33:53.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002243798 2023-12-06T18:33:53.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.514 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.662816+0000", 2023-12-06T18:33:53.514 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003557026 2023-12-06T18:33:53.514 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.514 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.514 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.732882+0000", 2023-12-06T18:33:53.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070066613 2023-12-06T18:33:53.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.757706+0000", 2023-12-06T18:33:53.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.024823224000000001 2023-12-06T18:33:53.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.515 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.758727+0000", 2023-12-06T18:33:53.515 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001021903 2023-12-06T18:33:53.515 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.515 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.515 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.516 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.759468+0000", 2023-12-06T18:33:53.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00074096199999999996 2023-12-06T18:33:53.516 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.516 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.516 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.516 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.516 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.516 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:87 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2023-12-06T18:33:53.516 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:53.516 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 414.69953787899999, 2023-12-06T18:33:53.516 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.110834558, 2023-12-06T18:33:53.516 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.517 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.517 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.517 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.517 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.517 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 87 2023-12-06T18:33:53.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.517 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:53.517 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.517 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.517 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.517 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.517 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656284+0000", 2023-12-06T18:33:53.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.656295+0000", 2023-12-06T18:33:53.518 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1063999999999999e-05 2023-12-06T18:33:53.518 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.518 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.518 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.518 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693454+0000", 2023-12-06T18:33:53.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037159338 2023-12-06T18:33:53.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693623+0000", 2023-12-06T18:33:53.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016914 2023-12-06T18:33:53.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.519 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.519 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.693784+0000", 2023-12-06T18:33:53.519 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000160407 2023-12-06T18:33:53.519 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.519 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.696573+0000", 2023-12-06T18:33:53.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027890369999999999 2023-12-06T18:33:53.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.698284+0000", 2023-12-06T18:33:53.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017114529999999999 2023-12-06T18:33:53.520 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.520 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.520 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.520 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.700977+0000", 2023-12-06T18:33:53.520 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026925759999999999 2023-12-06T18:33:53.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.738924+0000", 2023-12-06T18:33:53.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.037946885 2023-12-06T18:33:53.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.765261+0000", 2023-12-06T18:33:53.521 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.026337312000000002 2023-12-06T18:33:53.521 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.521 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.521 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.521 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.766307+0000", 2023-12-06T18:33:53.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010459480000000001 2023-12-06T18:33:53.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.522 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.522 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:26:58.767118+0000", 2023-12-06T18:33:53.522 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00081139799999999996 2023-12-06T18:33:53.522 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.522 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.522 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.522 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.522 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.522 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:104 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 0~512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:53.522 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:53.522 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 408.14980735400002, 2023-12-06T18:33:53.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.115775532, 2023-12-06T18:33:53.523 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.523 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.523 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.523 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.523 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.523 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 104 2023-12-06T18:33:53.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.523 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.523 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.523 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.523 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:53.523 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.523 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206014+0000", 2023-12-06T18:33:53.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.524 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.524 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206030+0000", 2023-12-06T18:33:53.524 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5702999999999999e-05 2023-12-06T18:33:53.524 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.524 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206247+0000", 2023-12-06T18:33:53.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00021665399999999999 2023-12-06T18:33:53.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206406+0000", 2023-12-06T18:33:53.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000159538 2023-12-06T18:33:53.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.525 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.525 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206559+0000", 2023-12-06T18:33:53.525 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015273700000000001 2023-12-06T18:33:53.525 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.525 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.214136+0000", 2023-12-06T18:33:53.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0075767480000000003 2023-12-06T18:33:53.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.216430+0000", 2023-12-06T18:33:53.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002294099 2023-12-06T18:33:53.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.526 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.526 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.526 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.219555+0000", 2023-12-06T18:33:53.526 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0031255459999999999 2023-12-06T18:33:53.526 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.318747+0000", 2023-12-06T18:33:53.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099191649000000007 2023-12-06T18:33:53.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319972+0000", 2023-12-06T18:33:53.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012247130000000001 2023-12-06T18:33:53.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.527 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.527 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.320968+0000", 2023-12-06T18:33:53.527 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00099636600000000005 2023-12-06T18:33:53.527 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.527 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.528 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.528 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321790+0000", 2023-12-06T18:33:53.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00082177900000000002 2023-12-06T18:33:53.528 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.528 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.528 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.528 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.528 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.528 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:53.528 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:53.528 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 408.14900174600001, 2023-12-06T18:33:53.528 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.116816135, 2023-12-06T18:33:53.528 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.528 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.529 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.529 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.529 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.529 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 105 2023-12-06T18:33:53.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.529 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:53.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.529 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.529 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.529 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206820+0000", 2023-12-06T18:33:53.529 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.529 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206831+0000", 2023-12-06T18:33:53.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1097000000000001e-05 2023-12-06T18:33:53.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.206964+0000", 2023-12-06T18:33:53.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013297800000000001 2023-12-06T18:33:53.530 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.530 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.530 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.530 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207106+0000", 2023-12-06T18:33:53.530 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000141588 2023-12-06T18:33:53.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207264+0000", 2023-12-06T18:33:53.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015794800000000001 2023-12-06T18:33:53.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.225384+0000", 2023-12-06T18:33:53.531 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018120022999999999 2023-12-06T18:33:53.531 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.531 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.531 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.531 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.227116+0000", 2023-12-06T18:33:53.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001732333 2023-12-06T18:33:53.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.229907+0000", 2023-12-06T18:33:53.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027911289999999998 2023-12-06T18:33:53.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.319618+0000", 2023-12-06T18:33:53.532 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.089710581999999997 2023-12-06T18:33:53.532 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.532 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.532 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.532 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.321936+0000", 2023-12-06T18:33:53.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0023182789999999999 2023-12-06T18:33:53.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.322868+0000", 2023-12-06T18:33:53.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000932324 2023-12-06T18:33:53.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.533 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.533 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.533 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.323636+0000", 2023-12-06T18:33:53.533 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000767854 2023-12-06T18:33:53.533 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.533 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.533 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.533 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.534 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:53.534 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:53.534 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 408.14831093700002, 2023-12-06T18:33:53.534 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15170763300000001, 2023-12-06T18:33:53.534 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.534 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.534 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.534 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.534 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.534 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 106 2023-12-06T18:33:53.534 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.534 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.534 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.534 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:53.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207511+0000", 2023-12-06T18:33:53.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.535 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.207522+0000", 2023-12-06T18:33:53.535 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0958000000000001e-05 2023-12-06T18:33:53.535 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.535 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.535 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211267+0000", 2023-12-06T18:33:53.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0037457329999999998 2023-12-06T18:33:53.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211430+0000", 2023-12-06T18:33:53.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00016213499999999999 2023-12-06T18:33:53.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.536 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.536 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211577+0000", 2023-12-06T18:33:53.536 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014779100000000001 2023-12-06T18:33:53.536 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.536 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.234661+0000", 2023-12-06T18:33:53.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.023083355 2023-12-06T18:33:53.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.236378+0000", 2023-12-06T18:33:53.537 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0017172999999999999 2023-12-06T18:33:53.537 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.537 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.537 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.537 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.239138+0000", 2023-12-06T18:33:53.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0027599069999999998 2023-12-06T18:33:53.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.538 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.338174+0000", 2023-12-06T18:33:53.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.099035896999999998 2023-12-06T18:33:53.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.538 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.353716+0000", 2023-12-06T18:33:53.538 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.015542245999999999 2023-12-06T18:33:53.538 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.538 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.538 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.538 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.355667+0000", 2023-12-06T18:33:53.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001951365 2023-12-06T18:33:53.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.539 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.539 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359218+0000", 2023-12-06T18:33:53.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0035509460000000001 2023-12-06T18:33:53.539 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.539 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.539 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.539 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.539 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.539 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:53.539 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:53.539 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 408.14401342899998, 2023-12-06T18:33:53.539 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15034866999999999, 2023-12-06T18:33:53.540 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.540 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.540 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.540 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.540 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.540 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 107 2023-12-06T18:33:53.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.540 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.540 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:53.540 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.540 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.540 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.540 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211808+0000", 2023-12-06T18:33:53.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211819+0000", 2023-12-06T18:33:53.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1121000000000001e-05 2023-12-06T18:33:53.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.541 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.541 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.211955+0000", 2023-12-06T18:33:53.541 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000135322 2023-12-06T18:33:53.541 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.541 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212096+0000", 2023-12-06T18:33:53.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000140932 2023-12-06T18:33:53.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212248+0000", 2023-12-06T18:33:53.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015234800000000001 2023-12-06T18:33:53.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.542 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.542 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.542 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.257417+0000", 2023-12-06T18:33:53.542 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045168551000000001 2023-12-06T18:33:53.542 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.259143+0000", 2023-12-06T18:33:53.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001726163 2023-12-06T18:33:53.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.261897+0000", 2023-12-06T18:33:53.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002754617 2023-12-06T18:33:53.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.543 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.543 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.543 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.339713+0000", 2023-12-06T18:33:53.543 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.077816138000000007 2023-12-06T18:33:53.543 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.359460+0000", 2023-12-06T18:33:53.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.019746738999999999 2023-12-06T18:33:53.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.361122+0000", 2023-12-06T18:33:53.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001661725 2023-12-06T18:33:53.544 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.544 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.544 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.544 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362157+0000", 2023-12-06T18:33:53.544 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0010350139999999999 2023-12-06T18:33:53.544 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.545 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.545 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.545 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.545 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:53.545 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:53.545 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 408.14334575399999, 2023-12-06T18:33:53.545 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.15282025099999999, 2023-12-06T18:33:53.545 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.545 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.545 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.545 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.545 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.545 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 108 2023-12-06T18:33:53.545 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.546 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:53.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212476+0000", 2023-12-06T18:33:53.546 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.546 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.546 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.546 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.546 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212487+0000", 2023-12-06T18:33:53.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.1105e-05 2023-12-06T18:33:53.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212619+0000", 2023-12-06T18:33:53.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013198300000000001 2023-12-06T18:33:53.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212759+0000", 2023-12-06T18:33:53.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013967499999999999 2023-12-06T18:33:53.547 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.547 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.547 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.547 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.212905+0000", 2023-12-06T18:33:53.547 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014657 2023-12-06T18:33:53.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.266266+0000", 2023-12-06T18:33:53.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.053360907999999999 2023-12-06T18:33:53.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.267940+0000", 2023-12-06T18:33:53.548 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016738809999999999 2023-12-06T18:33:53.548 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.548 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.548 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.548 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.270617+0000", 2023-12-06T18:33:53.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026764710000000001 2023-12-06T18:33:53.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.341235+0000", 2023-12-06T18:33:53.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.070618480999999997 2023-12-06T18:33:53.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.549 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.362391+0000", 2023-12-06T18:33:53.549 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021155617000000002 2023-12-06T18:33:53.549 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.549 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.549 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.364006+0000", 2023-12-06T18:33:53.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016156720000000001 2023-12-06T18:33:53.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.550 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.550 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.365296+0000", 2023-12-06T18:33:53.550 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0012898880000000001 2023-12-06T18:33:53.550 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.550 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.550 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.550 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.550 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.550 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:53.550 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:53.551 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 408.14269032599998, 2023-12-06T18:33:53.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.16021534300000001, 2023-12-06T18:33:53.551 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.551 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.551 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.551 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.551 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.551 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 109 2023-12-06T18:33:53.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.551 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.551 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.551 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:53.551 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.551 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.551 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213131+0000", 2023-12-06T18:33:53.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.213142+0000", 2023-12-06T18:33:53.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0818999999999999e-05 2023-12-06T18:33:53.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.552 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.552 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253168+0000", 2023-12-06T18:33:53.552 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.040026211999999999 2023-12-06T18:33:53.552 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.552 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253333+0000", 2023-12-06T18:33:53.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000164186 2023-12-06T18:33:53.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253498+0000", 2023-12-06T18:33:53.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000165339 2023-12-06T18:33:53.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.274837+0000", 2023-12-06T18:33:53.553 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.021338923999999999 2023-12-06T18:33:53.553 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.553 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.553 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.553 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.276501+0000", 2023-12-06T18:33:53.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001664614 2023-12-06T18:33:53.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.279148+0000", 2023-12-06T18:33:53.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026462209999999998 2023-12-06T18:33:53.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.554 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.342750+0000", 2023-12-06T18:33:53.554 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.063602487999999999 2023-12-06T18:33:53.554 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.554 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.554 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.370321+0000", 2023-12-06T18:33:53.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.027570395000000001 2023-12-06T18:33:53.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.371998+0000", 2023-12-06T18:33:53.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016773039999999999 2023-12-06T18:33:53.555 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.555 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.555 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.555 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373347+0000", 2023-12-06T18:33:53.555 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0013488409999999999 2023-12-06T18:33:53.555 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.555 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.555 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.556 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4160.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2023-12-06T18:33:53.556 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:53.556 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 408.102035779, 2023-12-06T18:33:53.556 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.123668766, 2023-12-06T18:33:53.556 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:53.556 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:53.556 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:53.556 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4160", 2023-12-06T18:33:53.556 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/2601653201", 2023-12-06T18:33:53.556 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 110 2023-12-06T18:33:53.556 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.556 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:53.556 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:53.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:53.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:53.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253786+0000", 2023-12-06T18:33:53.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:53.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:53.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253801+0000", 2023-12-06T18:33:53.557 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5291e-05 2023-12-06T18:33:53.557 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.557 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.557 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:53.557 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.253941+0000", 2023-12-06T18:33:53.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00013991900000000001 2023-12-06T18:33:53.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:53.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254083+0000", 2023-12-06T18:33:53.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00014188100000000001 2023-12-06T18:33:53.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:53.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.254239+0000", 2023-12-06T18:33:53.558 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00015639199999999999 2023-12-06T18:33:53.558 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.558 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.558 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:53.558 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.299728+0000", 2023-12-06T18:33:53.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.045488905000000003 2023-12-06T18:33:53.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:53.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.301487+0000", 2023-12-06T18:33:53.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001758833 2023-12-06T18:33:53.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.559 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "waiting for subops from 5", 2023-12-06T18:33:53.559 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.305214+0000", 2023-12-06T18:33:53.559 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.003726544 2023-12-06T18:33:53.559 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.559 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "sub_op_commit_rec", 2023-12-06T18:33:53.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.344276+0000", 2023-12-06T18:33:53.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.039062586000000003 2023-12-06T18:33:53.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "op_commit", 2023-12-06T18:33:53.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.373579+0000", 2023-12-06T18:33:53.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.02930257 2023-12-06T18:33:53.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.560 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "commit_sent", 2023-12-06T18:33:53.560 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.375003+0000", 2023-12-06T18:33:53.560 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.001424273 2023-12-06T18:33:53.560 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:53.560 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:53.561 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:53.561 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:05.377455+0000", 2023-12-06T18:33:53.561 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.002451572 2023-12-06T18:33:53.561 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.561 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.561 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.561 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:53.561 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:53.561 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:53.563 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2023-12-06T18:33:53.563 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:53.559+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:53.663 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:53.660+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:53.756 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:53.757 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:53.757 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:53.757 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:53.764 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:53.761+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:53.766 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2023-12-06T18:33:53.865 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:53.862+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:53.910 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:53.910 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:53.910 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:53.910 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:53.910 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:53.919 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2023-12-06T18:33:53.966 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:33:53.963+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:33:54.059 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:54.059 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:54.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:54.059 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:54.059 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.059 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "MOSDPGPush(1.0 26/25 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 23'76, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@23'76, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2023-12-06T18:33:54.059 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:54.059 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 392.24296676400002, 2023-12-06T18:33:54.059 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.030633288000000002, 2023-12-06T18:33:54.059 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:54.060 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "started", 2023-12-06T18:33:54.060 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:54.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:54.060 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:54.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:54.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:54.060 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810820+0000", 2023-12-06T18:33:54.060 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:54.060 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.060 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.060 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:54.060 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.810830+0000", 2023-12-06T18:33:54.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.0292000000000001e-05 2023-12-06T18:33:54.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.061 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:54.061 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.816726+0000", 2023-12-06T18:33:54.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0058962349999999997 2023-12-06T18:33:54.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.061 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:54.061 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.817106+0000", 2023-12-06T18:33:54.061 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000379706 2023-12-06T18:33:54.061 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.061 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.061 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:54.061 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.820071+0000", 2023-12-06T18:33:54.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0029652459999999999 2023-12-06T18:33:54.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:54.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.822696+0000", 2023-12-06T18:33:54.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0026251260000000002 2023-12-06T18:33:54.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "started", 2023-12-06T18:33:54.062 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.823172+0000", 2023-12-06T18:33:54.062 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00047522299999999999 2023-12-06T18:33:54.062 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.062 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.062 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:54.063 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:27:21.841453+0000", 2023-12-06T18:33:54.063 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.018281459999999999 2023-12-06T18:33:54.063 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:54.063 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:54.063 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:54.063 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:54.063 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:54.063 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:54.067 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:33:54.063+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:33:54.069 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2023-12-06T18:33:54.168 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:54.165+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:54.254 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:54.255 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:54.255 INFO:teuthology.orchestra.run.smithi061.stdout: "num_ops": 0 2023-12-06T18:33:54.255 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:54.265 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2023-12-06T18:33:54.268 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:54.264+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:54.290 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2023-12-06T18:33:54.369 INFO:tasks.ceph.osd.6.smithi152.stderr:2023-12-06T18:33:54.366+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 6 (PID: 87109) UID: 0 2023-12-06T18:33:54.409 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:54.409 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [], 2023-12-06T18:33:54.409 INFO:teuthology.orchestra.run.smithi061.stdout: "complaint_time": 30, 2023-12-06T18:33:54.409 INFO:teuthology.orchestra.run.smithi061.stdout: "num_blocked_ops": 0 2023-12-06T18:33:54.409 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:54.419 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2023-12-06T18:33:54.470 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:54.467+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:54.484 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2023-12-06T18:33:54.571 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:33:54.568+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:33:54.572 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:33:54.572 INFO:teuthology.orchestra.run.smithi061.stdout: "size": 20, 2023-12-06T18:33:54.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 600, 2023-12-06T18:33:54.572 INFO:teuthology.orchestra.run.smithi061.stdout: "ops": [ 2023-12-06T18:33:54.572 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.572 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:5 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:33:54.572 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:54.572 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 273.47007183800002, 2023-12-06T18:33:54.572 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.012817100999999999, 2023-12-06T18:33:54.572 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:54.573 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:54.573 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:54.573 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:54.573 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:54.573 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 5 2023-12-06T18:33:54.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.573 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:54.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.573 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:54.573 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:54.573 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:54.573 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.573 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:54.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092744+0000", 2023-12-06T18:33:54.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:54.574 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:54.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.092767+0000", 2023-12-06T18:33:54.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 2.3654000000000001e-05 2023-12-06T18:33:54.574 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.574 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.574 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:54.574 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093050+0000", 2023-12-06T18:33:54.574 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00028257299999999999 2023-12-06T18:33:54.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.575 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:54.575 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.093324+0000", 2023-12-06T18:33:54.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00027397700000000001 2023-12-06T18:33:54.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.575 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:54.575 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.097338+0000", 2023-12-06T18:33:54.575 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0040141150000000004 2023-12-06T18:33:54.575 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.575 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.575 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:54.576 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.103915+0000", 2023-12-06T18:33:54.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0065773150000000002 2023-12-06T18:33:54.576 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.576 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.576 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:54.576 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:29:21.105561+0000", 2023-12-06T18:33:54.576 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016454670000000001 2023-12-06T18:33:54.576 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:54.576 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:54.577 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:54.577 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.577 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.577 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:7 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2023-12-06T18:33:54.577 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:54.577 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 213.451814536, 2023-12-06T18:33:54.577 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0071375409999999999, 2023-12-06T18:33:54.577 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:54.577 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:54.578 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:54.578 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:54.578 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:54.578 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 7 2023-12-06T18:33:54.578 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.578 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:54.578 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.578 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:54.579 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:54.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:54.579 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.579 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.579 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:54.579 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111001+0000", 2023-12-06T18:33:54.579 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:54.580 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.580 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:54.580 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111016+0000", 2023-12-06T18:33:54.580 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.5438e-05 2023-12-06T18:33:54.580 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.580 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.581 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:54.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111222+0000", 2023-12-06T18:33:54.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020550000000000001 2023-12-06T18:33:54.581 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.581 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.581 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:54.581 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111426+0000", 2023-12-06T18:33:54.581 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020417800000000001 2023-12-06T18:33:54.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.582 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:54.582 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.111600+0000", 2023-12-06T18:33:54.582 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00017377500000000001 2023-12-06T18:33:54.582 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.582 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.582 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:54.582 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.116477+0000", 2023-12-06T18:33:54.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0048769659999999999 2023-12-06T18:33:54.583 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.583 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.583 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:54.583 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:30:21.118138+0000", 2023-12-06T18:33:54.583 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0016616840000000001 2023-12-06T18:33:54.583 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:54.583 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:54.584 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:54.584 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.584 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.584 INFO:teuthology.orchestra.run.smithi061.stdout: "description": "osd_op(client.4201.0:11 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71)", 2023-12-06T18:33:54.584 INFO:teuthology.orchestra.run.smithi061.stdout: "initiated_at": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:54.584 INFO:teuthology.orchestra.run.smithi061.stdout: "age": 93.427393418999998, 2023-12-06T18:33:54.584 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0086075449999999994, 2023-12-06T18:33:54.585 INFO:teuthology.orchestra.run.smithi061.stdout: "type_data": { 2023-12-06T18:33:54.585 INFO:teuthology.orchestra.run.smithi061.stdout: "flag_point": "reached pg", 2023-12-06T18:33:54.585 INFO:teuthology.orchestra.run.smithi061.stdout: "client_info": { 2023-12-06T18:33:54.585 INFO:teuthology.orchestra.run.smithi061.stdout: "client": "client.4201", 2023-12-06T18:33:54.585 INFO:teuthology.orchestra.run.smithi061.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:54.585 INFO:teuthology.orchestra.run.smithi061.stdout: "tid": 11 2023-12-06T18:33:54.585 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.586 INFO:teuthology.orchestra.run.smithi061.stdout: "events": [ 2023-12-06T18:33:54.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.586 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "initiated", 2023-12-06T18:33:54.586 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:54.586 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:54.586 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.586 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "header_read", 2023-12-06T18:33:54.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135422+0000", 2023-12-06T18:33:54.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0 2023-12-06T18:33:54.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "throttled", 2023-12-06T18:33:54.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135441+0000", 2023-12-06T18:33:54.587 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 1.8695999999999999e-05 2023-12-06T18:33:54.587 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.587 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.587 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "all_read", 2023-12-06T18:33:54.587 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135650+0000", 2023-12-06T18:33:54.588 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.00020883699999999999 2023-12-06T18:33:54.588 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.588 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.588 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "dispatched", 2023-12-06T18:33:54.588 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.135863+0000", 2023-12-06T18:33:54.588 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000213931 2023-12-06T18:33:54.588 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.588 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "queued_for_pg", 2023-12-06T18:33:54.589 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.136054+0000", 2023-12-06T18:33:54.589 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.000190416 2023-12-06T18:33:54.589 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.589 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "reached_pg", 2023-12-06T18:33:54.589 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.142056+0000", 2023-12-06T18:33:54.589 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0060019330000000001 2023-12-06T18:33:54.589 INFO:teuthology.orchestra.run.smithi061.stdout: }, 2023-12-06T18:33:54.589 INFO:teuthology.orchestra.run.smithi061.stdout: { 2023-12-06T18:33:54.589 INFO:teuthology.orchestra.run.smithi061.stdout: "event": "done", 2023-12-06T18:33:54.589 INFO:teuthology.orchestra.run.smithi061.stdout: "time": "2023-12-06T18:32:21.144030+0000", 2023-12-06T18:33:54.589 INFO:teuthology.orchestra.run.smithi061.stdout: "duration": 0.0019737320000000002 2023-12-06T18:33:54.589 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:54.589 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:54.590 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:54.590 INFO:teuthology.orchestra.run.smithi061.stdout: } 2023-12-06T18:33:54.590 INFO:teuthology.orchestra.run.smithi061.stdout: ] 2023-12-06T18:33:54.590 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:33:54.591 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2023-12-06T18:33:54.672 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:33:54.668+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:33:54.722 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:54.722 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:54.722 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:54.722 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:54.731 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2023-12-06T18:33:54.775 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:33:54.772+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:33:54.861 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:54.861 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:54.861 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:54.862 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:54.862 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:54.871 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2023-12-06T18:33:54.873 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:54.870+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:54.974 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:33:54.970+0000 b47e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:33:55.042 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:55.042 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:55.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:55.042 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:33:55.042 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.042 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:4 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2023-12-06T18:33:55.042 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:33:55.042 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 333.548590434, 2023-12-06T18:33:55.042 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.086011331999999996, 2023-12-06T18:33:55.042 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.043 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:55.043 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:55.043 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:55.043 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:55.043 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 4 2023-12-06T18:33:55.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.043 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:33:55.043 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.043 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.043 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.043 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.043 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488538+0000", 2023-12-06T18:33:55.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.488760+0000", 2023-12-06T18:33:55.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000222411 2023-12-06T18:33:55.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.493426+0000", 2023-12-06T18:33:55.044 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046662129999999998 2023-12-06T18:33:55.044 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.044 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.044 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.044 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.497778+0000", 2023-12-06T18:33:55.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0043514010000000004 2023-12-06T18:33:55.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.506787+0000", 2023-12-06T18:33:55.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0090093840000000005 2023-12-06T18:33:55.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.519734+0000", 2023-12-06T18:33:55.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012946338999999999 2023-12-06T18:33:55.045 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.045 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.045 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.045 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:28:21.574549+0000", 2023-12-06T18:33:55.045 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.054815584000000001 2023-12-06T18:33:55.046 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.046 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.046 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.046 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.046 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.046 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:55.046 INFO:teuthology.orchestra.run.smithi061.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540138322906710015","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"]},"monmap":{"epoch":1,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","modified":"2023-12-06T18:25:15.366722Z","created":"2023-12-06T18:25:15.366722Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3300","nonce":0},{"type":"v1","addr":"172.21.15.61:6789","nonce":0}]},"addr":"172.21.15.61:6789/0","public_addr":"172.21.15.61:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:3301","nonce":0},{"type":"v1","addr":"172.21.15.61:6790","nonce":0}]},"addr":"172.21.15.61:6790/0","public_addr":"172.21.15.61:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2023-12-06T18:33:55.051 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2023-12-06T18:33:55.059 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2023-12-06T18:33:55.059 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2023-12-06T18:33:55.068 INFO:tasks.thrashosds:joining thrashosds 2023-12-06T18:33:55.195 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:55.196 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:55.196 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:55.196 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:55.205 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2023-12-06T18:33:55.334 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:55.334 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:55.334 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:55.334 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:55.334 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:55.343 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2023-12-06T18:33:55.511 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:55.512 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:55.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:55.512 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:33:55.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.512 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:1 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'1, mlcod=0'0)", 2023-12-06T18:33:55.512 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:33:55.512 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 418.895781881, 2023-12-06T18:33:55.512 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.16758073300000001, 2023-12-06T18:33:55.512 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.512 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.512 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.512 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.512 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:33:55.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.572911+0000", 2023-12-06T18:33:55.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.513 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.573092+0000", 2023-12-06T18:33:55.513 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000180586 2023-12-06T18:33:55.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.513 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.513 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.577420+0000", 2023-12-06T18:33:55.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.004327961 2023-12-06T18:33:55.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.578309+0000", 2023-12-06T18:33:55.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088976500000000004 2023-12-06T18:33:55.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.514 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.514 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.586627+0000", 2023-12-06T18:33:55.514 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0083180140000000003 2023-12-06T18:33:55.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.514 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.602221+0000", 2023-12-06T18:33:55.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.015593877000000001 2023-12-06T18:33:55.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.612261+0000", 2023-12-06T18:33:55.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010039197 2023-12-06T18:33:55.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.515 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.515 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.727604+0000", 2023-12-06T18:33:55.515 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.115343644 2023-12-06T18:33:55.515 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.515 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.516 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.516 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:56.740492+0000", 2023-12-06T18:33:55.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.012887689000000001 2023-12-06T18:33:55.516 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.516 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.516 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.516 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:31 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'17, mlcod=18'14)", 2023-12-06T18:33:55.516 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:33:55.516 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 417.87295487199998, 2023-12-06T18:33:55.516 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.065613500000000005, 2023-12-06T18:33:55.516 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.516 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.516 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.516 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:33:55.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595738+0000", 2023-12-06T18:33:55.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.517 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.517 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595749+0000", 2023-12-06T18:33:55.517 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0539e-05 2023-12-06T18:33:55.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.517 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.595930+0000", 2023-12-06T18:33:55.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000181571 2023-12-06T18:33:55.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596289+0000", 2023-12-06T18:33:55.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035877499999999998 2023-12-06T18:33:55.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.518 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.518 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.518 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.596474+0000", 2023-12-06T18:33:55.518 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00018465500000000001 2023-12-06T18:33:55.518 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.597760+0000", 2023-12-06T18:33:55.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0012863530000000001 2023-12-06T18:33:55.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.598161+0000", 2023-12-06T18:33:55.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040052099999999998 2023-12-06T18:33:55.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.519 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.519 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.651313+0000", 2023-12-06T18:33:55.519 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053152600000000001 2023-12-06T18:33:55.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.519 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.520 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.520 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.661352+0000", 2023-12-06T18:33:55.520 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010038485999999999 2023-12-06T18:33:55.520 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.520 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.520 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.520 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.520 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:32 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'18, mlcod=18'14)", 2023-12-06T18:33:55.520 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:33:55.521 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 417.86090823400002, 2023-12-06T18:33:55.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.061866929000000001, 2023-12-06T18:33:55.521 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.521 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.521 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:33:55.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.521 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.521 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.521 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607785+0000", 2023-12-06T18:33:55.521 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.521 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.522 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.522 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.607796+0000", 2023-12-06T18:33:55.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.1334e-05 2023-12-06T18:33:55.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.522 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.522 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609111+0000", 2023-12-06T18:33:55.522 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001315026 2023-12-06T18:33:55.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.522 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609309+0000", 2023-12-06T18:33:55.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000198228 2023-12-06T18:33:55.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.609500+0000", 2023-12-06T18:33:55.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000190755 2023-12-06T18:33:55.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.523 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.523 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610434+0000", 2023-12-06T18:33:55.523 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00093392900000000001 2023-12-06T18:33:55.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.523 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.610817+0000", 2023-12-06T18:33:55.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038260399999999998 2023-12-06T18:33:55.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.668994+0000", 2023-12-06T18:33:55.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.058177779999999998 2023-12-06T18:33:55.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.524 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.524 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.524 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.669652+0000", 2023-12-06T18:33:55.524 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00065727299999999999 2023-12-06T18:33:55.524 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.524 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.524 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.525 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:33 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'19, mlcod=18'14)", 2023-12-06T18:33:55.525 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:33:55.525 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 417.83950762699999, 2023-12-06T18:33:55.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046497610000000002, 2023-12-06T18:33:55.525 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.525 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.525 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.525 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.525 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:33:55.525 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.525 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.525 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.526 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.526 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629185+0000", 2023-12-06T18:33:55.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.526 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.526 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629194+0000", 2023-12-06T18:33:55.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.9910000000000008e-06 2023-12-06T18:33:55.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.526 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.526 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629327+0000", 2023-12-06T18:33:55.526 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013299199999999999 2023-12-06T18:33:55.526 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.526 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629532+0000", 2023-12-06T18:33:55.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00020508200000000001 2023-12-06T18:33:55.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.629651+0000", 2023-12-06T18:33:55.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000118363 2023-12-06T18:33:55.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.527 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.527 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.631863+0000", 2023-12-06T18:33:55.527 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0022119150000000001 2023-12-06T18:33:55.527 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.527 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.632302+0000", 2023-12-06T18:33:55.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00043898100000000002 2023-12-06T18:33:55.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675104+0000", 2023-12-06T18:33:55.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042801993000000003 2023-12-06T18:33:55.528 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.528 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.528 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.528 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675683+0000", 2023-12-06T18:33:55.528 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057929300000000004 2023-12-06T18:33:55.528 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.528 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.528 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.529 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:34 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'20, mlcod=18'14)", 2023-12-06T18:33:55.529 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:33:55.529 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 417.83832830400002, 2023-12-06T18:33:55.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045870143000000002, 2023-12-06T18:33:55.529 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.529 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.529 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.529 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.529 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.529 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:33:55.529 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.529 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.530 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630365+0000", 2023-12-06T18:33:55.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.530 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.530 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630371+0000", 2023-12-06T18:33:55.530 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.8640000000000001e-06 2023-12-06T18:33:55.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.530 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630470+0000", 2023-12-06T18:33:55.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9092000000000006e-05 2023-12-06T18:33:55.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.531 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.531 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630666+0000", 2023-12-06T18:33:55.531 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00019593 2023-12-06T18:33:55.531 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.531 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630769+0000", 2023-12-06T18:33:55.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000103503 2023-12-06T18:33:55.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639029+0000", 2023-12-06T18:33:55.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0082599530000000004 2023-12-06T18:33:55.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.532 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.639410+0000", 2023-12-06T18:33:55.532 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038107100000000002 2023-12-06T18:33:55.532 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.532 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.532 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.533 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.675712+0000", 2023-12-06T18:33:55.533 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036301483000000002 2023-12-06T18:33:55.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.533 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.533 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676235+0000", 2023-12-06T18:33:55.533 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00052324700000000001 2023-12-06T18:33:55.533 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.533 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.533 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.533 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.533 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:35 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'21, mlcod=18'14)", 2023-12-06T18:33:55.533 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:33:55.533 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 417.83775632300001, 2023-12-06T18:33:55.534 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.045820806999999998, 2023-12-06T18:33:55.534 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.534 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.534 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.534 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.534 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.534 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:33:55.534 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.534 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.534 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.534 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.534 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630937+0000", 2023-12-06T18:33:55.534 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.534 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.534 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.535 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.535 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.630942+0000", 2023-12-06T18:33:55.535 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.6570000000000002e-06 2023-12-06T18:33:55.535 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.535 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.535 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.535 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645267+0000", 2023-12-06T18:33:55.535 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.014324187 2023-12-06T18:33:55.535 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.535 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.535 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.535 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645383+0000", 2023-12-06T18:33:55.535 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116555 2023-12-06T18:33:55.535 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.535 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.645484+0000", 2023-12-06T18:33:55.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00010139 2023-12-06T18:33:55.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647191+0000", 2023-12-06T18:33:55.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001706132 2023-12-06T18:33:55.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.536 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.647567+0000", 2023-12-06T18:33:55.536 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037626400000000001 2023-12-06T18:33:55.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.536 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.536 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676256+0000", 2023-12-06T18:33:55.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028689063000000001 2023-12-06T18:33:55.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.537 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.537 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:57.676758+0000", 2023-12-06T18:33:55.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050155899999999995 2023-12-06T18:33:55.537 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.537 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.537 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.537 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.537 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.537 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:54 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'32, mlcod=19'29)", 2023-12-06T18:33:55.537 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:33:55.537 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 417.39192085299999, 2023-12-06T18:33:55.537 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034322347000000003, 2023-12-06T18:33:55.538 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.538 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.538 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.538 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.538 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.538 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:33:55.538 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.538 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.538 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.538 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076772+0000", 2023-12-06T18:33:55.538 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.538 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.538 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.539 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.076780+0000", 2023-12-06T18:33:55.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.0779999999999996e-06 2023-12-06T18:33:55.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.539 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.539 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.539 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.077875+0000", 2023-12-06T18:33:55.539 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001094516 2023-12-06T18:33:55.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.539 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.539 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.539 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078016+0000", 2023-12-06T18:33:55.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014125100000000001 2023-12-06T18:33:55.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078129+0000", 2023-12-06T18:33:55.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000112589 2023-12-06T18:33:55.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.540 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.540 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079314+0000", 2023-12-06T18:33:55.540 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0011855220000000001 2023-12-06T18:33:55.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.540 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.079692+0000", 2023-12-06T18:33:55.541 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000377809 2023-12-06T18:33:55.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.541 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.541 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.541 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.110244+0000", 2023-12-06T18:33:55.542 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030552201000000001 2023-12-06T18:33:55.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.542 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.542 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.542 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.111095+0000", 2023-12-06T18:33:55.542 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085038099999999999 2023-12-06T18:33:55.542 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.542 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.542 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.542 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.542 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:55 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'33, mlcod=19'29)", 2023-12-06T18:33:55.542 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:33:55.543 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 417.390320997, 2023-12-06T18:33:55.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034323139000000003, 2023-12-06T18:33:55.543 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.543 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.543 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.543 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.543 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:33:55.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.543 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.543 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078372+0000", 2023-12-06T18:33:55.543 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.543 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.544 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.544 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.078378+0000", 2023-12-06T18:33:55.544 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.0100000000000001e-06 2023-12-06T18:33:55.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.544 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.544 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.544 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086205+0000", 2023-12-06T18:33:55.544 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0078271279999999992 2023-12-06T18:33:55.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.544 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.544 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.544 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086322+0000", 2023-12-06T18:33:55.544 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116741 2023-12-06T18:33:55.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.544 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.545 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.086422+0000", 2023-12-06T18:33:55.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.9666000000000004e-05 2023-12-06T18:33:55.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.545 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.088945+0000", 2023-12-06T18:33:55.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0025234879999999999 2023-12-06T18:33:55.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.545 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.089317+0000", 2023-12-06T18:33:55.545 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037186700000000001 2023-12-06T18:33:55.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.545 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.545 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.546 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112120+0000", 2023-12-06T18:33:55.546 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.022803159999999999 2023-12-06T18:33:55.546 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.546 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.546 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.546 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.112695+0000", 2023-12-06T18:33:55.546 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00057507900000000004 2023-12-06T18:33:55.546 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.546 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.546 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.546 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.546 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.546 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:71 1.0 e19/16 1:30306672:devicehealth::main.db.0000000000000000:head v 19'45, mlcod=19'43)", 2023-12-06T18:33:55.546 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:33:55.546 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 417.01111538100002, 2023-12-06T18:33:55.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033896880999999997, 2023-12-06T18:33:55.547 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.547 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.547 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.547 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.547 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:33:55.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.547 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.547 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457578+0000", 2023-12-06T18:33:55.547 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.547 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.547 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.457586+0000", 2023-12-06T18:33:55.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 8.6799999999999999e-06 2023-12-06T18:33:55.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.548 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458468+0000", 2023-12-06T18:33:55.548 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00088145300000000003 2023-12-06T18:33:55.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.548 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.548 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.548 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458600+0000", 2023-12-06T18:33:55.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013208100000000001 2023-12-06T18:33:55.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.458709+0000", 2023-12-06T18:33:55.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000109379 2023-12-06T18:33:55.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459429+0000", 2023-12-06T18:33:55.549 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00072024199999999995 2023-12-06T18:33:55.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.549 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.549 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.549 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.459812+0000", 2023-12-06T18:33:55.550 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038210700000000002 2023-12-06T18:33:55.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.550 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.550 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.550 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.490608+0000", 2023-12-06T18:33:55.550 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.030796219 2023-12-06T18:33:55.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.550 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.550 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.550 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.491475+0000", 2023-12-06T18:33:55.550 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086671999999999997 2023-12-06T18:33:55.550 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.550 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.550 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.550 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.551 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:82 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'52, mlcod=19'50)", 2023-12-06T18:33:55.551 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:33:55.551 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 416.77361319900001, 2023-12-06T18:33:55.551 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041175386000000001, 2023-12-06T18:33:55.551 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.551 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.551 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.551 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.551 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.551 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:33:55.551 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.551 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.551 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.552 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.552 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695080+0000", 2023-12-06T18:33:55.552 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.552 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.552 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695090+0000", 2023-12-06T18:33:55.552 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.0105999999999999e-05 2023-12-06T18:33:55.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.552 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.552 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.552 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695237+0000", 2023-12-06T18:33:55.552 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014683200000000001 2023-12-06T18:33:55.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695412+0000", 2023-12-06T18:33:55.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017485800000000001 2023-12-06T18:33:55.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695531+0000", 2023-12-06T18:33:55.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000119756 2023-12-06T18:33:55.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.553 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.553 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.553 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696387+0000", 2023-12-06T18:33:55.553 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085599199999999997 2023-12-06T18:33:55.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.554 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.554 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.696781+0000", 2023-12-06T18:33:55.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00039372799999999998 2023-12-06T18:33:55.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.554 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.554 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.735593+0000", 2023-12-06T18:33:55.554 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.038811934999999999 2023-12-06T18:33:55.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.554 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.554 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.554 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.736255+0000", 2023-12-06T18:33:55.555 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00066217900000000004 2023-12-06T18:33:55.555 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.555 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.555 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.555 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.555 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:83 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'53, mlcod=19'50)", 2023-12-06T18:33:55.555 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:33:55.555 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 416.77288520000002, 2023-12-06T18:33:55.555 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.050475619999999999, 2023-12-06T18:33:55.555 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.555 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.555 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.555 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.555 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.556 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:33:55.556 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.556 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.556 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.556 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695808+0000", 2023-12-06T18:33:55.556 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.556 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.556 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.556 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.695814+0000", 2023-12-06T18:33:55.556 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7880000000000003e-06 2023-12-06T18:33:55.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.556 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.556 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.556 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700383+0000", 2023-12-06T18:33:55.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0045697139999999999 2023-12-06T18:33:55.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.557 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700505+0000", 2023-12-06T18:33:55.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012118899999999999 2023-12-06T18:33:55.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.557 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700607+0000", 2023-12-06T18:33:55.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000102927 2023-12-06T18:33:55.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.557 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.557 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.557 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702227+0000", 2023-12-06T18:33:55.557 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0016194510000000001 2023-12-06T18:33:55.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.558 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.558 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.558 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.702613+0000", 2023-12-06T18:33:55.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038605999999999997 2023-12-06T18:33:55.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.558 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.558 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.558 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.745665+0000", 2023-12-06T18:33:55.558 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.043051850000000003 2023-12-06T18:33:55.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.558 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.559 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.559 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746283+0000", 2023-12-06T18:33:55.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00061864099999999996 2023-12-06T18:33:55.559 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.559 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.559 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.559 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.559 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:84 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'54, mlcod=19'50)", 2023-12-06T18:33:55.559 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:33:55.559 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 416.76787279000001, 2023-12-06T18:33:55.559 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.046000471000000001, 2023-12-06T18:33:55.559 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.559 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.559 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.560 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.560 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.560 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:33:55.560 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.560 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.560 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.560 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.560 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700820+0000", 2023-12-06T18:33:55.560 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.560 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.560 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.560 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.560 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.700826+0000", 2023-12-06T18:33:55.560 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.9499999999999998e-06 2023-12-06T18:33:55.560 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.560 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.561 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.561 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705844+0000", 2023-12-06T18:33:55.561 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0050174429999999999 2023-12-06T18:33:55.561 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.561 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.561 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.561 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.705961+0000", 2023-12-06T18:33:55.561 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000117422 2023-12-06T18:33:55.561 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.561 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.561 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.561 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706068+0000", 2023-12-06T18:33:55.561 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000107236 2023-12-06T18:33:55.561 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.561 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.562 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.562 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709073+0000", 2023-12-06T18:33:55.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0030051980000000002 2023-12-06T18:33:55.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.562 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.562 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.562 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.709431+0000", 2023-12-06T18:33:55.562 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035785299999999999 2023-12-06T18:33:55.562 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.562 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.563 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.563 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746311+0000", 2023-12-06T18:33:55.563 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.036879558 2023-12-06T18:33:55.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.563 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.563 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.563 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746821+0000", 2023-12-06T18:33:55.563 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00050981099999999999 2023-12-06T18:33:55.563 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.563 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.563 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.563 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.563 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.563 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:85 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'55, mlcod=19'50)", 2023-12-06T18:33:55.563 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:33:55.563 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 416.76234571100002, 2023-12-06T18:33:55.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041032343999999998, 2023-12-06T18:33:55.564 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.564 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.564 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.564 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.564 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:33:55.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.564 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.564 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706347+0000", 2023-12-06T18:33:55.564 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.564 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.564 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.565 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706354+0000", 2023-12-06T18:33:55.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.4749999999999998e-06 2023-12-06T18:33:55.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.565 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706474+0000", 2023-12-06T18:33:55.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012010000000000001 2023-12-06T18:33:55.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.565 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706591+0000", 2023-12-06T18:33:55.565 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011673200000000001 2023-12-06T18:33:55.565 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.565 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.565 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706702+0000", 2023-12-06T18:33:55.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011116 2023-12-06T18:33:55.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.712646+0000", 2023-12-06T18:33:55.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0059438490000000002 2023-12-06T18:33:55.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.566 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.566 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.713002+0000", 2023-12-06T18:33:55.566 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00035613700000000002 2023-12-06T18:33:55.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.566 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.746841+0000", 2023-12-06T18:33:55.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.033838821999999998 2023-12-06T18:33:55.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.567 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.567 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747380+0000", 2023-12-06T18:33:55.567 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053906899999999998 2023-12-06T18:33:55.567 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.567 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.567 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.567 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.567 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:86 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'56, mlcod=19'50)", 2023-12-06T18:33:55.567 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:33:55.568 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 416.76183487399999, 2023-12-06T18:33:55.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041021828000000003, 2023-12-06T18:33:55.568 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.568 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.568 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.568 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.568 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:33:55.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.568 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.568 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.568 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706858+0000", 2023-12-06T18:33:55.568 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706864+0000", 2023-12-06T18:33:55.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.3630000000000002e-06 2023-12-06T18:33:55.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.706957+0000", 2023-12-06T18:33:55.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.3876999999999996e-05 2023-12-06T18:33:55.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.569 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.569 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707071+0000", 2023-12-06T18:33:55.569 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00011333299999999999 2023-12-06T18:33:55.569 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.569 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707165+0000", 2023-12-06T18:33:55.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.4266999999999997e-05 2023-12-06T18:33:55.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716023+0000", 2023-12-06T18:33:55.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0088576980000000007 2023-12-06T18:33:55.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.570 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.570 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.570 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.716395+0000", 2023-12-06T18:33:55.570 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00037262200000000002 2023-12-06T18:33:55.570 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.571 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747400+0000", 2023-12-06T18:33:55.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031004306999999998 2023-12-06T18:33:55.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.571 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.571 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747880+0000", 2023-12-06T18:33:55.571 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000480361 2023-12-06T18:33:55.571 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.571 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.571 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.571 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.571 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.571 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:87 1.0 e19/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 19'57, mlcod=19'50)", 2023-12-06T18:33:55.571 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:33:55.572 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 416.76137561299998, 2023-12-06T18:33:55.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.041112181999999997, 2023-12-06T18:33:55.572 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.572 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.572 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.572 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.572 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:33:55.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.572 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.572 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.572 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.572 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707317+0000", 2023-12-06T18:33:55.572 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.573 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.573 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.573 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.707323+0000", 2023-12-06T18:33:55.573 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.2159999999999997e-06 2023-12-06T18:33:55.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.573 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.573 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.573 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708184+0000", 2023-12-06T18:33:55.573 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00086159300000000002 2023-12-06T18:33:55.573 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.573 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708293+0000", 2023-12-06T18:33:55.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000108958 2023-12-06T18:33:55.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.708388+0000", 2023-12-06T18:33:55.574 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 9.5249999999999998e-05 2023-12-06T18:33:55.574 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.574 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.574 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.574 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719206+0000", 2023-12-06T18:33:55.575 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010817087 2023-12-06T18:33:55.577 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.577 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.577 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.577 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.719555+0000", 2023-12-06T18:33:55.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00034940699999999998 2023-12-06T18:33:55.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.578 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.578 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.578 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.747899+0000", 2023-12-06T18:33:55.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.028344227 2023-12-06T18:33:55.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.578 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.578 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.578 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:26:58.748430+0000", 2023-12-06T18:33:55.578 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00053044400000000003 2023-12-06T18:33:55.578 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.578 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.578 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.579 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:106 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'68, mlcod=20'64)", 2023-12-06T18:33:55.579 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:33:55.579 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 410.18145980600002, 2023-12-06T18:33:55.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.035487758000000001, 2023-12-06T18:33:55.579 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.579 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.579 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.579 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.579 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:33:55.579 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.579 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.579 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.579 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287233+0000", 2023-12-06T18:33:55.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.287244+0000", 2023-12-06T18:33:55.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.112e-05 2023-12-06T18:33:55.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.580 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288098+0000", 2023-12-06T18:33:55.580 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00085334300000000001 2023-12-06T18:33:55.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.580 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.580 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288272+0000", 2023-12-06T18:33:55.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017433699999999999 2023-12-06T18:33:55.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.288392+0000", 2023-12-06T18:33:55.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000120148 2023-12-06T18:33:55.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.581 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.289743+0000", 2023-12-06T18:33:55.581 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001350626 2023-12-06T18:33:55.581 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.581 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.581 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.290183+0000", 2023-12-06T18:33:55.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044070099999999999 2023-12-06T18:33:55.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.582 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.582 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.321877+0000", 2023-12-06T18:33:55.582 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.031693156 2023-12-06T18:33:55.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.582 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.583 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.583 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.322721+0000", 2023-12-06T18:33:55.583 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00084432699999999997 2023-12-06T18:33:55.583 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.583 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.583 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.583 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.583 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.583 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:107 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'69, mlcod=20'64)", 2023-12-06T18:33:55.583 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:33:55.583 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 410.17522200100001, 2023-12-06T18:33:55.584 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.037552410000000001, 2023-12-06T18:33:55.584 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.584 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.584 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.584 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.584 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.584 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:33:55.584 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.584 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.584 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.584 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293471+0000", 2023-12-06T18:33:55.584 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.584 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.293479+0000", 2023-12-06T18:33:55.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 7.4880000000000001e-06 2023-12-06T18:33:55.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298489+0000", 2023-12-06T18:33:55.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.005010151 2023-12-06T18:33:55.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.585 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.585 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298634+0000", 2023-12-06T18:33:55.585 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00014520099999999999 2023-12-06T18:33:55.585 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.585 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.586 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.586 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.298750+0000", 2023-12-06T18:33:55.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000116023 2023-12-06T18:33:55.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.586 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.586 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.586 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300388+0000", 2023-12-06T18:33:55.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001638261 2023-12-06T18:33:55.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.586 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.586 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.586 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.300777+0000", 2023-12-06T18:33:55.586 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00038878900000000001 2023-12-06T18:33:55.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.587 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.587 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.587 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.330011+0000", 2023-12-06T18:33:55.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.029233762 2023-12-06T18:33:55.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.587 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.587 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.587 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.331023+0000", 2023-12-06T18:33:55.587 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010127350000000001 2023-12-06T18:33:55.587 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.587 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.587 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.587 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.588 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.588 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:109 1.0 e22/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 22'71, mlcod=20'64)", 2023-12-06T18:33:55.588 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:33:55.588 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 410.16915892200001, 2023-12-06T18:33:55.588 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.034117963000000001, 2023-12-06T18:33:55.588 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.588 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.588 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.588 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.588 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.588 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:33:55.588 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.589 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.589 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.589 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299534+0000", 2023-12-06T18:33:55.589 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.589 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.589 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.589 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.299540+0000", 2023-12-06T18:33:55.589 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.429e-06 2023-12-06T18:33:55.589 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.589 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.589 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.589 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309791+0000", 2023-12-06T18:33:55.589 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010251757 2023-12-06T18:33:55.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.590 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.590 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.590 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.309912+0000", 2023-12-06T18:33:55.590 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00012107700000000001 2023-12-06T18:33:55.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.590 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.590 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.590 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310045+0000", 2023-12-06T18:33:55.590 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013278000000000001 2023-12-06T18:33:55.590 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.590 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.590 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.590 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.310812+0000", 2023-12-06T18:33:55.590 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00076642500000000005 2023-12-06T18:33:55.591 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.591 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.591 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.591 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.311212+0000", 2023-12-06T18:33:55.591 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00040080200000000002 2023-12-06T18:33:55.591 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.591 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.591 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.591 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333031+0000", 2023-12-06T18:33:55.591 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.021818852999999999 2023-12-06T18:33:55.591 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.591 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.591 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.591 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.333652+0000", 2023-12-06T18:33:55.591 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00062084 2023-12-06T18:33:55.592 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.592 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.592 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.592 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.592 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.592 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:111 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'73, mlcod=22'71)", 2023-12-06T18:33:55.592 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:33:55.592 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 410.03177183700001, 2023-12-06T18:33:55.592 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.048053836000000003, 2023-12-06T18:33:55.592 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.592 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.592 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.592 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.592 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.592 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:33:55.593 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.593 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.593 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.593 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.593 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436921+0000", 2023-12-06T18:33:55.593 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.593 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.593 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.593 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.593 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.436927+0000", 2023-12-06T18:33:55.593 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 5.7860000000000002e-06 2023-12-06T18:33:55.593 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.593 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.593 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.593 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443037+0000", 2023-12-06T18:33:55.593 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0061097039999999997 2023-12-06T18:33:55.594 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.594 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.594 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.594 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443176+0000", 2023-12-06T18:33:55.594 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013961299999999999 2023-12-06T18:33:55.594 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.594 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.594 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.594 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443334+0000", 2023-12-06T18:33:55.594 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015731799999999999 2023-12-06T18:33:55.594 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.594 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.594 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.594 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444400+0000", 2023-12-06T18:33:55.595 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001066427 2023-12-06T18:33:55.595 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.595 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.595 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.595 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.444812+0000", 2023-12-06T18:33:55.595 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00041184700000000002 2023-12-06T18:33:55.595 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.595 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.595 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.595 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484050+0000", 2023-12-06T18:33:55.595 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.039238220999999997 2023-12-06T18:33:55.595 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.595 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.595 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.595 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.484975+0000", 2023-12-06T18:33:55.596 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00092491999999999997 2023-12-06T18:33:55.596 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.596 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.596 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.596 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.596 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.596 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_repop(client.4160.0:112 1.0 e23/16 1:30306672:devicehealth::main.db.0000000000000000:head v 23'74, mlcod=22'71)", 2023-12-06T18:33:55.596 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:33:55.596 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 410.02510110999998, 2023-12-06T18:33:55.596 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.042290493999999998, 2023-12-06T18:33:55.596 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:55.596 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "commit sent; apply or cleanup", 2023-12-06T18:33:55.596 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:55.596 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.596 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:55.597 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:33:55.597 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.597 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.597 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.597 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:55.597 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443592+0000", 2023-12-06T18:33:55.597 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:55.597 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.597 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.597 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:55.597 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.443598+0000", 2023-12-06T18:33:55.597 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 6.477e-06 2023-12-06T18:33:55.597 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.597 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.597 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:55.598 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451199+0000", 2023-12-06T18:33:55.598 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0076008589999999997 2023-12-06T18:33:55.598 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.598 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.598 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:55.598 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451335+0000", 2023-12-06T18:33:55.598 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00013604700000000001 2023-12-06T18:33:55.598 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.598 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.598 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:55.598 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.451489+0000", 2023-12-06T18:33:55.599 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00015377199999999999 2023-12-06T18:33:55.599 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.599 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.599 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:55.599 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.452572+0000", 2023-12-06T18:33:55.599 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.001083381 2023-12-06T18:33:55.599 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.599 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.599 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "started", 2023-12-06T18:33:55.599 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.453013+0000", 2023-12-06T18:33:55.599 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00044044400000000001 2023-12-06T18:33:55.599 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.599 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.599 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "commit_sent", 2023-12-06T18:33:55.599 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485363+0000", 2023-12-06T18:33:55.600 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.032350174000000002 2023-12-06T18:33:55.600 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:55.600 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:55.600 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:55.600 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:27:05.485882+0000", 2023-12-06T18:33:55.600 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00051933999999999997 2023-12-06T18:33:55.600 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.600 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.600 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.600 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:55.600 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:55.600 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:55.602 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2023-12-06T18:33:55.731 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:55.732 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:55.732 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:55.732 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:55.741 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2023-12-06T18:33:55.872 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:55.872 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:55.872 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:55.873 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:55.873 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:55.881 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2023-12-06T18:33:56.012 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is set 2023-12-06T18:33:56.018 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:56.018 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:56.018 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:56.018 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [] 2023-12-06T18:33:56.018 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:56.027 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2023-12-06T18:33:56.160 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:56.160 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:56.161 INFO:teuthology.orchestra.run.smithi152.stdout: "num_ops": 0 2023-12-06T18:33:56.161 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:56.172 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2023-12-06T18:33:56.248 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2023-12-06T18:33:56.304 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:56.304 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [], 2023-12-06T18:33:56.304 INFO:teuthology.orchestra.run.smithi152.stdout: "complaint_time": 30, 2023-12-06T18:33:56.305 INFO:teuthology.orchestra.run.smithi152.stdout: "num_blocked_ops": 0 2023-12-06T18:33:56.305 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:56.313 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2023-12-06T18:33:56.450 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2023-12-06T18:33:56.450 INFO:teuthology.orchestra.run.smithi152.stdout: "size": 20, 2023-12-06T18:33:56.450 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 600, 2023-12-06T18:33:56.450 INFO:teuthology.orchestra.run.smithi152.stdout: "ops": [ 2023-12-06T18:33:56.450 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:56.451 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:8 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2023-12-06T18:33:56.451 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:56.451 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 214.78642696399999, 2023-12-06T18:33:56.451 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.085582945999999993, 2023-12-06T18:33:56.451 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:56.451 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:56.451 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:56.451 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:56.452 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:56.452 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 8 2023-12-06T18:33:56.452 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:56.452 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:56.452 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:56.452 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:56.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:56.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:56.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:56.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:56.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:56.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657067+0000", 2023-12-06T18:33:56.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:56.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:56.453 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:56.453 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:56.453 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.657297+0000", 2023-12-06T18:33:56.453 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00022962899999999999 2023-12-06T18:33:56.453 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:56.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:56.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:56.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.662212+0000", 2023-12-06T18:33:56.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0049149770000000001 2023-12-06T18:33:56.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:56.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:56.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:56.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.666900+0000", 2023-12-06T18:33:56.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0046874120000000002 2023-12-06T18:33:56.454 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:56.454 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:56.454 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:56.454 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.677187+0000", 2023-12-06T18:33:56.454 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.010287088 2023-12-06T18:33:56.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:56.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:56.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:56.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.688921+0000", 2023-12-06T18:33:56.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.011733957999999999 2023-12-06T18:33:56.455 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:56.455 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:56.455 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:56.455 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:30:21.742650+0000", 2023-12-06T18:33:56.455 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.053729882 2023-12-06T18:33:56.455 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:56.455 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:56.455 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:56.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:56.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:56.456 INFO:teuthology.orchestra.run.smithi152.stdout: "description": "osd_op(client.4201.0:12 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2023-12-06T18:33:56.456 INFO:teuthology.orchestra.run.smithi152.stdout: "initiated_at": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:56.456 INFO:teuthology.orchestra.run.smithi152.stdout: "age": 94.636480339000002, 2023-12-06T18:33:56.456 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0032177460000000001, 2023-12-06T18:33:56.456 INFO:teuthology.orchestra.run.smithi152.stdout: "type_data": { 2023-12-06T18:33:56.456 INFO:teuthology.orchestra.run.smithi152.stdout: "flag_point": "reached pg", 2023-12-06T18:33:56.456 INFO:teuthology.orchestra.run.smithi152.stdout: "client_info": { 2023-12-06T18:33:56.456 INFO:teuthology.orchestra.run.smithi152.stdout: "client": "client.4201", 2023-12-06T18:33:56.456 INFO:teuthology.orchestra.run.smithi152.stdout: "client_addr": "172.21.15.61:0/4232608308", 2023-12-06T18:33:56.456 INFO:teuthology.orchestra.run.smithi152.stdout: "tid": 12 2023-12-06T18:33:56.456 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:56.456 INFO:teuthology.orchestra.run.smithi152.stdout: "events": [ 2023-12-06T18:33:56.456 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:56.456 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "initiated", 2023-12-06T18:33:56.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:56.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:56.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:56.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:56.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "header_read", 2023-12-06T18:33:56.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807014+0000", 2023-12-06T18:33:56.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0 2023-12-06T18:33:56.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:56.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:56.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "throttled", 2023-12-06T18:33:56.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807030+0000", 2023-12-06T18:33:56.457 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 1.6030000000000001e-05 2023-12-06T18:33:56.457 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:56.457 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:56.457 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "all_read", 2023-12-06T18:33:56.457 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807210+0000", 2023-12-06T18:33:56.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00017936800000000001 2023-12-06T18:33:56.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:56.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:56.458 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "dispatched", 2023-12-06T18:33:56.458 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807443+0000", 2023-12-06T18:33:56.458 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.00023339699999999999 2023-12-06T18:33:56.458 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:56.458 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:56.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "queued_for_pg", 2023-12-06T18:33:56.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.807614+0000", 2023-12-06T18:33:56.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.000171032 2023-12-06T18:33:56.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:56.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:56.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "reached_pg", 2023-12-06T18:33:56.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.808696+0000", 2023-12-06T18:33:56.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0010817190000000001 2023-12-06T18:33:56.459 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2023-12-06T18:33:56.459 INFO:teuthology.orchestra.run.smithi152.stdout: { 2023-12-06T18:33:56.459 INFO:teuthology.orchestra.run.smithi152.stdout: "event": "done", 2023-12-06T18:33:56.459 INFO:teuthology.orchestra.run.smithi152.stdout: "time": "2023-12-06T18:32:21.810232+0000", 2023-12-06T18:33:56.459 INFO:teuthology.orchestra.run.smithi152.stdout: "duration": 0.0015361999999999999 2023-12-06T18:33:56.460 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:56.460 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:56.460 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:56.460 INFO:teuthology.orchestra.run.smithi152.stdout: } 2023-12-06T18:33:56.460 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2023-12-06T18:33:56.460 INFO:teuthology.orchestra.run.smithi152.stdout:} 2023-12-06T18:33:56.551 INFO:teuthology.orchestra.run.smithi061.stdout:mon.a: {} 2023-12-06T18:33:56.552 INFO:teuthology.orchestra.run.smithi061.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2023-12-06T18:33:56.666 INFO:teuthology.orchestra.run.smithi061.stdout:mon.b: {} 2023-12-06T18:33:56.667 INFO:teuthology.orchestra.run.smithi061.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2023-12-06T18:33:58.026 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2023-12-06T18:33:58.360 INFO:teuthology.orchestra.run.smithi061.stdout:mon.c: {} 2023-12-06T18:33:58.361 INFO:teuthology.orchestra.run.smithi061.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2023-12-06T18:33:58.372 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2023-12-06T18:33:59.384 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.0 to 1 (10000) 2023-12-06T18:33:59.398 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2023-12-06T18:33:59.435 INFO:teuthology.orchestra.run.smithi061.stderr:noscrub is unset 2023-12-06T18:33:59.448 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2023-12-06T18:34:00.426 INFO:teuthology.orchestra.run.smithi061.stderr:nodeep-scrub is unset 2023-12-06T18:34:00.478 INFO:teuthology.orchestra.run.smithi061.stderr:set osd.0 primary-affinity to 1 (8655362) 2023-12-06T18:34:00.491 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2023-12-06T18:34:01.618 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.1 to 1 (10000) 2023-12-06T18:34:01.632 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2023-12-06T18:34:02.325 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:02:41.770 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:02.377 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:02:41.822 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:02.400 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:02:41.845 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:02.419 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:01:36.010 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:02.420 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:01:36.012 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:02.446 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:01:36.037 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:02.452 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:30.702 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:02.517 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:30.767 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:02.748 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:30.998 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:03.285 INFO:teuthology.orchestra.run.smithi061.stderr:set osd.1 primary-affinity to 1 (8655362) 2023-12-06T18:34:03.301 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2023-12-06T18:34:04.957 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.2 to 1 (10000) 2023-12-06T18:34:04.972 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2023-12-06T18:34:05.976 INFO:teuthology.orchestra.run.smithi061.stderr:set osd.2 primary-affinity to 1 (8655362) 2023-12-06T18:34:05.991 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2023-12-06T18:34:07.153 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.3 to 1 (10000) 2023-12-06T18:34:07.167 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2023-12-06T18:34:08.488 INFO:teuthology.orchestra.run.smithi061.stderr:set osd.3 primary-affinity to 1 (8655362) 2023-12-06T18:34:08.502 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2023-12-06T18:34:09.551 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.4 to 1 (10000) 2023-12-06T18:34:09.565 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2023-12-06T18:34:10.638 INFO:teuthology.orchestra.run.smithi061.stderr:set osd.4 primary-affinity to 1 (8655362) 2023-12-06T18:34:10.655 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2023-12-06T18:34:11.778 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.5 to 1 (10000) 2023-12-06T18:34:11.791 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2023-12-06T18:34:13.357 INFO:teuthology.orchestra.run.smithi061.stderr:set osd.5 primary-affinity to 1 (8655362) 2023-12-06T18:34:13.372 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2023-12-06T18:34:14.952 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.6 to 1 (10000) 2023-12-06T18:34:14.967 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2023-12-06T18:34:15.965 INFO:teuthology.orchestra.run.smithi061.stderr:set osd.6 primary-affinity to 1 (8655362) 2023-12-06T18:34:15.978 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2023-12-06T18:34:17.083 INFO:teuthology.orchestra.run.smithi061.stderr:reweighted osd.7 to 1 (10000) 2023-12-06T18:34:17.102 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2023-12-06T18:34:18.608 INFO:teuthology.orchestra.run.smithi061.stderr:set osd.7 primary-affinity to 1 (8655362) 2023-12-06T18:34:18.622 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2023-12-06T18:34:18.622 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2023-12-06T18:34:18.622 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2023-12-06T18:34:18.622 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2023-12-06T18:34:18.622 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-12-06T18:34:18.622 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-06T18:34:19.419 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:34:19.419 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":143,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","created":"2023-12-06T18:26:07.122862+0000","modified":"2023-12-06T18:34:18.515479+0000","last_up_change":"2023-12-06T18:30:04.956811+0000","last_in_change":"2023-12-06T18:33:50.507883+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-06T18:26:52.648377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-12-06T18:27:01.712581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5e6e17b9-e72e-4e34-a531-0830d659bfe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6805","nonce":281956997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":281956997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":281956997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6807","nonce":281956997}]},"public_addr":"172.21.15.61:6805/281956997","cluster_addr":"172.21.15.61:6806/281956997","heartbeat_back_addr":"172.21.15.61:6808/281956997","heartbeat_front_addr":"172.21.15.61:6807/281956997","state":["exists","up"]},{"osd":1,"uuid":"b78c49b0-360d-4351-b44e-65ccaaeb768f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6809","nonce":3306625622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":3306625622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":3306625622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6811","nonce":3306625622}]},"public_addr":"172.21.15.61:6809/3306625622","cluster_addr":"172.21.15.61:6810/3306625622","heartbeat_back_addr":"172.21.15.61:6812/3306625622","heartbeat_front_addr":"172.21.15.61:6811/3306625622","state":["exists","up"]},{"osd":2,"uuid":"a2af3151-a4da-4c34-8fb6-78abf035785b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6801","nonce":3589270120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3589270120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3589270120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6803","nonce":3589270120}]},"public_addr":"172.21.15.61:6801/3589270120","cluster_addr":"172.21.15.61:6802/3589270120","heartbeat_back_addr":"172.21.15.61:6804/3589270120","heartbeat_front_addr":"172.21.15.61:6803/3589270120","state":["exists","up"]},{"osd":3,"uuid":"1c754ed6-c535-454f-b35b-00f62cd6ce5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":2849920842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":2849920842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":2849920842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6815","nonce":2849920842}]},"public_addr":"172.21.15.61:6813/2849920842","cluster_addr":"172.21.15.61:6814/2849920842","heartbeat_back_addr":"172.21.15.61:6816/2849920842","heartbeat_front_addr":"172.21.15.61:6815/2849920842","state":["exists","up"]},{"osd":4,"uuid":"766572af-b9ca-4f38-927d-36f26101ba51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2778770069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6801","nonce":2778770069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6803","nonce":2778770069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2778770069}]},"public_addr":"172.21.15.152:6800/2778770069","cluster_addr":"172.21.15.152:6801/2778770069","heartbeat_back_addr":"172.21.15.152:6803/2778770069","heartbeat_front_addr":"172.21.15.152:6802/2778770069","state":["exists","up"]},{"osd":5,"uuid":"7a566b60-1543-41d7-894c-e2fbb63e48dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6809","nonce":263671335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":263671335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":263671335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":263671335}]},"public_addr":"172.21.15.152:6809/263671335","cluster_addr":"172.21.15.152:6810/263671335","heartbeat_back_addr":"172.21.15.152:6814/263671335","heartbeat_front_addr":"172.21.15.152:6812/263671335","state":["exists","up"]},{"osd":6,"uuid":"28cc2358-4ac4-40b2-91a4-20edfe625a73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":4245639079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":4245639079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":4245639079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6813","nonce":4245639079}]},"public_addr":"172.21.15.152:6808/4245639079","cluster_addr":"172.21.15.152:6811/4245639079","heartbeat_back_addr":"172.21.15.152:6815/4245639079","heartbeat_front_addr":"172.21.15.152:6813/4245639079","state":["exists","up"]},{"osd":7,"uuid":"8f79279b-d2d0-438e-b3c2-88df1241ff30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":37,"up_from":61,"up_thru":62,"down_at":38,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":3075390540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6805","nonce":3075390540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6807","nonce":3075390540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":3075390540}]},"public_addr":"172.21.15.152:6804/3075390540","cluster_addr":"172.21.15.152:6805/3075390540","heartbeat_back_addr":"172.21.15.152:6807/3075390540","heartbeat_front_addr":"172.21.15.152:6806/3075390540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:52.171904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:52.428840+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:51.681954+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:53.687417+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:50.158682+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:50.443807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:49.750276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:49.770987+0000","dead_epoch":38}],"pg_upmap":[{"pgid":"2.7","osds":[5,3]}],"pg_upmap_items":[{"pgid":"1.0","mappings":[{"from":1,"to":0}]},{"pgid":"2.3","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-06T18:34:19.432 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-12-06T18:34:19.432 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-06T18:34:20.164 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:34:20.164 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":144,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","created":"2023-12-06T18:26:07.122862+0000","modified":"2023-12-06T18:34:19.529910+0000","last_up_change":"2023-12-06T18:30:04.956811+0000","last_in_change":"2023-12-06T18:33:50.507883+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-06T18:26:52.648377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-12-06T18:27:01.712581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5e6e17b9-e72e-4e34-a531-0830d659bfe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6805","nonce":281956997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":281956997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":281956997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6807","nonce":281956997}]},"public_addr":"172.21.15.61:6805/281956997","cluster_addr":"172.21.15.61:6806/281956997","heartbeat_back_addr":"172.21.15.61:6808/281956997","heartbeat_front_addr":"172.21.15.61:6807/281956997","state":["exists","up"]},{"osd":1,"uuid":"b78c49b0-360d-4351-b44e-65ccaaeb768f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6809","nonce":3306625622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":3306625622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":3306625622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6811","nonce":3306625622}]},"public_addr":"172.21.15.61:6809/3306625622","cluster_addr":"172.21.15.61:6810/3306625622","heartbeat_back_addr":"172.21.15.61:6812/3306625622","heartbeat_front_addr":"172.21.15.61:6811/3306625622","state":["exists","up"]},{"osd":2,"uuid":"a2af3151-a4da-4c34-8fb6-78abf035785b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6801","nonce":3589270120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3589270120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3589270120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6803","nonce":3589270120}]},"public_addr":"172.21.15.61:6801/3589270120","cluster_addr":"172.21.15.61:6802/3589270120","heartbeat_back_addr":"172.21.15.61:6804/3589270120","heartbeat_front_addr":"172.21.15.61:6803/3589270120","state":["exists","up"]},{"osd":3,"uuid":"1c754ed6-c535-454f-b35b-00f62cd6ce5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":2849920842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":2849920842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":2849920842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6815","nonce":2849920842}]},"public_addr":"172.21.15.61:6813/2849920842","cluster_addr":"172.21.15.61:6814/2849920842","heartbeat_back_addr":"172.21.15.61:6816/2849920842","heartbeat_front_addr":"172.21.15.61:6815/2849920842","state":["exists","up"]},{"osd":4,"uuid":"766572af-b9ca-4f38-927d-36f26101ba51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2778770069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6801","nonce":2778770069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6803","nonce":2778770069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2778770069}]},"public_addr":"172.21.15.152:6800/2778770069","cluster_addr":"172.21.15.152:6801/2778770069","heartbeat_back_addr":"172.21.15.152:6803/2778770069","heartbeat_front_addr":"172.21.15.152:6802/2778770069","state":["exists","up"]},{"osd":5,"uuid":"7a566b60-1543-41d7-894c-e2fbb63e48dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6809","nonce":263671335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":263671335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":263671335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":263671335}]},"public_addr":"172.21.15.152:6809/263671335","cluster_addr":"172.21.15.152:6810/263671335","heartbeat_back_addr":"172.21.15.152:6814/263671335","heartbeat_front_addr":"172.21.15.152:6812/263671335","state":["exists","up"]},{"osd":6,"uuid":"28cc2358-4ac4-40b2-91a4-20edfe625a73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":4245639079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":4245639079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":4245639079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6813","nonce":4245639079}]},"public_addr":"172.21.15.152:6808/4245639079","cluster_addr":"172.21.15.152:6811/4245639079","heartbeat_back_addr":"172.21.15.152:6815/4245639079","heartbeat_front_addr":"172.21.15.152:6813/4245639079","state":["exists","up"]},{"osd":7,"uuid":"8f79279b-d2d0-438e-b3c2-88df1241ff30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":37,"up_from":61,"up_thru":143,"down_at":38,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":3075390540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6805","nonce":3075390540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6807","nonce":3075390540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":3075390540}]},"public_addr":"172.21.15.152:6804/3075390540","cluster_addr":"172.21.15.152:6805/3075390540","heartbeat_back_addr":"172.21.15.152:6807/3075390540","heartbeat_front_addr":"172.21.15.152:6806/3075390540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:52.171904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:52.428840+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:51.681954+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:53.687417+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:50.158682+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:50.443807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:49.750276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:49.770987+0000","dead_epoch":38}],"pg_upmap":[{"pgid":"2.7","osds":[5,3]}],"pg_upmap_items":[{"pgid":"1.0","mappings":[{"from":1,"to":0}]},{"pgid":"2.3","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-06T18:34:20.178 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-12-06T18:34:20.178 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-12-06T18:34:20.179 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-12-06T18:34:20.179 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-12-06T18:34:20.179 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-12-06T18:34:20.180 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-12-06T18:34:20.180 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-12-06T18:34:20.181 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-12-06T18:34:20.547 INFO:teuthology.orchestra.run.smithi061.stdout:68719476828 2023-12-06T18:34:20.547 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-06T18:34:20.556 INFO:teuthology.orchestra.run.smithi061.stdout:77309411420 2023-12-06T18:34:20.557 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-06T18:34:20.588 INFO:teuthology.orchestra.run.smithi061.stdout:64424509534 2023-12-06T18:34:20.588 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-06T18:34:20.589 INFO:teuthology.orchestra.run.smithi061.stdout:261993005110 2023-12-06T18:34:20.589 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-06T18:34:20.590 INFO:teuthology.orchestra.run.smithi061.stdout:68719476828 2023-12-06T18:34:20.591 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-06T18:34:20.635 INFO:teuthology.orchestra.run.smithi061.stdout:60129542236 2023-12-06T18:34:20.635 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-06T18:34:20.665 INFO:teuthology.orchestra.run.smithi061.stdout:68719476829 2023-12-06T18:34:20.665 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-06T18:34:20.690 INFO:teuthology.orchestra.run.smithi061.stdout:64424509535 2023-12-06T18:34:20.691 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-06T18:34:21.442 INFO:teuthology.orchestra.run.smithi061.stdout:68719476827 2023-12-06T18:34:21.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476828 got 68719476827 for osd.1 2023-12-06T18:34:21.828 INFO:teuthology.orchestra.run.smithi061.stdout:261993005109 2023-12-06T18:34:21.847 INFO:tasks.ceph.ceph_manager.ceph:need seq 261993005110 got 261993005109 for osd.7 2023-12-06T18:34:21.851 INFO:teuthology.orchestra.run.smithi061.stdout:77309411419 2023-12-06T18:34:21.871 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411420 got 77309411419 for osd.3 2023-12-06T18:34:21.999 INFO:teuthology.orchestra.run.smithi061.stdout:64424509533 2023-12-06T18:34:22.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509534 got 64424509533 for osd.5 2023-12-06T18:34:22.024 INFO:teuthology.orchestra.run.smithi061.stdout:64424509534 2023-12-06T18:34:22.026 INFO:teuthology.orchestra.run.smithi061.stdout:68719476826 2023-12-06T18:34:22.043 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509535 got 64424509534 for osd.4 2023-12-06T18:34:22.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476828 got 68719476826 for osd.2 2023-12-06T18:34:22.057 INFO:teuthology.orchestra.run.smithi061.stdout:60129542235 2023-12-06T18:34:22.071 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542236 got 60129542235 for osd.6 2023-12-06T18:34:22.210 INFO:teuthology.orchestra.run.smithi061.stdout:68719476828 2023-12-06T18:34:22.224 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476829 got 68719476828 for osd.0 2023-12-06T18:34:22.456 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-06T18:34:22.848 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-06T18:34:22.872 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-06T18:34:23.017 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-06T18:34:23.043 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-06T18:34:23.046 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-06T18:34:23.072 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-06T18:34:23.225 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-06T18:34:23.264 INFO:teuthology.orchestra.run.smithi061.stdout:68719476827 2023-12-06T18:34:23.278 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476828 got 68719476827 for osd.1 2023-12-06T18:34:23.551 INFO:teuthology.orchestra.run.smithi061.stdout:261993005109 2023-12-06T18:34:23.572 INFO:tasks.ceph.ceph_manager.ceph:need seq 261993005110 got 261993005109 for osd.7 2023-12-06T18:34:23.759 INFO:teuthology.orchestra.run.smithi061.stdout:68719476828 2023-12-06T18:34:23.772 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476828 got 68719476828 for osd.2 2023-12-06T18:34:23.772 DEBUG:teuthology.parallel:result is None 2023-12-06T18:34:23.841 INFO:teuthology.orchestra.run.smithi061.stdout:77309411420 2023-12-06T18:34:23.858 INFO:tasks.ceph.ceph_manager.ceph:need seq 77309411420 got 77309411420 for osd.3 2023-12-06T18:34:23.858 DEBUG:teuthology.parallel:result is None 2023-12-06T18:34:23.946 INFO:teuthology.orchestra.run.smithi061.stdout:64424509534 2023-12-06T18:34:23.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509534 got 64424509534 for osd.5 2023-12-06T18:34:23.959 DEBUG:teuthology.parallel:result is None 2023-12-06T18:34:24.005 INFO:teuthology.orchestra.run.smithi061.stdout:64424509535 2023-12-06T18:34:24.017 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509535 got 64424509535 for osd.4 2023-12-06T18:34:24.017 DEBUG:teuthology.parallel:result is None 2023-12-06T18:34:24.280 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-06T18:34:24.286 INFO:teuthology.orchestra.run.smithi061.stdout:60129542236 2023-12-06T18:34:24.301 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542236 got 60129542236 for osd.6 2023-12-06T18:34:24.301 DEBUG:teuthology.parallel:result is None 2023-12-06T18:34:24.573 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-06T18:34:24.707 INFO:teuthology.orchestra.run.smithi061.stdout:68719476830 2023-12-06T18:34:24.721 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476829 got 68719476830 for osd.0 2023-12-06T18:34:24.721 DEBUG:teuthology.parallel:result is None 2023-12-06T18:34:24.890 INFO:teuthology.orchestra.run.smithi061.stdout:68719476829 2023-12-06T18:34:24.903 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476828 got 68719476829 for osd.1 2023-12-06T18:34:24.903 DEBUG:teuthology.parallel:result is None 2023-12-06T18:34:25.178 INFO:teuthology.orchestra.run.smithi061.stdout:261993005111 2023-12-06T18:34:25.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 261993005110 got 261993005111 for osd.7 2023-12-06T18:34:25.191 DEBUG:teuthology.parallel:result is None 2023-12-06T18:34:25.191 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2023-12-06T18:34:25.191 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:34:25.778 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:34:25.779 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:34:25.791 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2023-12-06T18:34:24.291663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231748,"kb_used_data":10376,"kb_used_omap":15,"kb_used_meta":217008,"kb_avail":754742972,"statfs":{"total":773094113280,"available":772856803328,"internally_reserved":0,"allocated":10625024,"data_stored":4311316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15557,"internal_metadata":222217019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.213710"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.633082+0000","last_change":"2023-12-06T18:29:09.051293+0000","last_active":"2023-12-06T18:34:18.633082+0000","last_peered":"2023-12-06T18:34:18.633082+0000","last_clean":"2023-12-06T18:34:18.633082+0000","last_became_active":"2023-12-06T18:29:09.045331+0000","last_became_peered":"2023-12-06T18:29:09.045331+0000","last_unstale":"2023-12-06T18:34:18.633082+0000","last_undegraded":"2023-12-06T18:34:18.633082+0000","last_fullsized":"2023-12-06T18:34:18.633082+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T02:46:01.854508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":50,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-12-06T18:34:17.193022+0000","last_change":"2023-12-06T18:33:00.623558+0000","last_active":"2023-12-06T18:34:17.193022+0000","last_peered":"2023-12-06T18:34:17.193022+0000","last_clean":"2023-12-06T18:34:17.193022+0000","last_became_active":"2023-12-06T18:33:00.616353+0000","last_became_peered":"2023-12-06T18:33:00.616353+0000","last_unstale":"2023-12-06T18:34:17.193022+0000","last_undegraded":"2023-12-06T18:34:17.193022+0000","last_fullsized":"2023-12-06T18:34:17.193022+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T18:33:31.069235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.603990+0000","last_change":"2023-12-06T18:33:00.592340+0000","last_active":"2023-12-06T18:34:18.603990+0000","last_peered":"2023-12-06T18:34:18.603990+0000","last_clean":"2023-12-06T18:34:18.603990+0000","last_became_active":"2023-12-06T18:33:00.581037+0000","last_became_peered":"2023-12-06T18:33:00.581037+0000","last_unstale":"2023-12-06T18:34:18.603990+0000","last_undegraded":"2023-12-06T18:34:18.603990+0000","last_fullsized":"2023-12-06T18:34:18.603990+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T06:13:54.498147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":118,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.631290+0000","last_change":"2023-12-06T18:33:52.522397+0000","last_active":"2023-12-06T18:34:18.631290+0000","last_peered":"2023-12-06T18:34:18.631290+0000","last_clean":"2023-12-06T18:34:18.631290+0000","last_became_active":"2023-12-06T18:33:52.516242+0000","last_became_peered":"2023-12-06T18:33:52.516242+0000","last_unstale":"2023-12-06T18:34:18.631290+0000","last_undegraded":"2023-12-06T18:34:18.631290+0000","last_fullsized":"2023-12-06T18:34:18.631290+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T03:19:25.211436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.599973+0000","last_change":"2023-12-06T18:33:52.521292+0000","last_active":"2023-12-06T18:34:18.599973+0000","last_peered":"2023-12-06T18:34:18.599973+0000","last_clean":"2023-12-06T18:34:18.599973+0000","last_became_active":"2023-12-06T18:33:52.514832+0000","last_became_peered":"2023-12-06T18:33:52.514832+0000","last_unstale":"2023-12-06T18:34:18.599973+0000","last_undegraded":"2023-12-06T18:34:18.599973+0000","last_fullsized":"2023-12-06T18:34:18.599973+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T23:51:07.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":94,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-12-06T18:34:19.744131+0000","last_change":"2023-12-06T18:34:19.744131+0000","last_active":"2023-12-06T18:34:19.744131+0000","last_peered":"2023-12-06T18:34:19.744131+0000","last_clean":"2023-12-06T18:34:19.744131+0000","last_became_active":"2023-12-06T18:34:19.738308+0000","last_became_peered":"2023-12-06T18:34:19.738308+0000","last_unstale":"2023-12-06T18:34:19.744131+0000","last_undegraded":"2023-12-06T18:34:19.744131+0000","last_fullsized":"2023-12-06T18:34:19.744131+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T01:32:29.556210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.597827+0000","last_change":"2023-12-06T18:28:47.876556+0000","last_active":"2023-12-06T18:34:18.597827+0000","last_peered":"2023-12-06T18:34:18.597827+0000","last_clean":"2023-12-06T18:34:18.597827+0000","last_became_active":"2023-12-06T18:28:47.865169+0000","last_became_peered":"2023-12-06T18:28:47.865169+0000","last_unstale":"2023-12-06T18:34:18.597827+0000","last_undegraded":"2023-12-06T18:34:18.597827+0000","last_fullsized":"2023-12-06T18:34:18.597827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:16:40.095542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":71,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-12-06T18:34:17.287726+0000","last_change":"2023-12-06T18:31:57.187068+0000","last_active":"2023-12-06T18:34:17.287726+0000","last_peered":"2023-12-06T18:34:17.287726+0000","last_clean":"2023-12-06T18:34:17.287726+0000","last_became_active":"2023-12-06T18:31:57.175444+0000","last_became_peered":"2023-12-06T18:31:57.175444+0000","last_unstale":"2023-12-06T18:34:17.287726+0000","last_undegraded":"2023-12-06T18:34:17.287726+0000","last_fullsized":"2023-12-06T18:34:17.287726+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:25:09.547812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":49,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:19.324726+0000","last_change":"2023-12-06T18:33:22.203137+0000","last_active":"2023-12-06T18:34:19.324726+0000","last_peered":"2023-12-06T18:34:19.324726+0000","last_clean":"2023-12-06T18:34:19.324726+0000","last_became_active":"2023-12-06T18:33:21.485790+0000","last_became_peered":"2023-12-06T18:33:21.485790+0000","last_unstale":"2023-12-06T18:34:19.324726+0000","last_undegraded":"2023-12-06T18:34:19.324726+0000","last_fullsized":"2023-12-06T18:34:19.324726+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T01:25:32.645799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1605632,"data_stored":1591360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411421,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":11.292,"5min":11.291,"15min":10.609999999999999},"min":{"1min":5.069,"5min":3.835,"15min":3.835},"max":{"1min":27.620000000000001,"5min":108.57599999999999,"15min":108.57599999999999},"last":4.641},{"interface":"front","average":{"1min":11.958,"5min":11.699,"15min":12.119999999999999},"min":{"1min":6.3239999999999998,"5min":4.8639999999999999,"15min":3.7200000000000002},"max":{"1min":24.844000000000001,"5min":128.09399999999999,"15min":128.09399999999999},"last":10.798999999999999}]},{"osd":1,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":11.891999999999999,"5min":12.403,"15min":12.693},"min":{"1min":6.5359999999999996,"5min":6.2539999999999996,"15min":6.2539999999999996},"max":{"1min":19.538,"5min":70.072000000000003,"15min":70.072000000000003},"last":12.27},{"interface":"front","average":{"1min":12.019,"5min":12.824,"15min":14.788},"min":{"1min":6.2080000000000002,"5min":6.2080000000000002,"15min":6.2080000000000002},"max":{"1min":30.16,"5min":30.16,"15min":32.920000000000002},"last":11.516999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":15.49,"5min":15.119,"15min":14.845000000000001},"min":{"1min":9.4450000000000003,"5min":8.5079999999999991,"15min":7.5149999999999997},"max":{"1min":52.232999999999997,"5min":75.125,"15min":75.125},"last":21.791},{"interface":"front","average":{"1min":18.545000000000002,"5min":16.343,"15min":17.477},"min":{"1min":9.2599999999999998,"5min":7.5389999999999997,"15min":7.5389999999999997},"max":{"1min":54.323999999999998,"5min":69.403999999999996,"15min":69.403999999999996},"last":9.4269999999999996}]},{"osd":4,"last update":"Wed Dec 6 18:34:13 2023","interfaces":[{"interface":"back","average":{"1min":18.949999999999999,"5min":15.185,"15min":12.898999999999999},"min":{"1min":10.714,"5min":5.8879999999999999,"15min":5.8879999999999999},"max":{"1min":56.255000000000003,"5min":56.255000000000003,"15min":56.255000000000003},"last":15.881},{"interface":"front","average":{"1min":14.9,"5min":13.204000000000001,"15min":15.038},"min":{"1min":6.4189999999999996,"5min":6.3099999999999996,"15min":6.3099999999999996},"max":{"1min":54.616,"5min":71.718000000000004,"15min":71.718000000000004},"last":18.234999999999999}]},{"osd":5,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":17.082000000000001,"5min":18.359999999999999,"15min":18.027999999999999},"min":{"1min":11.888,"5min":10.417999999999999,"15min":8.3179999999999996},"max":{"1min":24.445,"5min":75.751999999999995,"15min":75.751999999999995},"last":23.423999999999999},{"interface":"front","average":{"1min":16.565999999999999,"5min":16.934000000000001,"15min":17.869},"min":{"1min":8.5389999999999997,"5min":7.9269999999999996,"15min":7.9269999999999996},"max":{"1min":32.219000000000001,"5min":70.933000000000007,"15min":70.933000000000007},"last":17.038}]},{"osd":6,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":19.771000000000001,"5min":15.018000000000001,"15min":15.869999999999999},"min":{"1min":10.798,"5min":8.2050000000000001,"15min":7.0919999999999996},"max":{"1min":30.991,"5min":30.991,"15min":30.991},"last":13.795},{"interface":"front","average":{"1min":16.899999999999999,"5min":18.946999999999999,"15min":20.498000000000001},"min":{"1min":7.2210000000000001,"5min":7.2210000000000001,"15min":7.2210000000000001},"max":{"1min":30.404,"5min":74.224000000000004,"15min":74.224000000000004},"last":14.584}]},{"osd":7,"last update":"Wed Dec 6 18:34:13 2023","interfaces":[{"interface":"back","average":{"1min":20.172999999999998,"5min":19.478999999999999,"15min":21.131},"min":{"1min":12.255000000000001,"5min":9.1859999999999999,"15min":9.1859999999999999},"max":{"1min":30.175000000000001,"5min":122.93000000000001,"15min":122.93000000000001},"last":20.257999999999999},{"interface":"front","average":{"1min":22.155000000000001,"5min":22.471,"15min":23.312999999999999},"min":{"1min":15.08,"5min":13.196,"15min":13.196},"max":{"1min":34.466999999999999,"5min":124.473,"15min":124.473},"last":19.452999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476828,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1220608,"data_stored":436729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":14.244999999999999,"5min":11.606999999999999,"15min":11.34},"min":{"1min":5.234,"5min":4.1639999999999997,"15min":4.1639999999999997},"max":{"1min":47.884999999999998,"5min":47.884999999999998,"15min":47.884999999999998},"last":16.073},{"interface":"front","average":{"1min":12.324999999999999,"5min":12.346,"15min":11.509},"min":{"1min":7.1399999999999997,"5min":4.3280000000000003,"15min":4.3280000000000003},"max":{"1min":48.305999999999997,"5min":48.305999999999997,"15min":48.305999999999997},"last":8.1489999999999991}]},{"osd":1,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":14.621,"5min":12.172000000000001,"15min":11.682},"min":{"1min":6.8860000000000001,"5min":5.9139999999999997,"15min":5.9139999999999997},"max":{"1min":28.242999999999999,"5min":42.920999999999999,"15min":42.920999999999999},"last":7.0469999999999997},{"interface":"front","average":{"1min":17.202999999999999,"5min":14.548,"15min":14.061999999999999},"min":{"1min":7.8840000000000003,"5min":6.1479999999999997,"15min":6.1479999999999997},"max":{"1min":29.478999999999999,"5min":34.607999999999997,"15min":34.607999999999997},"last":27.266999999999999}]},{"osd":3,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":16.474,"5min":15.353,"15min":17.821999999999999},"min":{"1min":9.0139999999999993,"5min":7.2549999999999999,"15min":6.1260000000000003},"max":{"1min":37.463000000000001,"5min":49.609000000000002,"15min":112.129},"last":19.754000000000001},{"interface":"front","average":{"1min":21.956,"5min":17.753,"15min":21.963999999999999},"min":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"max":{"1min":36.770000000000003,"5min":61.243000000000002,"15min":181.601},"last":32.057000000000002}]},{"osd":4,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":19.707999999999998,"5min":16.417999999999999,"15min":17.349},"min":{"1min":8.5250000000000004,"5min":7.8070000000000004,"15min":6.609},"max":{"1min":35.396999999999998,"5min":60.332000000000001,"15min":97.088999999999999},"last":25.579999999999998},{"interface":"front","average":{"1min":13.917999999999999,"5min":13.611000000000001,"15min":18.606999999999999},"min":{"1min":7.7640000000000002,"5min":6.0720000000000001,"15min":6.0720000000000001},"max":{"1min":24.061,"5min":43.198,"15min":98.069000000000003},"last":18.262}]},{"osd":5,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":19.113,"5min":18.113,"15min":18.097000000000001},"min":{"1min":10.743,"5min":8.2100000000000009,"15min":7.8170000000000002},"max":{"1min":26.042000000000002,"5min":57.628999999999998,"15min":57.628999999999998},"last":15.68},{"interface":"front","average":{"1min":15.692,"5min":15.933,"15min":15.824},"min":{"1min":10.292,"5min":8.5990000000000002,"15min":8.5990000000000002},"max":{"1min":39.875999999999998,"5min":47.749000000000002,"15min":47.749000000000002},"last":25.132999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":20.952999999999999,"5min":19.140000000000001,"15min":36.003999999999998},"min":{"1min":7.0419999999999998,"5min":7.0419999999999998,"15min":7.0419999999999998},"max":{"1min":47.667999999999999,"5min":64.034999999999997,"15min":498.38499999999999},"last":31.239999999999998},{"interface":"front","average":{"1min":24.446000000000002,"5min":20.491,"15min":37.847000000000001},"min":{"1min":12.461,"5min":9.7010000000000005,"15min":9.7010000000000005},"max":{"1min":44.993000000000002,"5min":54.920000000000002,"15min":496.17399999999998},"last":19.067}]},{"osd":7,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":21.704999999999998,"5min":19.547000000000001,"15min":19.131},"min":{"1min":13.574999999999999,"5min":10.064,"15min":10.064},"max":{"1min":35.945,"5min":49.456000000000003,"15min":49.456000000000003},"last":29.776},{"interface":"front","average":{"1min":21.277999999999999,"5min":20.684000000000001,"15min":20.332000000000001},"min":{"1min":12.81,"5min":12.288,"15min":12.288},"max":{"1min":41.965000000000003,"5min":41.965000000000003,"15min":41.965000000000003},"last":28.280999999999999}]}]},{"osd":1,"up_from":16,"seq":68719476829,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1220608,"data_stored":436729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":12.32,"5min":11.260999999999999,"15min":18.989000000000001},"min":{"1min":6.0650000000000004,"5min":5.0890000000000004,"15min":5.0890000000000004},"max":{"1min":34.816000000000003,"5min":34.816000000000003,"15min":238.87100000000001},"last":10.407},{"interface":"front","average":{"1min":11.336,"5min":10.768000000000001,"15min":34.408000000000001},"min":{"1min":5.0629999999999997,"5min":5.0629999999999997,"15min":4.4269999999999996},"max":{"1min":22.834,"5min":61.262,"15min":661.33600000000001},"last":6.8460000000000001}]},{"osd":2,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.962999999999999,"5min":13.722,"15min":27.556999999999999},"min":{"1min":6.6189999999999998,"5min":6.6189999999999998,"15min":6.6189999999999998},"max":{"1min":57.93,"5min":93.168000000000006,"15min":384.315},"last":9.7629999999999999},{"interface":"front","average":{"1min":22.902000000000001,"5min":19.120999999999999,"15min":39.317999999999998},"min":{"1min":12.282,"5min":10.724,"15min":5.7009999999999996},"max":{"1min":60.173000000000002,"5min":94.230000000000004,"15min":656.98699999999997},"last":18.166}]},{"osd":3,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":12.403,"5min":12.194000000000001,"15min":12.863},"min":{"1min":6.9180000000000001,"5min":5.9720000000000004,"15min":5.548},"max":{"1min":25.774999999999999,"5min":31.16,"15min":34.057000000000002},"last":11.914},{"interface":"front","average":{"1min":14.6,"5min":14.064,"15min":15.500999999999999},"min":{"1min":8.0180000000000007,"5min":6.5730000000000004,"15min":6.5730000000000004},"max":{"1min":31.417999999999999,"5min":65.537000000000006,"15min":65.537000000000006},"last":14.833}]},{"osd":4,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.082000000000001,"5min":13.308,"15min":33.085999999999999},"min":{"1min":8.6059999999999999,"5min":6.7009999999999996,"15min":6.2930000000000001},"max":{"1min":33.427999999999997,"5min":33.427999999999997,"15min":447.67200000000003},"last":15.486000000000001},{"interface":"front","average":{"1min":21.420000000000002,"5min":18.047999999999998,"15min":39.893000000000001},"min":{"1min":7.7300000000000004,"5min":7.7300000000000004,"15min":7.0919999999999996},"max":{"1min":40.564,"5min":44.043999999999997,"15min":630.572},"last":8.4939999999999998}]},{"osd":5,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.390000000000001,"5min":16.076000000000001,"15min":35.482999999999997},"min":{"1min":7.2549999999999999,"5min":7.0540000000000003,"15min":7.0540000000000003},"max":{"1min":38.991999999999997,"5min":68.013999999999996,"15min":389.55000000000001},"last":9.1639999999999997},{"interface":"front","average":{"1min":22.140999999999998,"5min":18.395,"15min":45.957999999999998},"min":{"1min":11.099,"5min":7.5359999999999996,"15min":7.0410000000000004},"max":{"1min":41.655999999999999,"5min":41.655999999999999,"15min":658.48599999999999},"last":14.227}]},{"osd":6,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.672000000000001,"5min":15.733000000000001,"15min":30.202000000000002},"min":{"1min":8.9760000000000009,"5min":7.5899999999999999,"15min":7.5899999999999999},"max":{"1min":32.039999999999999,"5min":54.383000000000003,"15min":439.56400000000002},"last":12.510999999999999},{"interface":"front","average":{"1min":23.643000000000001,"5min":20.5,"15min":32.639000000000003},"min":{"1min":13.058,"5min":10.641999999999999,"15min":10.641999999999999},"max":{"1min":43.052,"5min":55.423999999999999,"15min":386.71499999999997},"last":17.300999999999998}]},{"osd":7,"last update":"Wed Dec 6 18:34:16 2023","interfaces":[{"interface":"back","average":{"1min":18.527000000000001,"5min":18.222999999999999,"15min":18.600000000000001},"min":{"1min":8.9849999999999994,"5min":8.9849999999999994,"15min":8.9849999999999994},"max":{"1min":33.156999999999996,"5min":97.561999999999998,"15min":97.561999999999998},"last":16.169},{"interface":"front","average":{"1min":23.125,"5min":21.024000000000001,"15min":18.771999999999998},"min":{"1min":12.019,"5min":8.7219999999999995,"15min":8.7219999999999995},"max":{"1min":36.332000000000001,"5min":101.762,"15min":101.762},"last":19.048999999999999}]}]},{"osd":0,"up_from":16,"seq":68719476830,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343220,"statfs":{"total":96636764160,"available":96607457280,"internally_reserved":0,"allocated":1634304,"data_stored":838986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":13.518000000000001,"5min":11.452,"15min":15.608000000000001},"min":{"1min":5.6340000000000003,"5min":3.2360000000000002,"15min":3.2360000000000002},"max":{"1min":54.155000000000001,"5min":54.155000000000001,"15min":116.593},"last":5.2599999999999998},{"interface":"front","average":{"1min":13.859,"5min":12.935,"15min":16.576000000000001},"min":{"1min":5.5389999999999997,"5min":4.9039999999999999,"15min":4.9039999999999999},"max":{"1min":68.778999999999996,"5min":68.778999999999996,"15min":117.896},"last":11.353}]},{"osd":2,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":17.434000000000001,"5min":14.888,"15min":14.132999999999999},"min":{"1min":8.9870000000000001,"5min":5.694,"15min":5.694},"max":{"1min":72.131,"5min":72.131,"15min":72.131},"last":24.241},{"interface":"front","average":{"1min":14.316000000000001,"5min":13.224,"15min":12.055},"min":{"1min":6.7590000000000003,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":59.317999999999998,"5min":59.317999999999998,"15min":59.317999999999998},"last":25.105}]},{"osd":3,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":19.911999999999999,"5min":16.183,"15min":16.312999999999999},"min":{"1min":8.9600000000000009,"5min":6.1379999999999999,"15min":6.1379999999999999},"max":{"1min":75.439999999999998,"5min":75.439999999999998,"15min":75.439999999999998},"last":12.33},{"interface":"front","average":{"1min":19.082000000000001,"5min":16.047000000000001,"15min":16.204999999999998},"min":{"1min":7.593,"5min":7.5510000000000002,"15min":7.5510000000000002},"max":{"1min":96.688999999999993,"5min":96.688999999999993,"15min":96.688999999999993},"last":19.693999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":14.302,"5min":12.134,"15min":17.890999999999998},"min":{"1min":6.8239999999999998,"5min":5.9349999999999996,"15min":5.9349999999999996},"max":{"1min":57.694000000000003,"5min":57.694000000000003,"15min":82.432000000000002},"last":17.805},{"interface":"front","average":{"1min":19.693000000000001,"5min":18.327000000000002,"15min":19.846},"min":{"1min":7.7469999999999999,"5min":7.7469999999999999,"15min":6.2629999999999999},"max":{"1min":76.078000000000003,"5min":76.078000000000003,"15min":81.239999999999995},"last":9.0329999999999995}]},{"osd":5,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":19.27,"5min":16.798999999999999,"15min":17.507000000000001},"min":{"1min":8.1229999999999993,"5min":8.0099999999999998,"15min":8.0099999999999998},"max":{"1min":97.947000000000003,"5min":97.947000000000003,"15min":97.947000000000003},"last":16.667999999999999},{"interface":"front","average":{"1min":16.989000000000001,"5min":15.443,"15min":17.440000000000001},"min":{"1min":7.4770000000000003,"5min":7.1779999999999999,"15min":7.1779999999999999},"max":{"1min":69.915999999999997,"5min":69.915999999999997,"15min":72.307000000000002},"last":12.667999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":25.541,"5min":20.477,"15min":19.626000000000001},"min":{"1min":12.661,"5min":11.401,"15min":7.7560000000000002},"max":{"1min":82.775000000000006,"5min":82.775000000000006,"15min":82.775000000000006},"last":14.757999999999999},{"interface":"front","average":{"1min":21.75,"5min":17.940999999999999,"15min":18.914999999999999},"min":{"1min":9.7729999999999997,"5min":9.3520000000000003,"15min":9.3520000000000003},"max":{"1min":74.057000000000002,"5min":74.057000000000002,"15min":74.057000000000002},"last":21.257999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":26.137,"5min":21.724,"15min":19.988},"min":{"1min":16.024000000000001,"5min":12.009,"15min":12.009},"max":{"1min":95.287999999999997,"5min":95.287999999999997,"15min":95.287999999999997},"last":16.366},{"interface":"front","average":{"1min":23.494,"5min":20.041,"15min":18.611000000000001},"min":{"1min":10.24,"5min":10.24,"15min":10.24},"max":{"1min":98.995999999999995,"5min":98.995999999999995,"15min":98.995999999999995},"last":22.509}]}]},{"osd":5,"up_from":15,"seq":64424509534,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28628,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343212,"statfs":{"total":96636764160,"available":96607449088,"internally_reserved":0,"allocated":1634304,"data_stored":838986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":14.44,"5min":9.6449999999999996,"15min":13.73},"min":{"1min":4.0430000000000001,"5min":3.3660000000000001,"15min":3.3660000000000001},"max":{"1min":106.521,"5min":106.521,"15min":208.291},"last":12.817},{"interface":"front","average":{"1min":14.817,"5min":11.609,"15min":17.067},"min":{"1min":4.984,"5min":4.758,"15min":4.4299999999999997},"max":{"1min":107.342,"5min":107.342,"15min":199.94900000000001},"last":14.343999999999999}]},{"osd":1,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":11.631,"5min":13.298999999999999,"15min":45.225999999999999},"min":{"1min":6.7469999999999999,"5min":5.085,"15min":5.085},"max":{"1min":26.692,"5min":41.234999999999999,"15min":1211.6010000000001},"last":11.567},{"interface":"front","average":{"1min":12.16,"5min":10.502000000000001,"15min":41.011000000000003},"min":{"1min":5.359,"5min":4.7199999999999998,"15min":4.7199999999999998},"max":{"1min":22.664000000000001,"5min":39.802999999999997,"15min":1189.3869999999999},"last":7.7850000000000001}]},{"osd":2,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":14.343,"5min":14.629,"15min":21.640999999999998},"min":{"1min":9.0389999999999997,"5min":6.5129999999999999,"15min":6.5129999999999999},"max":{"1min":23.422000000000001,"5min":68.617999999999995,"15min":283.46800000000002},"last":9.1549999999999994},{"interface":"front","average":{"1min":19.576000000000001,"5min":18.463999999999999,"15min":25.539000000000001},"min":{"1min":12.851000000000001,"5min":8.173,"15min":8.173},"max":{"1min":40.737000000000002,"5min":69.513999999999996,"15min":296.47800000000001},"last":26.452999999999999}]},{"osd":3,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":11.484999999999999,"5min":13.811,"15min":17.414000000000001},"min":{"1min":7.3639999999999999,"5min":6.0300000000000002,"15min":6.0300000000000002},"max":{"1min":19.872,"5min":36.156999999999996,"15min":47.926000000000002},"last":10.923999999999999},{"interface":"front","average":{"1min":15.419,"5min":14.555999999999999,"15min":15.007999999999999},"min":{"1min":10.163,"5min":5.3979999999999997,"15min":5.3979999999999997},"max":{"1min":28.914999999999999,"5min":34.609000000000002,"15min":66.176000000000002},"last":25.614999999999998}]},{"osd":4,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":18.212,"5min":17.952000000000002,"15min":19.096},"min":{"1min":11.897,"5min":9.7210000000000001,"15min":9.7210000000000001},"max":{"1min":34.069000000000003,"5min":41.923999999999999,"15min":67.248000000000005},"last":12.191000000000001},{"interface":"front","average":{"1min":11.249000000000001,"5min":10.798,"15min":12.347},"min":{"1min":6.5279999999999996,"5min":5.3300000000000001,"15min":5.3300000000000001},"max":{"1min":26.850999999999999,"5min":43.101999999999997,"15min":77.266999999999996},"last":10.210000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":15.667999999999999,"5min":13.644,"15min":15.140000000000001},"min":{"1min":7.984,"5min":6.1280000000000001,"15min":5.915},"max":{"1min":25.727,"5min":44.587000000000003,"15min":76.751999999999995},"last":8.1690000000000005},{"interface":"front","average":{"1min":20.158999999999999,"5min":19.356999999999999,"15min":20.469000000000001},"min":{"1min":11.744999999999999,"5min":11.744999999999999,"15min":8.8390000000000004},"max":{"1min":36.883000000000003,"5min":45.366,"15min":57.101999999999997},"last":24.887}]},{"osd":7,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":18.687999999999999,"5min":16.294,"15min":14.904999999999999},"min":{"1min":11.298999999999999,"5min":7.5519999999999996,"15min":7.5519999999999996},"max":{"1min":32.219999999999999,"5min":32.219999999999999,"15min":32.219999999999999},"last":31.006},{"interface":"front","average":{"1min":14.401999999999999,"5min":12.872999999999999,"15min":12.215999999999999},"min":{"1min":7.133,"5min":7.133,"15min":7.133},"max":{"1min":29.376999999999999,"5min":29.376999999999999,"15min":29.376999999999999},"last":24.254000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509535,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28208,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343632,"statfs":{"total":96636764160,"available":96607879168,"internally_reserved":0,"allocated":1216512,"data_stored":436448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":8.0180000000000007,"5min":9.266,"15min":12.295999999999999},"min":{"1min":4.2789999999999999,"5min":3.7429999999999999,"15min":3.7429999999999999},"max":{"1min":13.465999999999999,"5min":41.076000000000001,"15min":53.298999999999999},"last":7.6959999999999997},{"interface":"front","average":{"1min":9.9160000000000004,"5min":10,"15min":10.609999999999999},"min":{"1min":5.1280000000000001,"5min":4.8140000000000001,"15min":3.9670000000000001},"max":{"1min":18.329000000000001,"5min":35.863,"15min":52.567},"last":12.557}]},{"osd":1,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":12.202999999999999,"5min":11.432,"15min":13.298999999999999},"min":{"1min":6.0529999999999999,"5min":5.8390000000000004,"15min":5.835},"max":{"1min":21.186,"5min":22.509,"15min":49.183},"last":22.41},{"interface":"front","average":{"1min":13.43,"5min":12.129,"15min":14.662000000000001},"min":{"1min":5.9770000000000003,"5min":5.9770000000000003,"15min":5.96},"max":{"1min":23.596,"5min":24.081,"15min":45.412999999999997},"last":25.928999999999998}]},{"osd":2,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":12.398999999999999,"5min":12.423,"15min":19.385000000000002},"min":{"1min":6.609,"5min":5.9539999999999997,"15min":5.6390000000000002},"max":{"1min":22.904,"5min":44.649000000000001,"15min":85.394000000000005},"last":21.577000000000002},{"interface":"front","average":{"1min":14.196,"5min":15.087999999999999,"15min":18.899000000000001},"min":{"1min":7.9269999999999996,"5min":7.5209999999999999,"15min":6.3639999999999999},"max":{"1min":27.780000000000001,"5min":42.500999999999998,"15min":69.530000000000001},"last":10.314}]},{"osd":3,"last update":"Wed Dec 6 18:34:17 2023","interfaces":[{"interface":"back","average":{"1min":16.875,"5min":15.66,"15min":14.851000000000001},"min":{"1min":8.6349999999999998,"5min":8.0250000000000004,"15min":7.008},"max":{"1min":35.418999999999997,"5min":37.561999999999998,"15min":37.561999999999998},"last":24.260999999999999},{"interface":"front","average":{"1min":18.140000000000001,"5min":14.212,"15min":14.936},"min":{"1min":8.0619999999999994,"5min":5.5549999999999997,"15min":5.5549999999999997},"max":{"1min":36.694000000000003,"5min":36.694000000000003,"15min":36.694000000000003},"last":36.694000000000003}]},{"osd":5,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":16.562000000000001,"5min":13.952999999999999,"15min":16.228999999999999},"min":{"1min":6.9370000000000003,"5min":5.4969999999999999,"15min":5.4969999999999999},"max":{"1min":27.283000000000001,"5min":53.840000000000003,"15min":78.798000000000002},"last":11.042999999999999},{"interface":"front","average":{"1min":16.690000000000001,"5min":17.414000000000001,"15min":18.402000000000001},"min":{"1min":10.425000000000001,"5min":8.0519999999999996,"15min":6.4450000000000003},"max":{"1min":29.184000000000001,"5min":52.781999999999996,"15min":55.814999999999998},"last":6.6500000000000004}]},{"osd":6,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":19,"5min":18.105,"15min":18.045000000000002},"min":{"1min":9.3170000000000002,"5min":9.2100000000000009,"15min":7.7850000000000001},"max":{"1min":32.869,"5min":41.723999999999997,"15min":51.149000000000001},"last":9.6039999999999992},{"interface":"front","average":{"1min":16.550000000000001,"5min":15.154999999999999,"15min":16.164999999999999},"min":{"1min":7.9359999999999999,"5min":7.9359999999999999,"15min":7.8700000000000001},"max":{"1min":29.792000000000002,"5min":34.929000000000002,"15min":40.426000000000002},"last":33.087000000000003}]},{"osd":7,"last update":"Wed Dec 6 18:34:17 2023","interfaces":[{"interface":"back","average":{"1min":17.311,"5min":17.135000000000002,"15min":18.212},"min":{"1min":11.176,"5min":8.093,"15min":8.093},"max":{"1min":24.667000000000002,"5min":60.390000000000001,"15min":60.390000000000001},"last":11.692},{"interface":"front","average":{"1min":21.228999999999999,"5min":21.193999999999999,"15min":22.27},"min":{"1min":11.083,"5min":11.083,"15min":11.083},"max":{"1min":33.914999999999999,"5min":57.606999999999999,"15min":57.606999999999999},"last":23.609000000000002}]}]},{"osd":7,"up_from":61,"seq":261993005111,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33412,"kb_used_data":1204,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94338428,"statfs":{"total":96636764160,"available":96602550272,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4395,"internal_metadata":28634837},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.206,"5min":11.058999999999999,"15min":10.446999999999999},"min":{"1min":4.0739999999999998,"5min":4.0739999999999998,"15min":4.0739999999999998},"max":{"1min":126.286,"5min":126.286,"15min":126.286},"last":4.2300000000000004},{"interface":"front","average":{"1min":15.504,"5min":11.65,"15min":11.226000000000001},"min":{"1min":5.2320000000000002,"5min":4.2530000000000001,"15min":4.2530000000000001},"max":{"1min":127.35599999999999,"5min":127.35599999999999,"15min":127.35599999999999},"last":7.7160000000000002}]},{"osd":1,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":12.824999999999999,"5min":13.972,"15min":16.297000000000001},"min":{"1min":5.9130000000000003,"5min":4.6929999999999996,"15min":4.6929999999999996},"max":{"1min":24.047999999999998,"5min":64.765000000000001,"15min":64.765000000000001},"last":6.2510000000000003},{"interface":"front","average":{"1min":13.289999999999999,"5min":15.44,"15min":17.219999999999999},"min":{"1min":6.2969999999999997,"5min":6.2969999999999997,"15min":6.2969999999999997},"max":{"1min":24.402999999999999,"5min":65.994,"15min":65.994},"last":13.346}]},{"osd":2,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.021000000000001,"5min":16.204000000000001,"15min":17.411999999999999},"min":{"1min":7.9569999999999999,"5min":7.9569999999999999,"15min":7.9569999999999999},"max":{"1min":24.219000000000001,"5min":46.478000000000002,"15min":46.478000000000002},"last":13.962999999999999},{"interface":"front","average":{"1min":14.906000000000001,"5min":14.161,"15min":14.132999999999999},"min":{"1min":6.415,"5min":6.415,"15min":6.415},"max":{"1min":23.727,"5min":41.603999999999999,"15min":41.603999999999999},"last":11.414}]},{"osd":3,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":17.462,"5min":15.693,"15min":16.739000000000001},"min":{"1min":7.8879999999999999,"5min":6.8319999999999999,"15min":6.8319999999999999},"max":{"1min":47.823,"5min":60.463999999999999,"15min":60.463999999999999},"last":12.75},{"interface":"front","average":{"1min":20.838999999999999,"5min":18.907,"15min":19.867999999999999},"min":{"1min":9.8889999999999993,"5min":9.7370000000000001,"15min":9.7370000000000001},"max":{"1min":44.863,"5min":56.286000000000001,"15min":56.286000000000001},"last":17.911999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":20.128,"5min":20.585000000000001,"15min":21.081},"min":{"1min":10.489000000000001,"5min":8.4960000000000004,"15min":8.4960000000000004},"max":{"1min":32.167999999999999,"5min":57.716999999999999,"15min":57.716999999999999},"last":18.552},{"interface":"front","average":{"1min":16.364000000000001,"5min":17.821999999999999,"15min":18.356000000000002},"min":{"1min":7.2699999999999996,"5min":7.2699999999999996,"15min":7.2699999999999996},"max":{"1min":30.831,"5min":67.159000000000006,"15min":67.159000000000006},"last":15.808}]},{"osd":5,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.968999999999999,"5min":14.587999999999999,"15min":14.688000000000001},"min":{"1min":7.4219999999999997,"5min":6.3399999999999999,"15min":6.3399999999999999},"max":{"1min":27.07,"5min":44.884,"15min":44.884},"last":9.7699999999999996},{"interface":"front","average":{"1min":20.382999999999999,"5min":18.724,"15min":19.527999999999999},"min":{"1min":9.3520000000000003,"5min":9.3520000000000003,"15min":9.3520000000000003},"max":{"1min":92.616,"5min":92.616,"15min":92.616},"last":10.680999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":21.416,"5min":17.657,"15min":17.068999999999999},"min":{"1min":7.625,"5min":7.625,"15min":7.625},"max":{"1min":95.501000000000005,"5min":95.501000000000005,"15min":95.501000000000005},"last":19.923999999999999},{"interface":"front","average":{"1min":23.699999999999999,"5min":20.664999999999999,"15min":20.841000000000001},"min":{"1min":10.811999999999999,"5min":10.144,"15min":10.144},"max":{"1min":94.533000000000001,"5min":94.533000000000001,"15min":94.533000000000001},"last":16.48}]}]},{"osd":6,"up_from":14,"seq":60129542237,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":9.9879999999999995,"5min":9.4030000000000005,"15min":9.8219999999999992},"min":{"1min":4.3010000000000002,"5min":4.1260000000000003,"15min":3.8519999999999999},"max":{"1min":22.216000000000001,"5min":27.033000000000001,"15min":46.947000000000003},"last":3.8140000000000001},{"interface":"front","average":{"1min":11.436,"5min":10.529,"15min":10.420999999999999},"min":{"1min":4.2919999999999998,"5min":4.0650000000000004,"15min":4.0650000000000004},"max":{"1min":18.338999999999999,"5min":33.152000000000001,"15min":45.953000000000003},"last":9.923}]},{"osd":1,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":10.019,"5min":9.423,"15min":11.574999999999999},"min":{"1min":5.6189999999999998,"5min":4.3760000000000003,"15min":4.3760000000000003},"max":{"1min":18.969000000000001,"5min":66.313000000000002,"15min":66.313000000000002},"last":7.0270000000000001},{"interface":"front","average":{"1min":12.763999999999999,"5min":12.657,"15min":13.935},"min":{"1min":6.875,"5min":5.9349999999999996,"15min":5.8540000000000001},"max":{"1min":21.193000000000001,"5min":67.533000000000001,"15min":67.533000000000001},"last":14.852}]},{"osd":2,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.625999999999999,"5min":13.667,"15min":18.298999999999999},"min":{"1min":9.8510000000000009,"5min":6.0640000000000001,"15min":6.0640000000000001},"max":{"1min":41.106000000000002,"5min":46.051000000000002,"15min":200.88499999999999},"last":11.593999999999999},{"interface":"front","average":{"1min":20.809999999999999,"5min":17.507000000000001,"15min":16.818000000000001},"min":{"1min":12.57,"5min":7.8129999999999997,"15min":7.4119999999999999},"max":{"1min":39.863,"5min":45.003999999999998,"15min":48.005000000000003},"last":20.276}]},{"osd":3,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.510000000000002,"5min":14.548999999999999,"15min":14.074},"min":{"1min":9.1600000000000001,"5min":5.9820000000000002,"15min":5.9820000000000002},"max":{"1min":26.93,"5min":36.283999999999999,"15min":36.283999999999999},"last":9.1609999999999996},{"interface":"front","average":{"1min":19.988,"5min":16.667000000000002,"15min":16.466000000000001},"min":{"1min":11.983000000000001,"5min":7.75,"15min":7.6079999999999997},"max":{"1min":31.489999999999998,"5min":34.317,"15min":47.006999999999998},"last":12.333}]},{"osd":4,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.172999999999998,"5min":16.164000000000001,"15min":15.500999999999999},"min":{"1min":10.173999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"max":{"1min":27.928000000000001,"5min":49.170000000000002,"15min":49.170000000000002},"last":10.673},{"interface":"front","average":{"1min":19.071000000000002,"5min":17.736000000000001,"15min":17.052},"min":{"1min":12.409000000000001,"5min":8.9019999999999992,"15min":7.7169999999999996},"max":{"1min":32.399000000000001,"5min":38.573999999999998,"15min":48.988999999999997},"last":19.446999999999999}]},{"osd":5,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":20.074000000000002,"5min":18.266999999999999,"15min":18.411999999999999},"min":{"1min":13.186999999999999,"5min":7.8879999999999999,"15min":7.8879999999999999},"max":{"1min":30.518999999999998,"5min":49.390999999999998,"15min":57.308},"last":21.152000000000001},{"interface":"front","average":{"1min":11.132,"5min":10.885,"15min":13.561999999999999},"min":{"1min":5.5380000000000003,"5min":5.5380000000000003,"15min":5.5380000000000003},"max":{"1min":15.542999999999999,"5min":47.415999999999997,"15min":68.463999999999999},"last":13.837}]},{"osd":7,"last update":"Wed Dec 6 18:34:18 2023","interfaces":[{"interface":"back","average":{"1min":18.114000000000001,"5min":16.631,"15min":15.811},"min":{"1min":9.1460000000000008,"5min":7.4390000000000001,"15min":7.4390000000000001},"max":{"1min":26.856000000000002,"5min":38.887999999999998,"15min":38.887999999999998},"last":15.589},{"interface":"front","average":{"1min":22.518999999999998,"5min":20.096,"15min":19.210000000000001},"min":{"1min":16.774999999999999,"5min":13.555999999999999,"15min":13.555999999999999},"max":{"1min":34.822000000000003,"5min":34.822000000000003,"15min":34.822000000000003},"last":22.228000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:34:25.793 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:34:26.360 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:34:26.361 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:34:26.372 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2023-12-06T18:34:26.291967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231776,"kb_used_data":10412,"kb_used_omap":15,"kb_used_meta":217008,"kb_avail":754742944,"statfs":{"total":773094113280,"available":772856774656,"internally_reserved":0,"allocated":10661888,"data_stored":4324567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15557,"internal_metadata":222217019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001391"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.633082+0000","last_change":"2023-12-06T18:29:09.051293+0000","last_active":"2023-12-06T18:34:18.633082+0000","last_peered":"2023-12-06T18:34:18.633082+0000","last_clean":"2023-12-06T18:34:18.633082+0000","last_became_active":"2023-12-06T18:29:09.045331+0000","last_became_peered":"2023-12-06T18:29:09.045331+0000","last_unstale":"2023-12-06T18:34:18.633082+0000","last_undegraded":"2023-12-06T18:34:18.633082+0000","last_fullsized":"2023-12-06T18:34:18.633082+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T02:46:01.854508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":51,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:19.332340+0000","last_change":"2023-12-06T18:33:00.623558+0000","last_active":"2023-12-06T18:34:19.332340+0000","last_peered":"2023-12-06T18:34:19.332340+0000","last_clean":"2023-12-06T18:34:19.332340+0000","last_became_active":"2023-12-06T18:33:00.616353+0000","last_became_peered":"2023-12-06T18:33:00.616353+0000","last_unstale":"2023-12-06T18:34:19.332340+0000","last_undegraded":"2023-12-06T18:34:19.332340+0000","last_fullsized":"2023-12-06T18:34:19.332340+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T18:33:31.069235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.603990+0000","last_change":"2023-12-06T18:33:00.592340+0000","last_active":"2023-12-06T18:34:18.603990+0000","last_peered":"2023-12-06T18:34:18.603990+0000","last_clean":"2023-12-06T18:34:18.603990+0000","last_became_active":"2023-12-06T18:33:00.581037+0000","last_became_peered":"2023-12-06T18:33:00.581037+0000","last_unstale":"2023-12-06T18:34:18.603990+0000","last_undegraded":"2023-12-06T18:34:18.603990+0000","last_fullsized":"2023-12-06T18:34:18.603990+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T06:13:54.498147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":118,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.631290+0000","last_change":"2023-12-06T18:33:52.522397+0000","last_active":"2023-12-06T18:34:18.631290+0000","last_peered":"2023-12-06T18:34:18.631290+0000","last_clean":"2023-12-06T18:34:18.631290+0000","last_became_active":"2023-12-06T18:33:52.516242+0000","last_became_peered":"2023-12-06T18:33:52.516242+0000","last_unstale":"2023-12-06T18:34:18.631290+0000","last_undegraded":"2023-12-06T18:34:18.631290+0000","last_fullsized":"2023-12-06T18:34:18.631290+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T03:19:25.211436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.599973+0000","last_change":"2023-12-06T18:33:52.521292+0000","last_active":"2023-12-06T18:34:18.599973+0000","last_peered":"2023-12-06T18:34:18.599973+0000","last_clean":"2023-12-06T18:34:18.599973+0000","last_became_active":"2023-12-06T18:33:52.514832+0000","last_became_peered":"2023-12-06T18:33:52.514832+0000","last_unstale":"2023-12-06T18:34:18.599973+0000","last_undegraded":"2023-12-06T18:34:18.599973+0000","last_fullsized":"2023-12-06T18:34:18.599973+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T23:51:07.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":94,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-12-06T18:34:19.744131+0000","last_change":"2023-12-06T18:34:19.744131+0000","last_active":"2023-12-06T18:34:19.744131+0000","last_peered":"2023-12-06T18:34:19.744131+0000","last_clean":"2023-12-06T18:34:19.744131+0000","last_became_active":"2023-12-06T18:34:19.738308+0000","last_became_peered":"2023-12-06T18:34:19.738308+0000","last_unstale":"2023-12-06T18:34:19.744131+0000","last_undegraded":"2023-12-06T18:34:19.744131+0000","last_fullsized":"2023-12-06T18:34:19.744131+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T01:32:29.556210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.597827+0000","last_change":"2023-12-06T18:28:47.876556+0000","last_active":"2023-12-06T18:34:18.597827+0000","last_peered":"2023-12-06T18:34:18.597827+0000","last_clean":"2023-12-06T18:34:18.597827+0000","last_became_active":"2023-12-06T18:28:47.865169+0000","last_became_peered":"2023-12-06T18:28:47.865169+0000","last_unstale":"2023-12-06T18:34:18.597827+0000","last_undegraded":"2023-12-06T18:34:18.597827+0000","last_fullsized":"2023-12-06T18:34:18.597827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:16:40.095542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":72,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.991987+0000","last_change":"2023-12-06T18:31:57.187068+0000","last_active":"2023-12-06T18:34:18.991987+0000","last_peered":"2023-12-06T18:34:18.991987+0000","last_clean":"2023-12-06T18:34:18.991987+0000","last_became_active":"2023-12-06T18:31:57.175444+0000","last_became_peered":"2023-12-06T18:31:57.175444+0000","last_unstale":"2023-12-06T18:34:18.991987+0000","last_undegraded":"2023-12-06T18:34:18.991987+0000","last_fullsized":"2023-12-06T18:34:18.991987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:25:09.547812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":49,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:19.324726+0000","last_change":"2023-12-06T18:33:22.203137+0000","last_active":"2023-12-06T18:34:19.324726+0000","last_peered":"2023-12-06T18:34:19.324726+0000","last_clean":"2023-12-06T18:34:19.324726+0000","last_became_active":"2023-12-06T18:33:21.485790+0000","last_became_peered":"2023-12-06T18:33:21.485790+0000","last_unstale":"2023-12-06T18:34:19.324726+0000","last_undegraded":"2023-12-06T18:34:19.324726+0000","last_fullsized":"2023-12-06T18:34:19.324726+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T01:25:32.645799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1605632,"data_stored":1591360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411421,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":11.292,"5min":11.291,"15min":10.609999999999999},"min":{"1min":5.069,"5min":3.835,"15min":3.835},"max":{"1min":27.620000000000001,"5min":108.57599999999999,"15min":108.57599999999999},"last":4.641},{"interface":"front","average":{"1min":11.958,"5min":11.699,"15min":12.119999999999999},"min":{"1min":6.3239999999999998,"5min":4.8639999999999999,"15min":3.7200000000000002},"max":{"1min":24.844000000000001,"5min":128.09399999999999,"15min":128.09399999999999},"last":10.798999999999999}]},{"osd":1,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":11.891999999999999,"5min":12.403,"15min":12.693},"min":{"1min":6.5359999999999996,"5min":6.2539999999999996,"15min":6.2539999999999996},"max":{"1min":19.538,"5min":70.072000000000003,"15min":70.072000000000003},"last":12.27},{"interface":"front","average":{"1min":12.019,"5min":12.824,"15min":14.788},"min":{"1min":6.2080000000000002,"5min":6.2080000000000002,"15min":6.2080000000000002},"max":{"1min":30.16,"5min":30.16,"15min":32.920000000000002},"last":11.516999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":15.49,"5min":15.119,"15min":14.845000000000001},"min":{"1min":9.4450000000000003,"5min":8.5079999999999991,"15min":7.5149999999999997},"max":{"1min":52.232999999999997,"5min":75.125,"15min":75.125},"last":21.791},{"interface":"front","average":{"1min":18.545000000000002,"5min":16.343,"15min":17.477},"min":{"1min":9.2599999999999998,"5min":7.5389999999999997,"15min":7.5389999999999997},"max":{"1min":54.323999999999998,"5min":69.403999999999996,"15min":69.403999999999996},"last":9.4269999999999996}]},{"osd":4,"last update":"Wed Dec 6 18:34:13 2023","interfaces":[{"interface":"back","average":{"1min":18.949999999999999,"5min":15.185,"15min":12.898999999999999},"min":{"1min":10.714,"5min":5.8879999999999999,"15min":5.8879999999999999},"max":{"1min":56.255000000000003,"5min":56.255000000000003,"15min":56.255000000000003},"last":15.881},{"interface":"front","average":{"1min":14.9,"5min":13.204000000000001,"15min":15.038},"min":{"1min":6.4189999999999996,"5min":6.3099999999999996,"15min":6.3099999999999996},"max":{"1min":54.616,"5min":71.718000000000004,"15min":71.718000000000004},"last":18.234999999999999}]},{"osd":5,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":17.082000000000001,"5min":18.359999999999999,"15min":18.027999999999999},"min":{"1min":11.888,"5min":10.417999999999999,"15min":8.3179999999999996},"max":{"1min":24.445,"5min":75.751999999999995,"15min":75.751999999999995},"last":23.423999999999999},{"interface":"front","average":{"1min":16.565999999999999,"5min":16.934000000000001,"15min":17.869},"min":{"1min":8.5389999999999997,"5min":7.9269999999999996,"15min":7.9269999999999996},"max":{"1min":32.219000000000001,"5min":70.933000000000007,"15min":70.933000000000007},"last":17.038}]},{"osd":6,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":19.771000000000001,"5min":15.018000000000001,"15min":15.869999999999999},"min":{"1min":10.798,"5min":8.2050000000000001,"15min":7.0919999999999996},"max":{"1min":30.991,"5min":30.991,"15min":30.991},"last":13.795},{"interface":"front","average":{"1min":16.899999999999999,"5min":18.946999999999999,"15min":20.498000000000001},"min":{"1min":7.2210000000000001,"5min":7.2210000000000001,"15min":7.2210000000000001},"max":{"1min":30.404,"5min":74.224000000000004,"15min":74.224000000000004},"last":14.584}]},{"osd":7,"last update":"Wed Dec 6 18:34:13 2023","interfaces":[{"interface":"back","average":{"1min":20.172999999999998,"5min":19.478999999999999,"15min":21.131},"min":{"1min":12.255000000000001,"5min":9.1859999999999999,"15min":9.1859999999999999},"max":{"1min":30.175000000000001,"5min":122.93000000000001,"15min":122.93000000000001},"last":20.257999999999999},{"interface":"front","average":{"1min":22.155000000000001,"5min":22.471,"15min":23.312999999999999},"min":{"1min":15.08,"5min":13.196,"15min":13.196},"max":{"1min":34.466999999999999,"5min":124.473,"15min":124.473},"last":19.452999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476829,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":14.244999999999999,"5min":11.606999999999999,"15min":11.34},"min":{"1min":5.234,"5min":4.1639999999999997,"15min":4.1639999999999997},"max":{"1min":47.884999999999998,"5min":47.884999999999998,"15min":47.884999999999998},"last":10.805999999999999},{"interface":"front","average":{"1min":12.324999999999999,"5min":12.346,"15min":11.509},"min":{"1min":7.1399999999999997,"5min":4.3280000000000003,"15min":4.3280000000000003},"max":{"1min":48.305999999999997,"5min":48.305999999999997,"15min":48.305999999999997},"last":9.0329999999999995}]},{"osd":1,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":14.621,"5min":12.172000000000001,"15min":11.682},"min":{"1min":6.8860000000000001,"5min":5.9139999999999997,"15min":5.9139999999999997},"max":{"1min":28.242999999999999,"5min":42.920999999999999,"15min":42.920999999999999},"last":11.391999999999999},{"interface":"front","average":{"1min":17.202999999999999,"5min":14.548,"15min":14.061999999999999},"min":{"1min":7.8840000000000003,"5min":6.1479999999999997,"15min":6.1479999999999997},"max":{"1min":29.478999999999999,"5min":34.607999999999997,"15min":34.607999999999997},"last":13.411}]},{"osd":3,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":16.474,"5min":15.353,"15min":17.821999999999999},"min":{"1min":9.0139999999999993,"5min":7.2549999999999999,"15min":6.1260000000000003},"max":{"1min":37.463000000000001,"5min":49.609000000000002,"15min":112.129},"last":17.719000000000001},{"interface":"front","average":{"1min":21.956,"5min":17.753,"15min":21.963999999999999},"min":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"max":{"1min":36.770000000000003,"5min":61.243000000000002,"15min":181.601},"last":14.285}]},{"osd":4,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":19.707999999999998,"5min":16.417999999999999,"15min":17.349},"min":{"1min":8.5250000000000004,"5min":7.8070000000000004,"15min":6.609},"max":{"1min":35.396999999999998,"5min":60.332000000000001,"15min":97.088999999999999},"last":25.579999999999998},{"interface":"front","average":{"1min":13.917999999999999,"5min":13.611000000000001,"15min":18.606999999999999},"min":{"1min":7.7640000000000002,"5min":6.0720000000000001,"15min":6.0720000000000001},"max":{"1min":24.061,"5min":43.198,"15min":98.069000000000003},"last":18.262}]},{"osd":5,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":19.113,"5min":18.113,"15min":18.097000000000001},"min":{"1min":10.743,"5min":8.2100000000000009,"15min":7.8170000000000002},"max":{"1min":26.042000000000002,"5min":57.628999999999998,"15min":57.628999999999998},"last":17.033999999999999},{"interface":"front","average":{"1min":15.692,"5min":15.933,"15min":15.824},"min":{"1min":10.292,"5min":8.5990000000000002,"15min":8.5990000000000002},"max":{"1min":39.875999999999998,"5min":47.749000000000002,"15min":47.749000000000002},"last":20.780999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":20.952999999999999,"5min":19.140000000000001,"15min":36.003999999999998},"min":{"1min":7.0419999999999998,"5min":7.0419999999999998,"15min":7.0419999999999998},"max":{"1min":47.667999999999999,"5min":64.034999999999997,"15min":498.38499999999999},"last":21.663},{"interface":"front","average":{"1min":24.446000000000002,"5min":20.491,"15min":37.847000000000001},"min":{"1min":12.461,"5min":9.7010000000000005,"15min":9.7010000000000005},"max":{"1min":44.993000000000002,"5min":54.920000000000002,"15min":496.17399999999998},"last":20.18}]},{"osd":7,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":21.704999999999998,"5min":19.547000000000001,"15min":19.131},"min":{"1min":13.574999999999999,"5min":10.064,"15min":10.064},"max":{"1min":35.945,"5min":49.456000000000003,"15min":49.456000000000003},"last":24.835000000000001},{"interface":"front","average":{"1min":21.277999999999999,"5min":20.684000000000001,"15min":20.332000000000001},"min":{"1min":12.81,"5min":12.288,"15min":12.288},"max":{"1min":41.965000000000003,"5min":41.965000000000003,"15min":41.965000000000003},"last":26.071000000000002}]}]},{"osd":1,"up_from":16,"seq":68719476830,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":12.32,"5min":11.260999999999999,"15min":18.989000000000001},"min":{"1min":6.0650000000000004,"5min":5.0890000000000004,"15min":5.0890000000000004},"max":{"1min":34.816000000000003,"5min":34.816000000000003,"15min":238.87100000000001},"last":8.7870000000000008},{"interface":"front","average":{"1min":11.336,"5min":10.768000000000001,"15min":34.408000000000001},"min":{"1min":5.0629999999999997,"5min":5.0629999999999997,"15min":4.4269999999999996},"max":{"1min":22.834,"5min":61.262,"15min":661.33600000000001},"last":9.4359999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.962999999999999,"5min":13.722,"15min":27.556999999999999},"min":{"1min":6.6189999999999998,"5min":6.6189999999999998,"15min":6.6189999999999998},"max":{"1min":57.93,"5min":93.168000000000006,"15min":384.315},"last":12.364000000000001},{"interface":"front","average":{"1min":22.902000000000001,"5min":19.120999999999999,"15min":39.317999999999998},"min":{"1min":12.282,"5min":10.724,"15min":5.7009999999999996},"max":{"1min":60.173000000000002,"5min":94.230000000000004,"15min":656.98699999999997},"last":21.670000000000002}]},{"osd":3,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":12.403,"5min":12.194000000000001,"15min":12.863},"min":{"1min":6.9180000000000001,"5min":5.9720000000000004,"15min":5.548},"max":{"1min":25.774999999999999,"5min":31.16,"15min":34.057000000000002},"last":10.497},{"interface":"front","average":{"1min":14.6,"5min":14.064,"15min":15.500999999999999},"min":{"1min":8.0180000000000007,"5min":6.5730000000000004,"15min":6.5730000000000004},"max":{"1min":31.417999999999999,"5min":65.537000000000006,"15min":65.537000000000006},"last":13.625999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.082000000000001,"5min":13.308,"15min":33.085999999999999},"min":{"1min":8.6059999999999999,"5min":6.7009999999999996,"15min":6.2930000000000001},"max":{"1min":33.427999999999997,"5min":33.427999999999997,"15min":447.67200000000003},"last":17.629999999999999},{"interface":"front","average":{"1min":21.420000000000002,"5min":18.047999999999998,"15min":39.893000000000001},"min":{"1min":7.7300000000000004,"5min":7.7300000000000004,"15min":7.0919999999999996},"max":{"1min":40.564,"5min":44.043999999999997,"15min":630.572},"last":11.143000000000001}]},{"osd":5,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.390000000000001,"5min":16.076000000000001,"15min":35.482999999999997},"min":{"1min":7.2549999999999999,"5min":7.0540000000000003,"15min":7.0540000000000003},"max":{"1min":38.991999999999997,"5min":68.013999999999996,"15min":389.55000000000001},"last":12.994},{"interface":"front","average":{"1min":22.140999999999998,"5min":18.395,"15min":45.957999999999998},"min":{"1min":11.099,"5min":7.5359999999999996,"15min":7.0410000000000004},"max":{"1min":41.655999999999999,"5min":41.655999999999999,"15min":658.48599999999999},"last":20.786000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.672000000000001,"5min":15.733000000000001,"15min":30.202000000000002},"min":{"1min":8.9760000000000009,"5min":7.5899999999999999,"15min":7.5899999999999999},"max":{"1min":32.039999999999999,"5min":54.383000000000003,"15min":439.56400000000002},"last":14.907999999999999},{"interface":"front","average":{"1min":23.643000000000001,"5min":20.5,"15min":32.639000000000003},"min":{"1min":13.058,"5min":10.641999999999999,"15min":10.641999999999999},"max":{"1min":43.052,"5min":55.423999999999999,"15min":386.71499999999997},"last":22.509}]},{"osd":7,"last update":"Wed Dec 6 18:34:16 2023","interfaces":[{"interface":"back","average":{"1min":18.527000000000001,"5min":18.222999999999999,"15min":18.600000000000001},"min":{"1min":8.9849999999999994,"5min":8.9849999999999994,"15min":8.9849999999999994},"max":{"1min":33.156999999999996,"5min":97.561999999999998,"15min":97.561999999999998},"last":16.126000000000001},{"interface":"front","average":{"1min":23.125,"5min":21.024000000000001,"15min":18.771999999999998},"min":{"1min":12.019,"5min":8.7219999999999995,"15min":8.7219999999999995},"max":{"1min":36.332000000000001,"5min":101.762,"15min":101.762},"last":15.502000000000001}]}]},{"osd":0,"up_from":16,"seq":68719476831,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343220,"statfs":{"total":96636764160,"available":96607457280,"internally_reserved":0,"allocated":1634304,"data_stored":838986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":13.518000000000001,"5min":11.452,"15min":15.608000000000001},"min":{"1min":5.6340000000000003,"5min":3.2360000000000002,"15min":3.2360000000000002},"max":{"1min":54.155000000000001,"5min":54.155000000000001,"15min":116.593},"last":7.984},{"interface":"front","average":{"1min":13.859,"5min":12.935,"15min":16.576000000000001},"min":{"1min":5.5389999999999997,"5min":4.9039999999999999,"15min":4.9039999999999999},"max":{"1min":68.778999999999996,"5min":68.778999999999996,"15min":117.896},"last":11.999000000000001}]},{"osd":2,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":17.434000000000001,"5min":14.888,"15min":14.132999999999999},"min":{"1min":8.9870000000000001,"5min":5.694,"15min":5.694},"max":{"1min":72.131,"5min":72.131,"15min":72.131},"last":9.8000000000000007},{"interface":"front","average":{"1min":14.316000000000001,"5min":13.224,"15min":12.055},"min":{"1min":6.7590000000000003,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":59.317999999999998,"5min":59.317999999999998,"15min":59.317999999999998},"last":16.341000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":19.911999999999999,"5min":16.183,"15min":16.312999999999999},"min":{"1min":8.9600000000000009,"5min":6.1379999999999999,"15min":6.1379999999999999},"max":{"1min":75.439999999999998,"5min":75.439999999999998,"15min":75.439999999999998},"last":23.076000000000001},{"interface":"front","average":{"1min":19.082000000000001,"5min":16.047000000000001,"15min":16.204999999999998},"min":{"1min":7.593,"5min":7.5510000000000002,"15min":7.5510000000000002},"max":{"1min":96.688999999999993,"5min":96.688999999999993,"15min":96.688999999999993},"last":9.141}]},{"osd":4,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":14.302,"5min":12.134,"15min":17.890999999999998},"min":{"1min":6.8239999999999998,"5min":5.9349999999999996,"15min":5.9349999999999996},"max":{"1min":57.694000000000003,"5min":57.694000000000003,"15min":82.432000000000002},"last":20.741},{"interface":"front","average":{"1min":19.693000000000001,"5min":18.327000000000002,"15min":19.846},"min":{"1min":7.7469999999999999,"5min":7.7469999999999999,"15min":6.2629999999999999},"max":{"1min":76.078000000000003,"5min":76.078000000000003,"15min":81.239999999999995},"last":19.042000000000002}]},{"osd":5,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":19.27,"5min":16.798999999999999,"15min":17.507000000000001},"min":{"1min":8.1229999999999993,"5min":8.0099999999999998,"15min":8.0099999999999998},"max":{"1min":97.947000000000003,"5min":97.947000000000003,"15min":97.947000000000003},"last":24.241},{"interface":"front","average":{"1min":16.989000000000001,"5min":15.443,"15min":17.440000000000001},"min":{"1min":7.4770000000000003,"5min":7.1779999999999999,"15min":7.1779999999999999},"max":{"1min":69.915999999999997,"5min":69.915999999999997,"15min":72.307000000000002},"last":13.196999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":25.541,"5min":20.477,"15min":19.626000000000001},"min":{"1min":12.661,"5min":11.401,"15min":7.7560000000000002},"max":{"1min":82.775000000000006,"5min":82.775000000000006,"15min":82.775000000000006},"last":25.363},{"interface":"front","average":{"1min":21.75,"5min":17.940999999999999,"15min":18.914999999999999},"min":{"1min":9.7729999999999997,"5min":9.3520000000000003,"15min":9.3520000000000003},"max":{"1min":74.057000000000002,"5min":74.057000000000002,"15min":74.057000000000002},"last":21.91}]},{"osd":7,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":26.137,"5min":21.724,"15min":19.988},"min":{"1min":16.024000000000001,"5min":12.009,"15min":12.009},"max":{"1min":95.287999999999997,"5min":95.287999999999997,"15min":95.287999999999997},"last":26.545000000000002},{"interface":"front","average":{"1min":23.494,"5min":20.041,"15min":18.611000000000001},"min":{"1min":10.24,"5min":10.24,"15min":10.24},"max":{"1min":98.995999999999995,"5min":98.995999999999995,"15min":98.995999999999995},"last":27.462}]}]},{"osd":5,"up_from":15,"seq":64424509535,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28628,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343212,"statfs":{"total":96636764160,"available":96607449088,"internally_reserved":0,"allocated":1634304,"data_stored":838986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":14.44,"5min":9.6449999999999996,"15min":13.73},"min":{"1min":4.0430000000000001,"5min":3.3660000000000001,"15min":3.3660000000000001},"max":{"1min":106.521,"5min":106.521,"15min":208.291},"last":4.9500000000000002},{"interface":"front","average":{"1min":14.817,"5min":11.609,"15min":17.067},"min":{"1min":4.984,"5min":4.758,"15min":4.4299999999999997},"max":{"1min":107.342,"5min":107.342,"15min":199.94900000000001},"last":10.15}]},{"osd":1,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":11.631,"5min":13.298999999999999,"15min":45.225999999999999},"min":{"1min":6.7469999999999999,"5min":5.085,"15min":5.085},"max":{"1min":26.692,"5min":41.234999999999999,"15min":1211.6010000000001},"last":9.1539999999999999},{"interface":"front","average":{"1min":12.16,"5min":10.502000000000001,"15min":41.011000000000003},"min":{"1min":5.359,"5min":4.7199999999999998,"15min":4.7199999999999998},"max":{"1min":22.664000000000001,"5min":39.802999999999997,"15min":1189.3869999999999},"last":13.106999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":14.343,"5min":14.629,"15min":21.640999999999998},"min":{"1min":9.0389999999999997,"5min":6.5129999999999999,"15min":6.5129999999999999},"max":{"1min":23.422000000000001,"5min":68.617999999999995,"15min":283.46800000000002},"last":13.855},{"interface":"front","average":{"1min":19.576000000000001,"5min":18.463999999999999,"15min":25.539000000000001},"min":{"1min":12.851000000000001,"5min":8.173,"15min":8.173},"max":{"1min":40.737000000000002,"5min":69.513999999999996,"15min":296.47800000000001},"last":17.058}]},{"osd":3,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":11.484999999999999,"5min":13.811,"15min":17.414000000000001},"min":{"1min":7.3639999999999999,"5min":6.0300000000000002,"15min":6.0300000000000002},"max":{"1min":19.872,"5min":36.156999999999996,"15min":47.926000000000002},"last":10.849},{"interface":"front","average":{"1min":15.419,"5min":14.555999999999999,"15min":15.007999999999999},"min":{"1min":10.163,"5min":5.3979999999999997,"15min":5.3979999999999997},"max":{"1min":28.914999999999999,"5min":34.609000000000002,"15min":66.176000000000002},"last":15.17}]},{"osd":4,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":18.212,"5min":17.952000000000002,"15min":19.096},"min":{"1min":11.897,"5min":9.7210000000000001,"15min":9.7210000000000001},"max":{"1min":34.069000000000003,"5min":41.923999999999999,"15min":67.248000000000005},"last":15.784000000000001},{"interface":"front","average":{"1min":11.249000000000001,"5min":10.798,"15min":12.347},"min":{"1min":6.5279999999999996,"5min":5.3300000000000001,"15min":5.3300000000000001},"max":{"1min":26.850999999999999,"5min":43.101999999999997,"15min":77.266999999999996},"last":6.9100000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":15.667999999999999,"5min":13.644,"15min":15.140000000000001},"min":{"1min":7.984,"5min":6.1280000000000001,"15min":5.915},"max":{"1min":25.727,"5min":44.587000000000003,"15min":76.751999999999995},"last":11.791},{"interface":"front","average":{"1min":20.158999999999999,"5min":19.356999999999999,"15min":20.469000000000001},"min":{"1min":11.744999999999999,"5min":11.744999999999999,"15min":8.8390000000000004},"max":{"1min":36.883000000000003,"5min":45.366,"15min":57.101999999999997},"last":16.425999999999998}]},{"osd":7,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":18.687999999999999,"5min":16.294,"15min":14.904999999999999},"min":{"1min":11.298999999999999,"5min":7.5519999999999996,"15min":7.5519999999999996},"max":{"1min":32.219999999999999,"5min":32.219999999999999,"15min":32.219999999999999},"last":12.474},{"interface":"front","average":{"1min":14.401999999999999,"5min":12.872999999999999,"15min":12.215999999999999},"min":{"1min":7.133,"5min":7.133,"15min":7.133},"max":{"1min":29.376999999999999,"5min":29.376999999999999,"15min":29.376999999999999},"last":8.2439999999999998}]}]},{"osd":4,"up_from":15,"seq":64424509536,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1228800,"data_stored":440865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":8.0180000000000007,"5min":9.266,"15min":12.295999999999999},"min":{"1min":4.2789999999999999,"5min":3.7429999999999999,"15min":3.7429999999999999},"max":{"1min":13.465999999999999,"5min":41.076000000000001,"15min":53.298999999999999},"last":9.5990000000000002},{"interface":"front","average":{"1min":9.9160000000000004,"5min":10,"15min":10.609999999999999},"min":{"1min":5.1280000000000001,"5min":4.8140000000000001,"15min":3.9670000000000001},"max":{"1min":18.329000000000001,"5min":35.863,"15min":52.567},"last":7.3300000000000001}]},{"osd":1,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":12.202999999999999,"5min":11.432,"15min":13.298999999999999},"min":{"1min":6.0529999999999999,"5min":5.8390000000000004,"15min":5.835},"max":{"1min":21.186,"5min":22.509,"15min":49.183},"last":16.628},{"interface":"front","average":{"1min":13.43,"5min":12.129,"15min":14.662000000000001},"min":{"1min":5.9770000000000003,"5min":5.9770000000000003,"15min":5.96},"max":{"1min":23.596,"5min":24.081,"15min":45.412999999999997},"last":19.776}]},{"osd":2,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":12.398999999999999,"5min":12.423,"15min":19.385000000000002},"min":{"1min":6.609,"5min":5.9539999999999997,"15min":5.6390000000000002},"max":{"1min":22.904,"5min":44.649000000000001,"15min":85.394000000000005},"last":17.27},{"interface":"front","average":{"1min":14.196,"5min":15.087999999999999,"15min":18.899000000000001},"min":{"1min":7.9269999999999996,"5min":7.5209999999999999,"15min":6.3639999999999999},"max":{"1min":27.780000000000001,"5min":42.500999999999998,"15min":69.530000000000001},"last":8.5999999999999996}]},{"osd":3,"last update":"Wed Dec 6 18:34:17 2023","interfaces":[{"interface":"back","average":{"1min":16.875,"5min":15.66,"15min":14.851000000000001},"min":{"1min":8.6349999999999998,"5min":8.0250000000000004,"15min":7.008},"max":{"1min":35.418999999999997,"5min":37.561999999999998,"15min":37.561999999999998},"last":10.789},{"interface":"front","average":{"1min":18.140000000000001,"5min":14.212,"15min":14.936},"min":{"1min":8.0619999999999994,"5min":5.5549999999999997,"15min":5.5549999999999997},"max":{"1min":36.694000000000003,"5min":36.694000000000003,"15min":36.694000000000003},"last":18.012}]},{"osd":5,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":16.562000000000001,"5min":13.952999999999999,"15min":16.228999999999999},"min":{"1min":6.9370000000000003,"5min":5.4969999999999999,"15min":5.4969999999999999},"max":{"1min":27.283000000000001,"5min":53.840000000000003,"15min":78.798000000000002},"last":15.369},{"interface":"front","average":{"1min":16.690000000000001,"5min":17.414000000000001,"15min":18.402000000000001},"min":{"1min":10.425000000000001,"5min":8.0519999999999996,"15min":6.4450000000000003},"max":{"1min":29.184000000000001,"5min":52.781999999999996,"15min":55.814999999999998},"last":12.109}]},{"osd":6,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":19,"5min":18.105,"15min":18.045000000000002},"min":{"1min":9.3170000000000002,"5min":9.2100000000000009,"15min":7.7850000000000001},"max":{"1min":32.869,"5min":41.723999999999997,"15min":51.149000000000001},"last":11.468999999999999},{"interface":"front","average":{"1min":16.550000000000001,"5min":15.154999999999999,"15min":16.164999999999999},"min":{"1min":7.9359999999999999,"5min":7.9359999999999999,"15min":7.8700000000000001},"max":{"1min":29.792000000000002,"5min":34.929000000000002,"15min":40.426000000000002},"last":13.619999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:34:17 2023","interfaces":[{"interface":"back","average":{"1min":17.311,"5min":17.135000000000002,"15min":18.212},"min":{"1min":11.176,"5min":8.093,"15min":8.093},"max":{"1min":24.667000000000002,"5min":60.390000000000001,"15min":60.390000000000001},"last":15.988},{"interface":"front","average":{"1min":21.228999999999999,"5min":21.193999999999999,"15min":22.27},"min":{"1min":11.083,"5min":11.083,"15min":11.083},"max":{"1min":33.914999999999999,"5min":57.606999999999999,"15min":57.606999999999999},"last":12.869}]}]},{"osd":7,"up_from":61,"seq":261993005111,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33412,"kb_used_data":1204,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94338428,"statfs":{"total":96636764160,"available":96602550272,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4395,"internal_metadata":28634837},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.206,"5min":11.058999999999999,"15min":10.446999999999999},"min":{"1min":4.0739999999999998,"5min":4.0739999999999998,"15min":4.0739999999999998},"max":{"1min":126.286,"5min":126.286,"15min":126.286},"last":4.2300000000000004},{"interface":"front","average":{"1min":15.504,"5min":11.65,"15min":11.226000000000001},"min":{"1min":5.2320000000000002,"5min":4.2530000000000001,"15min":4.2530000000000001},"max":{"1min":127.35599999999999,"5min":127.35599999999999,"15min":127.35599999999999},"last":7.7160000000000002}]},{"osd":1,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":12.824999999999999,"5min":13.972,"15min":16.297000000000001},"min":{"1min":5.9130000000000003,"5min":4.6929999999999996,"15min":4.6929999999999996},"max":{"1min":24.047999999999998,"5min":64.765000000000001,"15min":64.765000000000001},"last":6.2510000000000003},{"interface":"front","average":{"1min":13.289999999999999,"5min":15.44,"15min":17.219999999999999},"min":{"1min":6.2969999999999997,"5min":6.2969999999999997,"15min":6.2969999999999997},"max":{"1min":24.402999999999999,"5min":65.994,"15min":65.994},"last":13.346}]},{"osd":2,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.021000000000001,"5min":16.204000000000001,"15min":17.411999999999999},"min":{"1min":7.9569999999999999,"5min":7.9569999999999999,"15min":7.9569999999999999},"max":{"1min":24.219000000000001,"5min":46.478000000000002,"15min":46.478000000000002},"last":13.962999999999999},{"interface":"front","average":{"1min":14.906000000000001,"5min":14.161,"15min":14.132999999999999},"min":{"1min":6.415,"5min":6.415,"15min":6.415},"max":{"1min":23.727,"5min":41.603999999999999,"15min":41.603999999999999},"last":11.414}]},{"osd":3,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":17.462,"5min":15.693,"15min":16.739000000000001},"min":{"1min":7.8879999999999999,"5min":6.8319999999999999,"15min":6.8319999999999999},"max":{"1min":47.823,"5min":60.463999999999999,"15min":60.463999999999999},"last":12.75},{"interface":"front","average":{"1min":20.838999999999999,"5min":18.907,"15min":19.867999999999999},"min":{"1min":9.8889999999999993,"5min":9.7370000000000001,"15min":9.7370000000000001},"max":{"1min":44.863,"5min":56.286000000000001,"15min":56.286000000000001},"last":17.911999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":20.128,"5min":20.585000000000001,"15min":21.081},"min":{"1min":10.489000000000001,"5min":8.4960000000000004,"15min":8.4960000000000004},"max":{"1min":32.167999999999999,"5min":57.716999999999999,"15min":57.716999999999999},"last":18.552},{"interface":"front","average":{"1min":16.364000000000001,"5min":17.821999999999999,"15min":18.356000000000002},"min":{"1min":7.2699999999999996,"5min":7.2699999999999996,"15min":7.2699999999999996},"max":{"1min":30.831,"5min":67.159000000000006,"15min":67.159000000000006},"last":15.808}]},{"osd":5,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.968999999999999,"5min":14.587999999999999,"15min":14.688000000000001},"min":{"1min":7.4219999999999997,"5min":6.3399999999999999,"15min":6.3399999999999999},"max":{"1min":27.07,"5min":44.884,"15min":44.884},"last":9.7699999999999996},{"interface":"front","average":{"1min":20.382999999999999,"5min":18.724,"15min":19.527999999999999},"min":{"1min":9.3520000000000003,"5min":9.3520000000000003,"15min":9.3520000000000003},"max":{"1min":92.616,"5min":92.616,"15min":92.616},"last":10.680999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":21.416,"5min":17.657,"15min":17.068999999999999},"min":{"1min":7.625,"5min":7.625,"15min":7.625},"max":{"1min":95.501000000000005,"5min":95.501000000000005,"15min":95.501000000000005},"last":19.923999999999999},{"interface":"front","average":{"1min":23.699999999999999,"5min":20.664999999999999,"15min":20.841000000000001},"min":{"1min":10.811999999999999,"5min":10.144,"15min":10.144},"max":{"1min":94.533000000000001,"5min":94.533000000000001,"15min":94.533000000000001},"last":16.48}]}]},{"osd":6,"up_from":14,"seq":60129542237,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":9.9879999999999995,"5min":9.4030000000000005,"15min":9.8219999999999992},"min":{"1min":4.3010000000000002,"5min":4.1260000000000003,"15min":3.8519999999999999},"max":{"1min":22.216000000000001,"5min":27.033000000000001,"15min":46.947000000000003},"last":3.8140000000000001},{"interface":"front","average":{"1min":11.436,"5min":10.529,"15min":10.420999999999999},"min":{"1min":4.2919999999999998,"5min":4.0650000000000004,"15min":4.0650000000000004},"max":{"1min":18.338999999999999,"5min":33.152000000000001,"15min":45.953000000000003},"last":9.923}]},{"osd":1,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":10.019,"5min":9.423,"15min":11.574999999999999},"min":{"1min":5.6189999999999998,"5min":4.3760000000000003,"15min":4.3760000000000003},"max":{"1min":18.969000000000001,"5min":66.313000000000002,"15min":66.313000000000002},"last":7.0270000000000001},{"interface":"front","average":{"1min":12.763999999999999,"5min":12.657,"15min":13.935},"min":{"1min":6.875,"5min":5.9349999999999996,"15min":5.8540000000000001},"max":{"1min":21.193000000000001,"5min":67.533000000000001,"15min":67.533000000000001},"last":14.852}]},{"osd":2,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.625999999999999,"5min":13.667,"15min":18.298999999999999},"min":{"1min":9.8510000000000009,"5min":6.0640000000000001,"15min":6.0640000000000001},"max":{"1min":41.106000000000002,"5min":46.051000000000002,"15min":200.88499999999999},"last":11.593999999999999},{"interface":"front","average":{"1min":20.809999999999999,"5min":17.507000000000001,"15min":16.818000000000001},"min":{"1min":12.57,"5min":7.8129999999999997,"15min":7.4119999999999999},"max":{"1min":39.863,"5min":45.003999999999998,"15min":48.005000000000003},"last":20.276}]},{"osd":3,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.510000000000002,"5min":14.548999999999999,"15min":14.074},"min":{"1min":9.1600000000000001,"5min":5.9820000000000002,"15min":5.9820000000000002},"max":{"1min":26.93,"5min":36.283999999999999,"15min":36.283999999999999},"last":9.1609999999999996},{"interface":"front","average":{"1min":19.988,"5min":16.667000000000002,"15min":16.466000000000001},"min":{"1min":11.983000000000001,"5min":7.75,"15min":7.6079999999999997},"max":{"1min":31.489999999999998,"5min":34.317,"15min":47.006999999999998},"last":12.333}]},{"osd":4,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.172999999999998,"5min":16.164000000000001,"15min":15.500999999999999},"min":{"1min":10.173999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"max":{"1min":27.928000000000001,"5min":49.170000000000002,"15min":49.170000000000002},"last":10.673},{"interface":"front","average":{"1min":19.071000000000002,"5min":17.736000000000001,"15min":17.052},"min":{"1min":12.409000000000001,"5min":8.9019999999999992,"15min":7.7169999999999996},"max":{"1min":32.399000000000001,"5min":38.573999999999998,"15min":48.988999999999997},"last":19.446999999999999}]},{"osd":5,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":20.074000000000002,"5min":18.266999999999999,"15min":18.411999999999999},"min":{"1min":13.186999999999999,"5min":7.8879999999999999,"15min":7.8879999999999999},"max":{"1min":30.518999999999998,"5min":49.390999999999998,"15min":57.308},"last":21.152000000000001},{"interface":"front","average":{"1min":11.132,"5min":10.885,"15min":13.561999999999999},"min":{"1min":5.5380000000000003,"5min":5.5380000000000003,"15min":5.5380000000000003},"max":{"1min":15.542999999999999,"5min":47.415999999999997,"15min":68.463999999999999},"last":13.837}]},{"osd":7,"last update":"Wed Dec 6 18:34:18 2023","interfaces":[{"interface":"back","average":{"1min":18.114000000000001,"5min":16.631,"15min":15.811},"min":{"1min":9.1460000000000008,"5min":7.4390000000000001,"15min":7.4390000000000001},"max":{"1min":26.856000000000002,"5min":38.887999999999998,"15min":38.887999999999998},"last":15.589},{"interface":"front","average":{"1min":22.518999999999998,"5min":20.096,"15min":19.210000000000001},"min":{"1min":16.774999999999999,"5min":13.555999999999999,"15min":13.555999999999999},"max":{"1min":34.822000000000003,"5min":34.822000000000003,"15min":34.822000000000003},"last":22.228000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:34:26.374 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2023-12-06T18:34:26.374 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2023-12-06T18:34:27.100 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-12-06T18:34:27.109 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-06T18:34:27.110 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:34:27.674 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:03:07.119 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:27.676 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:03:07.122 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:27.690 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:34:27.690 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:34:27.703 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2023-12-06T18:34:26.291967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231776,"kb_used_data":10412,"kb_used_omap":15,"kb_used_meta":217008,"kb_avail":754742944,"statfs":{"total":773094113280,"available":772856774656,"internally_reserved":0,"allocated":10661888,"data_stored":4324567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15557,"internal_metadata":222217019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001391"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.633082+0000","last_change":"2023-12-06T18:29:09.051293+0000","last_active":"2023-12-06T18:34:18.633082+0000","last_peered":"2023-12-06T18:34:18.633082+0000","last_clean":"2023-12-06T18:34:18.633082+0000","last_became_active":"2023-12-06T18:29:09.045331+0000","last_became_peered":"2023-12-06T18:29:09.045331+0000","last_unstale":"2023-12-06T18:34:18.633082+0000","last_undegraded":"2023-12-06T18:34:18.633082+0000","last_fullsized":"2023-12-06T18:34:18.633082+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T02:46:01.854508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":51,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:19.332340+0000","last_change":"2023-12-06T18:33:00.623558+0000","last_active":"2023-12-06T18:34:19.332340+0000","last_peered":"2023-12-06T18:34:19.332340+0000","last_clean":"2023-12-06T18:34:19.332340+0000","last_became_active":"2023-12-06T18:33:00.616353+0000","last_became_peered":"2023-12-06T18:33:00.616353+0000","last_unstale":"2023-12-06T18:34:19.332340+0000","last_undegraded":"2023-12-06T18:34:19.332340+0000","last_fullsized":"2023-12-06T18:34:19.332340+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T18:33:31.069235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.603990+0000","last_change":"2023-12-06T18:33:00.592340+0000","last_active":"2023-12-06T18:34:18.603990+0000","last_peered":"2023-12-06T18:34:18.603990+0000","last_clean":"2023-12-06T18:34:18.603990+0000","last_became_active":"2023-12-06T18:33:00.581037+0000","last_became_peered":"2023-12-06T18:33:00.581037+0000","last_unstale":"2023-12-06T18:34:18.603990+0000","last_undegraded":"2023-12-06T18:34:18.603990+0000","last_fullsized":"2023-12-06T18:34:18.603990+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T06:13:54.498147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":118,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.631290+0000","last_change":"2023-12-06T18:33:52.522397+0000","last_active":"2023-12-06T18:34:18.631290+0000","last_peered":"2023-12-06T18:34:18.631290+0000","last_clean":"2023-12-06T18:34:18.631290+0000","last_became_active":"2023-12-06T18:33:52.516242+0000","last_became_peered":"2023-12-06T18:33:52.516242+0000","last_unstale":"2023-12-06T18:34:18.631290+0000","last_undegraded":"2023-12-06T18:34:18.631290+0000","last_fullsized":"2023-12-06T18:34:18.631290+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T03:19:25.211436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.599973+0000","last_change":"2023-12-06T18:33:52.521292+0000","last_active":"2023-12-06T18:34:18.599973+0000","last_peered":"2023-12-06T18:34:18.599973+0000","last_clean":"2023-12-06T18:34:18.599973+0000","last_became_active":"2023-12-06T18:33:52.514832+0000","last_became_peered":"2023-12-06T18:33:52.514832+0000","last_unstale":"2023-12-06T18:34:18.599973+0000","last_undegraded":"2023-12-06T18:34:18.599973+0000","last_fullsized":"2023-12-06T18:34:18.599973+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T23:51:07.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":94,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-12-06T18:34:19.744131+0000","last_change":"2023-12-06T18:34:19.744131+0000","last_active":"2023-12-06T18:34:19.744131+0000","last_peered":"2023-12-06T18:34:19.744131+0000","last_clean":"2023-12-06T18:34:19.744131+0000","last_became_active":"2023-12-06T18:34:19.738308+0000","last_became_peered":"2023-12-06T18:34:19.738308+0000","last_unstale":"2023-12-06T18:34:19.744131+0000","last_undegraded":"2023-12-06T18:34:19.744131+0000","last_fullsized":"2023-12-06T18:34:19.744131+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T01:32:29.556210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.597827+0000","last_change":"2023-12-06T18:28:47.876556+0000","last_active":"2023-12-06T18:34:18.597827+0000","last_peered":"2023-12-06T18:34:18.597827+0000","last_clean":"2023-12-06T18:34:18.597827+0000","last_became_active":"2023-12-06T18:28:47.865169+0000","last_became_peered":"2023-12-06T18:28:47.865169+0000","last_unstale":"2023-12-06T18:34:18.597827+0000","last_undegraded":"2023-12-06T18:34:18.597827+0000","last_fullsized":"2023-12-06T18:34:18.597827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:16:40.095542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":72,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.991987+0000","last_change":"2023-12-06T18:31:57.187068+0000","last_active":"2023-12-06T18:34:18.991987+0000","last_peered":"2023-12-06T18:34:18.991987+0000","last_clean":"2023-12-06T18:34:18.991987+0000","last_became_active":"2023-12-06T18:31:57.175444+0000","last_became_peered":"2023-12-06T18:31:57.175444+0000","last_unstale":"2023-12-06T18:34:18.991987+0000","last_undegraded":"2023-12-06T18:34:18.991987+0000","last_fullsized":"2023-12-06T18:34:18.991987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:25:09.547812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":49,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:19.324726+0000","last_change":"2023-12-06T18:33:22.203137+0000","last_active":"2023-12-06T18:34:19.324726+0000","last_peered":"2023-12-06T18:34:19.324726+0000","last_clean":"2023-12-06T18:34:19.324726+0000","last_became_active":"2023-12-06T18:33:21.485790+0000","last_became_peered":"2023-12-06T18:33:21.485790+0000","last_unstale":"2023-12-06T18:34:19.324726+0000","last_undegraded":"2023-12-06T18:34:19.324726+0000","last_fullsized":"2023-12-06T18:34:19.324726+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T01:25:32.645799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1605632,"data_stored":1591360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411421,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":11.292,"5min":11.291,"15min":10.609999999999999},"min":{"1min":5.069,"5min":3.835,"15min":3.835},"max":{"1min":27.620000000000001,"5min":108.57599999999999,"15min":108.57599999999999},"last":4.641},{"interface":"front","average":{"1min":11.958,"5min":11.699,"15min":12.119999999999999},"min":{"1min":6.3239999999999998,"5min":4.8639999999999999,"15min":3.7200000000000002},"max":{"1min":24.844000000000001,"5min":128.09399999999999,"15min":128.09399999999999},"last":10.798999999999999}]},{"osd":1,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":11.891999999999999,"5min":12.403,"15min":12.693},"min":{"1min":6.5359999999999996,"5min":6.2539999999999996,"15min":6.2539999999999996},"max":{"1min":19.538,"5min":70.072000000000003,"15min":70.072000000000003},"last":12.27},{"interface":"front","average":{"1min":12.019,"5min":12.824,"15min":14.788},"min":{"1min":6.2080000000000002,"5min":6.2080000000000002,"15min":6.2080000000000002},"max":{"1min":30.16,"5min":30.16,"15min":32.920000000000002},"last":11.516999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":15.49,"5min":15.119,"15min":14.845000000000001},"min":{"1min":9.4450000000000003,"5min":8.5079999999999991,"15min":7.5149999999999997},"max":{"1min":52.232999999999997,"5min":75.125,"15min":75.125},"last":21.791},{"interface":"front","average":{"1min":18.545000000000002,"5min":16.343,"15min":17.477},"min":{"1min":9.2599999999999998,"5min":7.5389999999999997,"15min":7.5389999999999997},"max":{"1min":54.323999999999998,"5min":69.403999999999996,"15min":69.403999999999996},"last":9.4269999999999996}]},{"osd":4,"last update":"Wed Dec 6 18:34:13 2023","interfaces":[{"interface":"back","average":{"1min":18.949999999999999,"5min":15.185,"15min":12.898999999999999},"min":{"1min":10.714,"5min":5.8879999999999999,"15min":5.8879999999999999},"max":{"1min":56.255000000000003,"5min":56.255000000000003,"15min":56.255000000000003},"last":15.881},{"interface":"front","average":{"1min":14.9,"5min":13.204000000000001,"15min":15.038},"min":{"1min":6.4189999999999996,"5min":6.3099999999999996,"15min":6.3099999999999996},"max":{"1min":54.616,"5min":71.718000000000004,"15min":71.718000000000004},"last":18.234999999999999}]},{"osd":5,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":17.082000000000001,"5min":18.359999999999999,"15min":18.027999999999999},"min":{"1min":11.888,"5min":10.417999999999999,"15min":8.3179999999999996},"max":{"1min":24.445,"5min":75.751999999999995,"15min":75.751999999999995},"last":23.423999999999999},{"interface":"front","average":{"1min":16.565999999999999,"5min":16.934000000000001,"15min":17.869},"min":{"1min":8.5389999999999997,"5min":7.9269999999999996,"15min":7.9269999999999996},"max":{"1min":32.219000000000001,"5min":70.933000000000007,"15min":70.933000000000007},"last":17.038}]},{"osd":6,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":19.771000000000001,"5min":15.018000000000001,"15min":15.869999999999999},"min":{"1min":10.798,"5min":8.2050000000000001,"15min":7.0919999999999996},"max":{"1min":30.991,"5min":30.991,"15min":30.991},"last":13.795},{"interface":"front","average":{"1min":16.899999999999999,"5min":18.946999999999999,"15min":20.498000000000001},"min":{"1min":7.2210000000000001,"5min":7.2210000000000001,"15min":7.2210000000000001},"max":{"1min":30.404,"5min":74.224000000000004,"15min":74.224000000000004},"last":14.584}]},{"osd":7,"last update":"Wed Dec 6 18:34:13 2023","interfaces":[{"interface":"back","average":{"1min":20.172999999999998,"5min":19.478999999999999,"15min":21.131},"min":{"1min":12.255000000000001,"5min":9.1859999999999999,"15min":9.1859999999999999},"max":{"1min":30.175000000000001,"5min":122.93000000000001,"15min":122.93000000000001},"last":20.257999999999999},{"interface":"front","average":{"1min":22.155000000000001,"5min":22.471,"15min":23.312999999999999},"min":{"1min":15.08,"5min":13.196,"15min":13.196},"max":{"1min":34.466999999999999,"5min":124.473,"15min":124.473},"last":19.452999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476829,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":14.244999999999999,"5min":11.606999999999999,"15min":11.34},"min":{"1min":5.234,"5min":4.1639999999999997,"15min":4.1639999999999997},"max":{"1min":47.884999999999998,"5min":47.884999999999998,"15min":47.884999999999998},"last":10.805999999999999},{"interface":"front","average":{"1min":12.324999999999999,"5min":12.346,"15min":11.509},"min":{"1min":7.1399999999999997,"5min":4.3280000000000003,"15min":4.3280000000000003},"max":{"1min":48.305999999999997,"5min":48.305999999999997,"15min":48.305999999999997},"last":9.0329999999999995}]},{"osd":1,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":14.621,"5min":12.172000000000001,"15min":11.682},"min":{"1min":6.8860000000000001,"5min":5.9139999999999997,"15min":5.9139999999999997},"max":{"1min":28.242999999999999,"5min":42.920999999999999,"15min":42.920999999999999},"last":11.391999999999999},{"interface":"front","average":{"1min":17.202999999999999,"5min":14.548,"15min":14.061999999999999},"min":{"1min":7.8840000000000003,"5min":6.1479999999999997,"15min":6.1479999999999997},"max":{"1min":29.478999999999999,"5min":34.607999999999997,"15min":34.607999999999997},"last":13.411}]},{"osd":3,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":16.474,"5min":15.353,"15min":17.821999999999999},"min":{"1min":9.0139999999999993,"5min":7.2549999999999999,"15min":6.1260000000000003},"max":{"1min":37.463000000000001,"5min":49.609000000000002,"15min":112.129},"last":17.719000000000001},{"interface":"front","average":{"1min":21.956,"5min":17.753,"15min":21.963999999999999},"min":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"max":{"1min":36.770000000000003,"5min":61.243000000000002,"15min":181.601},"last":14.285}]},{"osd":4,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":19.707999999999998,"5min":16.417999999999999,"15min":17.349},"min":{"1min":8.5250000000000004,"5min":7.8070000000000004,"15min":6.609},"max":{"1min":35.396999999999998,"5min":60.332000000000001,"15min":97.088999999999999},"last":25.579999999999998},{"interface":"front","average":{"1min":13.917999999999999,"5min":13.611000000000001,"15min":18.606999999999999},"min":{"1min":7.7640000000000002,"5min":6.0720000000000001,"15min":6.0720000000000001},"max":{"1min":24.061,"5min":43.198,"15min":98.069000000000003},"last":18.262}]},{"osd":5,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":19.113,"5min":18.113,"15min":18.097000000000001},"min":{"1min":10.743,"5min":8.2100000000000009,"15min":7.8170000000000002},"max":{"1min":26.042000000000002,"5min":57.628999999999998,"15min":57.628999999999998},"last":17.033999999999999},{"interface":"front","average":{"1min":15.692,"5min":15.933,"15min":15.824},"min":{"1min":10.292,"5min":8.5990000000000002,"15min":8.5990000000000002},"max":{"1min":39.875999999999998,"5min":47.749000000000002,"15min":47.749000000000002},"last":20.780999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":20.952999999999999,"5min":19.140000000000001,"15min":36.003999999999998},"min":{"1min":7.0419999999999998,"5min":7.0419999999999998,"15min":7.0419999999999998},"max":{"1min":47.667999999999999,"5min":64.034999999999997,"15min":498.38499999999999},"last":21.663},{"interface":"front","average":{"1min":24.446000000000002,"5min":20.491,"15min":37.847000000000001},"min":{"1min":12.461,"5min":9.7010000000000005,"15min":9.7010000000000005},"max":{"1min":44.993000000000002,"5min":54.920000000000002,"15min":496.17399999999998},"last":20.18}]},{"osd":7,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":21.704999999999998,"5min":19.547000000000001,"15min":19.131},"min":{"1min":13.574999999999999,"5min":10.064,"15min":10.064},"max":{"1min":35.945,"5min":49.456000000000003,"15min":49.456000000000003},"last":24.835000000000001},{"interface":"front","average":{"1min":21.277999999999999,"5min":20.684000000000001,"15min":20.332000000000001},"min":{"1min":12.81,"5min":12.288,"15min":12.288},"max":{"1min":41.965000000000003,"5min":41.965000000000003,"15min":41.965000000000003},"last":26.071000000000002}]}]},{"osd":1,"up_from":16,"seq":68719476830,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":12.32,"5min":11.260999999999999,"15min":18.989000000000001},"min":{"1min":6.0650000000000004,"5min":5.0890000000000004,"15min":5.0890000000000004},"max":{"1min":34.816000000000003,"5min":34.816000000000003,"15min":238.87100000000001},"last":8.7870000000000008},{"interface":"front","average":{"1min":11.336,"5min":10.768000000000001,"15min":34.408000000000001},"min":{"1min":5.0629999999999997,"5min":5.0629999999999997,"15min":4.4269999999999996},"max":{"1min":22.834,"5min":61.262,"15min":661.33600000000001},"last":9.4359999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.962999999999999,"5min":13.722,"15min":27.556999999999999},"min":{"1min":6.6189999999999998,"5min":6.6189999999999998,"15min":6.6189999999999998},"max":{"1min":57.93,"5min":93.168000000000006,"15min":384.315},"last":12.364000000000001},{"interface":"front","average":{"1min":22.902000000000001,"5min":19.120999999999999,"15min":39.317999999999998},"min":{"1min":12.282,"5min":10.724,"15min":5.7009999999999996},"max":{"1min":60.173000000000002,"5min":94.230000000000004,"15min":656.98699999999997},"last":21.670000000000002}]},{"osd":3,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":12.403,"5min":12.194000000000001,"15min":12.863},"min":{"1min":6.9180000000000001,"5min":5.9720000000000004,"15min":5.548},"max":{"1min":25.774999999999999,"5min":31.16,"15min":34.057000000000002},"last":10.497},{"interface":"front","average":{"1min":14.6,"5min":14.064,"15min":15.500999999999999},"min":{"1min":8.0180000000000007,"5min":6.5730000000000004,"15min":6.5730000000000004},"max":{"1min":31.417999999999999,"5min":65.537000000000006,"15min":65.537000000000006},"last":13.625999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.082000000000001,"5min":13.308,"15min":33.085999999999999},"min":{"1min":8.6059999999999999,"5min":6.7009999999999996,"15min":6.2930000000000001},"max":{"1min":33.427999999999997,"5min":33.427999999999997,"15min":447.67200000000003},"last":17.629999999999999},{"interface":"front","average":{"1min":21.420000000000002,"5min":18.047999999999998,"15min":39.893000000000001},"min":{"1min":7.7300000000000004,"5min":7.7300000000000004,"15min":7.0919999999999996},"max":{"1min":40.564,"5min":44.043999999999997,"15min":630.572},"last":11.143000000000001}]},{"osd":5,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.390000000000001,"5min":16.076000000000001,"15min":35.482999999999997},"min":{"1min":7.2549999999999999,"5min":7.0540000000000003,"15min":7.0540000000000003},"max":{"1min":38.991999999999997,"5min":68.013999999999996,"15min":389.55000000000001},"last":12.994},{"interface":"front","average":{"1min":22.140999999999998,"5min":18.395,"15min":45.957999999999998},"min":{"1min":11.099,"5min":7.5359999999999996,"15min":7.0410000000000004},"max":{"1min":41.655999999999999,"5min":41.655999999999999,"15min":658.48599999999999},"last":20.786000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.672000000000001,"5min":15.733000000000001,"15min":30.202000000000002},"min":{"1min":8.9760000000000009,"5min":7.5899999999999999,"15min":7.5899999999999999},"max":{"1min":32.039999999999999,"5min":54.383000000000003,"15min":439.56400000000002},"last":14.907999999999999},{"interface":"front","average":{"1min":23.643000000000001,"5min":20.5,"15min":32.639000000000003},"min":{"1min":13.058,"5min":10.641999999999999,"15min":10.641999999999999},"max":{"1min":43.052,"5min":55.423999999999999,"15min":386.71499999999997},"last":22.509}]},{"osd":7,"last update":"Wed Dec 6 18:34:16 2023","interfaces":[{"interface":"back","average":{"1min":18.527000000000001,"5min":18.222999999999999,"15min":18.600000000000001},"min":{"1min":8.9849999999999994,"5min":8.9849999999999994,"15min":8.9849999999999994},"max":{"1min":33.156999999999996,"5min":97.561999999999998,"15min":97.561999999999998},"last":16.126000000000001},{"interface":"front","average":{"1min":23.125,"5min":21.024000000000001,"15min":18.771999999999998},"min":{"1min":12.019,"5min":8.7219999999999995,"15min":8.7219999999999995},"max":{"1min":36.332000000000001,"5min":101.762,"15min":101.762},"last":15.502000000000001}]}]},{"osd":0,"up_from":16,"seq":68719476831,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343220,"statfs":{"total":96636764160,"available":96607457280,"internally_reserved":0,"allocated":1634304,"data_stored":838986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":13.518000000000001,"5min":11.452,"15min":15.608000000000001},"min":{"1min":5.6340000000000003,"5min":3.2360000000000002,"15min":3.2360000000000002},"max":{"1min":54.155000000000001,"5min":54.155000000000001,"15min":116.593},"last":7.984},{"interface":"front","average":{"1min":13.859,"5min":12.935,"15min":16.576000000000001},"min":{"1min":5.5389999999999997,"5min":4.9039999999999999,"15min":4.9039999999999999},"max":{"1min":68.778999999999996,"5min":68.778999999999996,"15min":117.896},"last":11.999000000000001}]},{"osd":2,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":17.434000000000001,"5min":14.888,"15min":14.132999999999999},"min":{"1min":8.9870000000000001,"5min":5.694,"15min":5.694},"max":{"1min":72.131,"5min":72.131,"15min":72.131},"last":9.8000000000000007},{"interface":"front","average":{"1min":14.316000000000001,"5min":13.224,"15min":12.055},"min":{"1min":6.7590000000000003,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":59.317999999999998,"5min":59.317999999999998,"15min":59.317999999999998},"last":16.341000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":19.911999999999999,"5min":16.183,"15min":16.312999999999999},"min":{"1min":8.9600000000000009,"5min":6.1379999999999999,"15min":6.1379999999999999},"max":{"1min":75.439999999999998,"5min":75.439999999999998,"15min":75.439999999999998},"last":23.076000000000001},{"interface":"front","average":{"1min":19.082000000000001,"5min":16.047000000000001,"15min":16.204999999999998},"min":{"1min":7.593,"5min":7.5510000000000002,"15min":7.5510000000000002},"max":{"1min":96.688999999999993,"5min":96.688999999999993,"15min":96.688999999999993},"last":9.141}]},{"osd":4,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":14.302,"5min":12.134,"15min":17.890999999999998},"min":{"1min":6.8239999999999998,"5min":5.9349999999999996,"15min":5.9349999999999996},"max":{"1min":57.694000000000003,"5min":57.694000000000003,"15min":82.432000000000002},"last":20.741},{"interface":"front","average":{"1min":19.693000000000001,"5min":18.327000000000002,"15min":19.846},"min":{"1min":7.7469999999999999,"5min":7.7469999999999999,"15min":6.2629999999999999},"max":{"1min":76.078000000000003,"5min":76.078000000000003,"15min":81.239999999999995},"last":19.042000000000002}]},{"osd":5,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":19.27,"5min":16.798999999999999,"15min":17.507000000000001},"min":{"1min":8.1229999999999993,"5min":8.0099999999999998,"15min":8.0099999999999998},"max":{"1min":97.947000000000003,"5min":97.947000000000003,"15min":97.947000000000003},"last":24.241},{"interface":"front","average":{"1min":16.989000000000001,"5min":15.443,"15min":17.440000000000001},"min":{"1min":7.4770000000000003,"5min":7.1779999999999999,"15min":7.1779999999999999},"max":{"1min":69.915999999999997,"5min":69.915999999999997,"15min":72.307000000000002},"last":13.196999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":25.541,"5min":20.477,"15min":19.626000000000001},"min":{"1min":12.661,"5min":11.401,"15min":7.7560000000000002},"max":{"1min":82.775000000000006,"5min":82.775000000000006,"15min":82.775000000000006},"last":25.363},{"interface":"front","average":{"1min":21.75,"5min":17.940999999999999,"15min":18.914999999999999},"min":{"1min":9.7729999999999997,"5min":9.3520000000000003,"15min":9.3520000000000003},"max":{"1min":74.057000000000002,"5min":74.057000000000002,"15min":74.057000000000002},"last":21.91}]},{"osd":7,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":26.137,"5min":21.724,"15min":19.988},"min":{"1min":16.024000000000001,"5min":12.009,"15min":12.009},"max":{"1min":95.287999999999997,"5min":95.287999999999997,"15min":95.287999999999997},"last":26.545000000000002},{"interface":"front","average":{"1min":23.494,"5min":20.041,"15min":18.611000000000001},"min":{"1min":10.24,"5min":10.24,"15min":10.24},"max":{"1min":98.995999999999995,"5min":98.995999999999995,"15min":98.995999999999995},"last":27.462}]}]},{"osd":5,"up_from":15,"seq":64424509535,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28628,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343212,"statfs":{"total":96636764160,"available":96607449088,"internally_reserved":0,"allocated":1634304,"data_stored":838986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":14.44,"5min":9.6449999999999996,"15min":13.73},"min":{"1min":4.0430000000000001,"5min":3.3660000000000001,"15min":3.3660000000000001},"max":{"1min":106.521,"5min":106.521,"15min":208.291},"last":4.9500000000000002},{"interface":"front","average":{"1min":14.817,"5min":11.609,"15min":17.067},"min":{"1min":4.984,"5min":4.758,"15min":4.4299999999999997},"max":{"1min":107.342,"5min":107.342,"15min":199.94900000000001},"last":10.15}]},{"osd":1,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":11.631,"5min":13.298999999999999,"15min":45.225999999999999},"min":{"1min":6.7469999999999999,"5min":5.085,"15min":5.085},"max":{"1min":26.692,"5min":41.234999999999999,"15min":1211.6010000000001},"last":9.1539999999999999},{"interface":"front","average":{"1min":12.16,"5min":10.502000000000001,"15min":41.011000000000003},"min":{"1min":5.359,"5min":4.7199999999999998,"15min":4.7199999999999998},"max":{"1min":22.664000000000001,"5min":39.802999999999997,"15min":1189.3869999999999},"last":13.106999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":14.343,"5min":14.629,"15min":21.640999999999998},"min":{"1min":9.0389999999999997,"5min":6.5129999999999999,"15min":6.5129999999999999},"max":{"1min":23.422000000000001,"5min":68.617999999999995,"15min":283.46800000000002},"last":13.855},{"interface":"front","average":{"1min":19.576000000000001,"5min":18.463999999999999,"15min":25.539000000000001},"min":{"1min":12.851000000000001,"5min":8.173,"15min":8.173},"max":{"1min":40.737000000000002,"5min":69.513999999999996,"15min":296.47800000000001},"last":17.058}]},{"osd":3,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":11.484999999999999,"5min":13.811,"15min":17.414000000000001},"min":{"1min":7.3639999999999999,"5min":6.0300000000000002,"15min":6.0300000000000002},"max":{"1min":19.872,"5min":36.156999999999996,"15min":47.926000000000002},"last":10.849},{"interface":"front","average":{"1min":15.419,"5min":14.555999999999999,"15min":15.007999999999999},"min":{"1min":10.163,"5min":5.3979999999999997,"15min":5.3979999999999997},"max":{"1min":28.914999999999999,"5min":34.609000000000002,"15min":66.176000000000002},"last":15.17}]},{"osd":4,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":18.212,"5min":17.952000000000002,"15min":19.096},"min":{"1min":11.897,"5min":9.7210000000000001,"15min":9.7210000000000001},"max":{"1min":34.069000000000003,"5min":41.923999999999999,"15min":67.248000000000005},"last":15.784000000000001},{"interface":"front","average":{"1min":11.249000000000001,"5min":10.798,"15min":12.347},"min":{"1min":6.5279999999999996,"5min":5.3300000000000001,"15min":5.3300000000000001},"max":{"1min":26.850999999999999,"5min":43.101999999999997,"15min":77.266999999999996},"last":6.9100000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":15.667999999999999,"5min":13.644,"15min":15.140000000000001},"min":{"1min":7.984,"5min":6.1280000000000001,"15min":5.915},"max":{"1min":25.727,"5min":44.587000000000003,"15min":76.751999999999995},"last":11.791},{"interface":"front","average":{"1min":20.158999999999999,"5min":19.356999999999999,"15min":20.469000000000001},"min":{"1min":11.744999999999999,"5min":11.744999999999999,"15min":8.8390000000000004},"max":{"1min":36.883000000000003,"5min":45.366,"15min":57.101999999999997},"last":16.425999999999998}]},{"osd":7,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":18.687999999999999,"5min":16.294,"15min":14.904999999999999},"min":{"1min":11.298999999999999,"5min":7.5519999999999996,"15min":7.5519999999999996},"max":{"1min":32.219999999999999,"5min":32.219999999999999,"15min":32.219999999999999},"last":12.474},{"interface":"front","average":{"1min":14.401999999999999,"5min":12.872999999999999,"15min":12.215999999999999},"min":{"1min":7.133,"5min":7.133,"15min":7.133},"max":{"1min":29.376999999999999,"5min":29.376999999999999,"15min":29.376999999999999},"last":8.2439999999999998}]}]},{"osd":4,"up_from":15,"seq":64424509536,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1228800,"data_stored":440865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":8.0180000000000007,"5min":9.266,"15min":12.295999999999999},"min":{"1min":4.2789999999999999,"5min":3.7429999999999999,"15min":3.7429999999999999},"max":{"1min":13.465999999999999,"5min":41.076000000000001,"15min":53.298999999999999},"last":9.5990000000000002},{"interface":"front","average":{"1min":9.9160000000000004,"5min":10,"15min":10.609999999999999},"min":{"1min":5.1280000000000001,"5min":4.8140000000000001,"15min":3.9670000000000001},"max":{"1min":18.329000000000001,"5min":35.863,"15min":52.567},"last":7.3300000000000001}]},{"osd":1,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":12.202999999999999,"5min":11.432,"15min":13.298999999999999},"min":{"1min":6.0529999999999999,"5min":5.8390000000000004,"15min":5.835},"max":{"1min":21.186,"5min":22.509,"15min":49.183},"last":16.628},{"interface":"front","average":{"1min":13.43,"5min":12.129,"15min":14.662000000000001},"min":{"1min":5.9770000000000003,"5min":5.9770000000000003,"15min":5.96},"max":{"1min":23.596,"5min":24.081,"15min":45.412999999999997},"last":19.776}]},{"osd":2,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":12.398999999999999,"5min":12.423,"15min":19.385000000000002},"min":{"1min":6.609,"5min":5.9539999999999997,"15min":5.6390000000000002},"max":{"1min":22.904,"5min":44.649000000000001,"15min":85.394000000000005},"last":17.27},{"interface":"front","average":{"1min":14.196,"5min":15.087999999999999,"15min":18.899000000000001},"min":{"1min":7.9269999999999996,"5min":7.5209999999999999,"15min":6.3639999999999999},"max":{"1min":27.780000000000001,"5min":42.500999999999998,"15min":69.530000000000001},"last":8.5999999999999996}]},{"osd":3,"last update":"Wed Dec 6 18:34:17 2023","interfaces":[{"interface":"back","average":{"1min":16.875,"5min":15.66,"15min":14.851000000000001},"min":{"1min":8.6349999999999998,"5min":8.0250000000000004,"15min":7.008},"max":{"1min":35.418999999999997,"5min":37.561999999999998,"15min":37.561999999999998},"last":10.789},{"interface":"front","average":{"1min":18.140000000000001,"5min":14.212,"15min":14.936},"min":{"1min":8.0619999999999994,"5min":5.5549999999999997,"15min":5.5549999999999997},"max":{"1min":36.694000000000003,"5min":36.694000000000003,"15min":36.694000000000003},"last":18.012}]},{"osd":5,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":16.562000000000001,"5min":13.952999999999999,"15min":16.228999999999999},"min":{"1min":6.9370000000000003,"5min":5.4969999999999999,"15min":5.4969999999999999},"max":{"1min":27.283000000000001,"5min":53.840000000000003,"15min":78.798000000000002},"last":15.369},{"interface":"front","average":{"1min":16.690000000000001,"5min":17.414000000000001,"15min":18.402000000000001},"min":{"1min":10.425000000000001,"5min":8.0519999999999996,"15min":6.4450000000000003},"max":{"1min":29.184000000000001,"5min":52.781999999999996,"15min":55.814999999999998},"last":12.109}]},{"osd":6,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":19,"5min":18.105,"15min":18.045000000000002},"min":{"1min":9.3170000000000002,"5min":9.2100000000000009,"15min":7.7850000000000001},"max":{"1min":32.869,"5min":41.723999999999997,"15min":51.149000000000001},"last":11.468999999999999},{"interface":"front","average":{"1min":16.550000000000001,"5min":15.154999999999999,"15min":16.164999999999999},"min":{"1min":7.9359999999999999,"5min":7.9359999999999999,"15min":7.8700000000000001},"max":{"1min":29.792000000000002,"5min":34.929000000000002,"15min":40.426000000000002},"last":13.619999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:34:17 2023","interfaces":[{"interface":"back","average":{"1min":17.311,"5min":17.135000000000002,"15min":18.212},"min":{"1min":11.176,"5min":8.093,"15min":8.093},"max":{"1min":24.667000000000002,"5min":60.390000000000001,"15min":60.390000000000001},"last":15.988},{"interface":"front","average":{"1min":21.228999999999999,"5min":21.193999999999999,"15min":22.27},"min":{"1min":11.083,"5min":11.083,"15min":11.083},"max":{"1min":33.914999999999999,"5min":57.606999999999999,"15min":57.606999999999999},"last":12.869}]}]},{"osd":7,"up_from":61,"seq":261993005111,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33412,"kb_used_data":1204,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94338428,"statfs":{"total":96636764160,"available":96602550272,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4395,"internal_metadata":28634837},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.206,"5min":11.058999999999999,"15min":10.446999999999999},"min":{"1min":4.0739999999999998,"5min":4.0739999999999998,"15min":4.0739999999999998},"max":{"1min":126.286,"5min":126.286,"15min":126.286},"last":4.2300000000000004},{"interface":"front","average":{"1min":15.504,"5min":11.65,"15min":11.226000000000001},"min":{"1min":5.2320000000000002,"5min":4.2530000000000001,"15min":4.2530000000000001},"max":{"1min":127.35599999999999,"5min":127.35599999999999,"15min":127.35599999999999},"last":7.7160000000000002}]},{"osd":1,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":12.824999999999999,"5min":13.972,"15min":16.297000000000001},"min":{"1min":5.9130000000000003,"5min":4.6929999999999996,"15min":4.6929999999999996},"max":{"1min":24.047999999999998,"5min":64.765000000000001,"15min":64.765000000000001},"last":6.2510000000000003},{"interface":"front","average":{"1min":13.289999999999999,"5min":15.44,"15min":17.219999999999999},"min":{"1min":6.2969999999999997,"5min":6.2969999999999997,"15min":6.2969999999999997},"max":{"1min":24.402999999999999,"5min":65.994,"15min":65.994},"last":13.346}]},{"osd":2,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.021000000000001,"5min":16.204000000000001,"15min":17.411999999999999},"min":{"1min":7.9569999999999999,"5min":7.9569999999999999,"15min":7.9569999999999999},"max":{"1min":24.219000000000001,"5min":46.478000000000002,"15min":46.478000000000002},"last":13.962999999999999},{"interface":"front","average":{"1min":14.906000000000001,"5min":14.161,"15min":14.132999999999999},"min":{"1min":6.415,"5min":6.415,"15min":6.415},"max":{"1min":23.727,"5min":41.603999999999999,"15min":41.603999999999999},"last":11.414}]},{"osd":3,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":17.462,"5min":15.693,"15min":16.739000000000001},"min":{"1min":7.8879999999999999,"5min":6.8319999999999999,"15min":6.8319999999999999},"max":{"1min":47.823,"5min":60.463999999999999,"15min":60.463999999999999},"last":12.75},{"interface":"front","average":{"1min":20.838999999999999,"5min":18.907,"15min":19.867999999999999},"min":{"1min":9.8889999999999993,"5min":9.7370000000000001,"15min":9.7370000000000001},"max":{"1min":44.863,"5min":56.286000000000001,"15min":56.286000000000001},"last":17.911999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":20.128,"5min":20.585000000000001,"15min":21.081},"min":{"1min":10.489000000000001,"5min":8.4960000000000004,"15min":8.4960000000000004},"max":{"1min":32.167999999999999,"5min":57.716999999999999,"15min":57.716999999999999},"last":18.552},{"interface":"front","average":{"1min":16.364000000000001,"5min":17.821999999999999,"15min":18.356000000000002},"min":{"1min":7.2699999999999996,"5min":7.2699999999999996,"15min":7.2699999999999996},"max":{"1min":30.831,"5min":67.159000000000006,"15min":67.159000000000006},"last":15.808}]},{"osd":5,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.968999999999999,"5min":14.587999999999999,"15min":14.688000000000001},"min":{"1min":7.4219999999999997,"5min":6.3399999999999999,"15min":6.3399999999999999},"max":{"1min":27.07,"5min":44.884,"15min":44.884},"last":9.7699999999999996},{"interface":"front","average":{"1min":20.382999999999999,"5min":18.724,"15min":19.527999999999999},"min":{"1min":9.3520000000000003,"5min":9.3520000000000003,"15min":9.3520000000000003},"max":{"1min":92.616,"5min":92.616,"15min":92.616},"last":10.680999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":21.416,"5min":17.657,"15min":17.068999999999999},"min":{"1min":7.625,"5min":7.625,"15min":7.625},"max":{"1min":95.501000000000005,"5min":95.501000000000005,"15min":95.501000000000005},"last":19.923999999999999},{"interface":"front","average":{"1min":23.699999999999999,"5min":20.664999999999999,"15min":20.841000000000001},"min":{"1min":10.811999999999999,"5min":10.144,"15min":10.144},"max":{"1min":94.533000000000001,"5min":94.533000000000001,"15min":94.533000000000001},"last":16.48}]}]},{"osd":6,"up_from":14,"seq":60129542237,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":9.9879999999999995,"5min":9.4030000000000005,"15min":9.8219999999999992},"min":{"1min":4.3010000000000002,"5min":4.1260000000000003,"15min":3.8519999999999999},"max":{"1min":22.216000000000001,"5min":27.033000000000001,"15min":46.947000000000003},"last":3.8140000000000001},{"interface":"front","average":{"1min":11.436,"5min":10.529,"15min":10.420999999999999},"min":{"1min":4.2919999999999998,"5min":4.0650000000000004,"15min":4.0650000000000004},"max":{"1min":18.338999999999999,"5min":33.152000000000001,"15min":45.953000000000003},"last":9.923}]},{"osd":1,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":10.019,"5min":9.423,"15min":11.574999999999999},"min":{"1min":5.6189999999999998,"5min":4.3760000000000003,"15min":4.3760000000000003},"max":{"1min":18.969000000000001,"5min":66.313000000000002,"15min":66.313000000000002},"last":7.0270000000000001},{"interface":"front","average":{"1min":12.763999999999999,"5min":12.657,"15min":13.935},"min":{"1min":6.875,"5min":5.9349999999999996,"15min":5.8540000000000001},"max":{"1min":21.193000000000001,"5min":67.533000000000001,"15min":67.533000000000001},"last":14.852}]},{"osd":2,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.625999999999999,"5min":13.667,"15min":18.298999999999999},"min":{"1min":9.8510000000000009,"5min":6.0640000000000001,"15min":6.0640000000000001},"max":{"1min":41.106000000000002,"5min":46.051000000000002,"15min":200.88499999999999},"last":11.593999999999999},{"interface":"front","average":{"1min":20.809999999999999,"5min":17.507000000000001,"15min":16.818000000000001},"min":{"1min":12.57,"5min":7.8129999999999997,"15min":7.4119999999999999},"max":{"1min":39.863,"5min":45.003999999999998,"15min":48.005000000000003},"last":20.276}]},{"osd":3,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.510000000000002,"5min":14.548999999999999,"15min":14.074},"min":{"1min":9.1600000000000001,"5min":5.9820000000000002,"15min":5.9820000000000002},"max":{"1min":26.93,"5min":36.283999999999999,"15min":36.283999999999999},"last":9.1609999999999996},{"interface":"front","average":{"1min":19.988,"5min":16.667000000000002,"15min":16.466000000000001},"min":{"1min":11.983000000000001,"5min":7.75,"15min":7.6079999999999997},"max":{"1min":31.489999999999998,"5min":34.317,"15min":47.006999999999998},"last":12.333}]},{"osd":4,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.172999999999998,"5min":16.164000000000001,"15min":15.500999999999999},"min":{"1min":10.173999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"max":{"1min":27.928000000000001,"5min":49.170000000000002,"15min":49.170000000000002},"last":10.673},{"interface":"front","average":{"1min":19.071000000000002,"5min":17.736000000000001,"15min":17.052},"min":{"1min":12.409000000000001,"5min":8.9019999999999992,"15min":7.7169999999999996},"max":{"1min":32.399000000000001,"5min":38.573999999999998,"15min":48.988999999999997},"last":19.446999999999999}]},{"osd":5,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":20.074000000000002,"5min":18.266999999999999,"15min":18.411999999999999},"min":{"1min":13.186999999999999,"5min":7.8879999999999999,"15min":7.8879999999999999},"max":{"1min":30.518999999999998,"5min":49.390999999999998,"15min":57.308},"last":21.152000000000001},{"interface":"front","average":{"1min":11.132,"5min":10.885,"15min":13.561999999999999},"min":{"1min":5.5380000000000003,"5min":5.5380000000000003,"15min":5.5380000000000003},"max":{"1min":15.542999999999999,"5min":47.415999999999997,"15min":68.463999999999999},"last":13.837}]},{"osd":7,"last update":"Wed Dec 6 18:34:18 2023","interfaces":[{"interface":"back","average":{"1min":18.114000000000001,"5min":16.631,"15min":15.811},"min":{"1min":9.1460000000000008,"5min":7.4390000000000001,"15min":7.4390000000000001},"max":{"1min":26.856000000000002,"5min":38.887999999999998,"15min":38.887999999999998},"last":15.589},{"interface":"front","average":{"1min":22.518999999999998,"5min":20.096,"15min":19.210000000000001},"min":{"1min":16.774999999999999,"5min":13.555999999999999,"15min":13.555999999999999},"max":{"1min":34.822000000000003,"5min":34.822000000000003,"15min":34.822000000000003},"last":22.228000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:34:27.705 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:34:27.706 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:03:07.149 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:27.721 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:02:01.312 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:27.723 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:02:01.314 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:27.728 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:55.979 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:27.730 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:55.980 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:27.749 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:02:01.341 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:27.759 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:00:56.009 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:28.388 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:34:28.388 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:34:28.401 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2023-12-06T18:34:28.292294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231776,"kb_used_data":10412,"kb_used_omap":15,"kb_used_meta":217008,"kb_avail":754742944,"statfs":{"total":773094113280,"available":772856774656,"internally_reserved":0,"allocated":10661888,"data_stored":4324567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15557,"internal_metadata":222217019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.707578"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.633082+0000","last_change":"2023-12-06T18:29:09.051293+0000","last_active":"2023-12-06T18:34:18.633082+0000","last_peered":"2023-12-06T18:34:18.633082+0000","last_clean":"2023-12-06T18:34:18.633082+0000","last_became_active":"2023-12-06T18:29:09.045331+0000","last_became_peered":"2023-12-06T18:29:09.045331+0000","last_unstale":"2023-12-06T18:34:18.633082+0000","last_undegraded":"2023-12-06T18:34:18.633082+0000","last_fullsized":"2023-12-06T18:34:18.633082+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T02:46:01.854508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":51,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:19.332340+0000","last_change":"2023-12-06T18:33:00.623558+0000","last_active":"2023-12-06T18:34:19.332340+0000","last_peered":"2023-12-06T18:34:19.332340+0000","last_clean":"2023-12-06T18:34:19.332340+0000","last_became_active":"2023-12-06T18:33:00.616353+0000","last_became_peered":"2023-12-06T18:33:00.616353+0000","last_unstale":"2023-12-06T18:34:19.332340+0000","last_undegraded":"2023-12-06T18:34:19.332340+0000","last_fullsized":"2023-12-06T18:34:19.332340+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T18:33:31.069235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.603990+0000","last_change":"2023-12-06T18:33:00.592340+0000","last_active":"2023-12-06T18:34:18.603990+0000","last_peered":"2023-12-06T18:34:18.603990+0000","last_clean":"2023-12-06T18:34:18.603990+0000","last_became_active":"2023-12-06T18:33:00.581037+0000","last_became_peered":"2023-12-06T18:33:00.581037+0000","last_unstale":"2023-12-06T18:34:18.603990+0000","last_undegraded":"2023-12-06T18:34:18.603990+0000","last_fullsized":"2023-12-06T18:34:18.603990+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T06:13:54.498147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":118,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.631290+0000","last_change":"2023-12-06T18:33:52.522397+0000","last_active":"2023-12-06T18:34:18.631290+0000","last_peered":"2023-12-06T18:34:18.631290+0000","last_clean":"2023-12-06T18:34:18.631290+0000","last_became_active":"2023-12-06T18:33:52.516242+0000","last_became_peered":"2023-12-06T18:33:52.516242+0000","last_unstale":"2023-12-06T18:34:18.631290+0000","last_undegraded":"2023-12-06T18:34:18.631290+0000","last_fullsized":"2023-12-06T18:34:18.631290+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T03:19:25.211436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.599973+0000","last_change":"2023-12-06T18:33:52.521292+0000","last_active":"2023-12-06T18:34:18.599973+0000","last_peered":"2023-12-06T18:34:18.599973+0000","last_clean":"2023-12-06T18:34:18.599973+0000","last_became_active":"2023-12-06T18:33:52.514832+0000","last_became_peered":"2023-12-06T18:33:52.514832+0000","last_unstale":"2023-12-06T18:34:18.599973+0000","last_undegraded":"2023-12-06T18:34:18.599973+0000","last_fullsized":"2023-12-06T18:34:18.599973+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T23:51:07.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":94,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-12-06T18:34:19.744131+0000","last_change":"2023-12-06T18:34:19.744131+0000","last_active":"2023-12-06T18:34:19.744131+0000","last_peered":"2023-12-06T18:34:19.744131+0000","last_clean":"2023-12-06T18:34:19.744131+0000","last_became_active":"2023-12-06T18:34:19.738308+0000","last_became_peered":"2023-12-06T18:34:19.738308+0000","last_unstale":"2023-12-06T18:34:19.744131+0000","last_undegraded":"2023-12-06T18:34:19.744131+0000","last_fullsized":"2023-12-06T18:34:19.744131+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T01:32:29.556210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.597827+0000","last_change":"2023-12-06T18:28:47.876556+0000","last_active":"2023-12-06T18:34:18.597827+0000","last_peered":"2023-12-06T18:34:18.597827+0000","last_clean":"2023-12-06T18:34:18.597827+0000","last_became_active":"2023-12-06T18:28:47.865169+0000","last_became_peered":"2023-12-06T18:28:47.865169+0000","last_unstale":"2023-12-06T18:34:18.597827+0000","last_undegraded":"2023-12-06T18:34:18.597827+0000","last_fullsized":"2023-12-06T18:34:18.597827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:16:40.095542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":72,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.991987+0000","last_change":"2023-12-06T18:31:57.187068+0000","last_active":"2023-12-06T18:34:18.991987+0000","last_peered":"2023-12-06T18:34:18.991987+0000","last_clean":"2023-12-06T18:34:18.991987+0000","last_became_active":"2023-12-06T18:31:57.175444+0000","last_became_peered":"2023-12-06T18:31:57.175444+0000","last_unstale":"2023-12-06T18:34:18.991987+0000","last_undegraded":"2023-12-06T18:34:18.991987+0000","last_fullsized":"2023-12-06T18:34:18.991987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:25:09.547812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":49,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:19.324726+0000","last_change":"2023-12-06T18:33:22.203137+0000","last_active":"2023-12-06T18:34:19.324726+0000","last_peered":"2023-12-06T18:34:19.324726+0000","last_clean":"2023-12-06T18:34:19.324726+0000","last_became_active":"2023-12-06T18:33:21.485790+0000","last_became_peered":"2023-12-06T18:33:21.485790+0000","last_unstale":"2023-12-06T18:34:19.324726+0000","last_undegraded":"2023-12-06T18:34:19.324726+0000","last_fullsized":"2023-12-06T18:34:19.324726+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T01:25:32.645799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1605632,"data_stored":1591360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411422,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":11.292,"5min":11.291,"15min":10.609999999999999},"min":{"1min":5.069,"5min":3.835,"15min":3.835},"max":{"1min":27.620000000000001,"5min":108.57599999999999,"15min":108.57599999999999},"last":6.984},{"interface":"front","average":{"1min":11.958,"5min":11.699,"15min":12.119999999999999},"min":{"1min":6.3239999999999998,"5min":4.8639999999999999,"15min":3.7200000000000002},"max":{"1min":24.844000000000001,"5min":128.09399999999999,"15min":128.09399999999999},"last":12.19}]},{"osd":1,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":11.891999999999999,"5min":12.403,"15min":12.693},"min":{"1min":6.5359999999999996,"5min":6.2539999999999996,"15min":6.2539999999999996},"max":{"1min":19.538,"5min":70.072000000000003,"15min":70.072000000000003},"last":14.590999999999999},{"interface":"front","average":{"1min":12.019,"5min":12.824,"15min":14.788},"min":{"1min":6.2080000000000002,"5min":6.2080000000000002,"15min":6.2080000000000002},"max":{"1min":30.16,"5min":30.16,"15min":32.920000000000002},"last":10.211}]},{"osd":2,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":15.49,"5min":15.119,"15min":14.845000000000001},"min":{"1min":9.4450000000000003,"5min":8.5079999999999991,"15min":7.5149999999999997},"max":{"1min":52.232999999999997,"5min":75.125,"15min":75.125},"last":18.074000000000002},{"interface":"front","average":{"1min":18.545000000000002,"5min":16.343,"15min":17.477},"min":{"1min":9.2599999999999998,"5min":7.5389999999999997,"15min":7.5389999999999997},"max":{"1min":54.323999999999998,"5min":69.403999999999996,"15min":69.403999999999996},"last":15.433999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:34:13 2023","interfaces":[{"interface":"back","average":{"1min":18.949999999999999,"5min":15.185,"15min":12.898999999999999},"min":{"1min":10.714,"5min":5.8879999999999999,"15min":5.8879999999999999},"max":{"1min":56.255000000000003,"5min":56.255000000000003,"15min":56.255000000000003},"last":19.283000000000001},{"interface":"front","average":{"1min":14.9,"5min":13.204000000000001,"15min":15.038},"min":{"1min":6.4189999999999996,"5min":6.3099999999999996,"15min":6.3099999999999996},"max":{"1min":54.616,"5min":71.718000000000004,"15min":71.718000000000004},"last":8.9879999999999995}]},{"osd":5,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":17.082000000000001,"5min":18.359999999999999,"15min":18.027999999999999},"min":{"1min":11.888,"5min":10.417999999999999,"15min":8.3179999999999996},"max":{"1min":24.445,"5min":75.751999999999995,"15min":75.751999999999995},"last":23.257000000000001},{"interface":"front","average":{"1min":16.565999999999999,"5min":16.934000000000001,"15min":17.869},"min":{"1min":8.5389999999999997,"5min":7.9269999999999996,"15min":7.9269999999999996},"max":{"1min":32.219000000000001,"5min":70.933000000000007,"15min":70.933000000000007},"last":16.760000000000002}]},{"osd":6,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":19.771000000000001,"5min":15.018000000000001,"15min":15.869999999999999},"min":{"1min":10.798,"5min":8.2050000000000001,"15min":7.0919999999999996},"max":{"1min":30.991,"5min":30.991,"15min":30.991},"last":20.698},{"interface":"front","average":{"1min":16.899999999999999,"5min":18.946999999999999,"15min":20.498000000000001},"min":{"1min":7.2210000000000001,"5min":7.2210000000000001,"15min":7.2210000000000001},"max":{"1min":30.404,"5min":74.224000000000004,"15min":74.224000000000004},"last":11.503}]},{"osd":7,"last update":"Wed Dec 6 18:34:13 2023","interfaces":[{"interface":"back","average":{"1min":20.172999999999998,"5min":19.478999999999999,"15min":21.131},"min":{"1min":12.255000000000001,"5min":9.1859999999999999,"15min":9.1859999999999999},"max":{"1min":30.175000000000001,"5min":122.93000000000001,"15min":122.93000000000001},"last":12.834},{"interface":"front","average":{"1min":22.155000000000001,"5min":22.471,"15min":23.312999999999999},"min":{"1min":15.08,"5min":13.196,"15min":13.196},"max":{"1min":34.466999999999999,"5min":124.473,"15min":124.473},"last":22.026}]}]},{"osd":2,"up_from":16,"seq":68719476829,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":14.244999999999999,"5min":11.606999999999999,"15min":11.34},"min":{"1min":5.234,"5min":4.1639999999999997,"15min":4.1639999999999997},"max":{"1min":47.884999999999998,"5min":47.884999999999998,"15min":47.884999999999998},"last":10.805999999999999},{"interface":"front","average":{"1min":12.324999999999999,"5min":12.346,"15min":11.509},"min":{"1min":7.1399999999999997,"5min":4.3280000000000003,"15min":4.3280000000000003},"max":{"1min":48.305999999999997,"5min":48.305999999999997,"15min":48.305999999999997},"last":9.0329999999999995}]},{"osd":1,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":14.621,"5min":12.172000000000001,"15min":11.682},"min":{"1min":6.8860000000000001,"5min":5.9139999999999997,"15min":5.9139999999999997},"max":{"1min":28.242999999999999,"5min":42.920999999999999,"15min":42.920999999999999},"last":11.391999999999999},{"interface":"front","average":{"1min":17.202999999999999,"5min":14.548,"15min":14.061999999999999},"min":{"1min":7.8840000000000003,"5min":6.1479999999999997,"15min":6.1479999999999997},"max":{"1min":29.478999999999999,"5min":34.607999999999997,"15min":34.607999999999997},"last":13.411}]},{"osd":3,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":16.474,"5min":15.353,"15min":17.821999999999999},"min":{"1min":9.0139999999999993,"5min":7.2549999999999999,"15min":6.1260000000000003},"max":{"1min":37.463000000000001,"5min":49.609000000000002,"15min":112.129},"last":17.719000000000001},{"interface":"front","average":{"1min":21.956,"5min":17.753,"15min":21.963999999999999},"min":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"max":{"1min":36.770000000000003,"5min":61.243000000000002,"15min":181.601},"last":14.285}]},{"osd":4,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":19.707999999999998,"5min":16.417999999999999,"15min":17.349},"min":{"1min":8.5250000000000004,"5min":7.8070000000000004,"15min":6.609},"max":{"1min":35.396999999999998,"5min":60.332000000000001,"15min":97.088999999999999},"last":25.579999999999998},{"interface":"front","average":{"1min":13.917999999999999,"5min":13.611000000000001,"15min":18.606999999999999},"min":{"1min":7.7640000000000002,"5min":6.0720000000000001,"15min":6.0720000000000001},"max":{"1min":24.061,"5min":43.198,"15min":98.069000000000003},"last":18.262}]},{"osd":5,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":19.113,"5min":18.113,"15min":18.097000000000001},"min":{"1min":10.743,"5min":8.2100000000000009,"15min":7.8170000000000002},"max":{"1min":26.042000000000002,"5min":57.628999999999998,"15min":57.628999999999998},"last":17.033999999999999},{"interface":"front","average":{"1min":15.692,"5min":15.933,"15min":15.824},"min":{"1min":10.292,"5min":8.5990000000000002,"15min":8.5990000000000002},"max":{"1min":39.875999999999998,"5min":47.749000000000002,"15min":47.749000000000002},"last":20.780999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":20.952999999999999,"5min":19.140000000000001,"15min":36.003999999999998},"min":{"1min":7.0419999999999998,"5min":7.0419999999999998,"15min":7.0419999999999998},"max":{"1min":47.667999999999999,"5min":64.034999999999997,"15min":498.38499999999999},"last":21.663},{"interface":"front","average":{"1min":24.446000000000002,"5min":20.491,"15min":37.847000000000001},"min":{"1min":12.461,"5min":9.7010000000000005,"15min":9.7010000000000005},"max":{"1min":44.993000000000002,"5min":54.920000000000002,"15min":496.17399999999998},"last":20.18}]},{"osd":7,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":21.704999999999998,"5min":19.547000000000001,"15min":19.131},"min":{"1min":13.574999999999999,"5min":10.064,"15min":10.064},"max":{"1min":35.945,"5min":49.456000000000003,"15min":49.456000000000003},"last":24.835000000000001},{"interface":"front","average":{"1min":21.277999999999999,"5min":20.684000000000001,"15min":20.332000000000001},"min":{"1min":12.81,"5min":12.288,"15min":12.288},"max":{"1min":41.965000000000003,"5min":41.965000000000003,"15min":41.965000000000003},"last":26.071000000000002}]}]},{"osd":1,"up_from":16,"seq":68719476830,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":12.32,"5min":11.260999999999999,"15min":18.989000000000001},"min":{"1min":6.0650000000000004,"5min":5.0890000000000004,"15min":5.0890000000000004},"max":{"1min":34.816000000000003,"5min":34.816000000000003,"15min":238.87100000000001},"last":8.7870000000000008},{"interface":"front","average":{"1min":11.336,"5min":10.768000000000001,"15min":34.408000000000001},"min":{"1min":5.0629999999999997,"5min":5.0629999999999997,"15min":4.4269999999999996},"max":{"1min":22.834,"5min":61.262,"15min":661.33600000000001},"last":9.4359999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.962999999999999,"5min":13.722,"15min":27.556999999999999},"min":{"1min":6.6189999999999998,"5min":6.6189999999999998,"15min":6.6189999999999998},"max":{"1min":57.93,"5min":93.168000000000006,"15min":384.315},"last":12.364000000000001},{"interface":"front","average":{"1min":22.902000000000001,"5min":19.120999999999999,"15min":39.317999999999998},"min":{"1min":12.282,"5min":10.724,"15min":5.7009999999999996},"max":{"1min":60.173000000000002,"5min":94.230000000000004,"15min":656.98699999999997},"last":21.670000000000002}]},{"osd":3,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":12.403,"5min":12.194000000000001,"15min":12.863},"min":{"1min":6.9180000000000001,"5min":5.9720000000000004,"15min":5.548},"max":{"1min":25.774999999999999,"5min":31.16,"15min":34.057000000000002},"last":10.497},{"interface":"front","average":{"1min":14.6,"5min":14.064,"15min":15.500999999999999},"min":{"1min":8.0180000000000007,"5min":6.5730000000000004,"15min":6.5730000000000004},"max":{"1min":31.417999999999999,"5min":65.537000000000006,"15min":65.537000000000006},"last":13.625999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.082000000000001,"5min":13.308,"15min":33.085999999999999},"min":{"1min":8.6059999999999999,"5min":6.7009999999999996,"15min":6.2930000000000001},"max":{"1min":33.427999999999997,"5min":33.427999999999997,"15min":447.67200000000003},"last":17.629999999999999},{"interface":"front","average":{"1min":21.420000000000002,"5min":18.047999999999998,"15min":39.893000000000001},"min":{"1min":7.7300000000000004,"5min":7.7300000000000004,"15min":7.0919999999999996},"max":{"1min":40.564,"5min":44.043999999999997,"15min":630.572},"last":11.143000000000001}]},{"osd":5,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.390000000000001,"5min":16.076000000000001,"15min":35.482999999999997},"min":{"1min":7.2549999999999999,"5min":7.0540000000000003,"15min":7.0540000000000003},"max":{"1min":38.991999999999997,"5min":68.013999999999996,"15min":389.55000000000001},"last":12.994},{"interface":"front","average":{"1min":22.140999999999998,"5min":18.395,"15min":45.957999999999998},"min":{"1min":11.099,"5min":7.5359999999999996,"15min":7.0410000000000004},"max":{"1min":41.655999999999999,"5min":41.655999999999999,"15min":658.48599999999999},"last":20.786000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.672000000000001,"5min":15.733000000000001,"15min":30.202000000000002},"min":{"1min":8.9760000000000009,"5min":7.5899999999999999,"15min":7.5899999999999999},"max":{"1min":32.039999999999999,"5min":54.383000000000003,"15min":439.56400000000002},"last":14.907999999999999},{"interface":"front","average":{"1min":23.643000000000001,"5min":20.5,"15min":32.639000000000003},"min":{"1min":13.058,"5min":10.641999999999999,"15min":10.641999999999999},"max":{"1min":43.052,"5min":55.423999999999999,"15min":386.71499999999997},"last":22.509}]},{"osd":7,"last update":"Wed Dec 6 18:34:16 2023","interfaces":[{"interface":"back","average":{"1min":18.527000000000001,"5min":18.222999999999999,"15min":18.600000000000001},"min":{"1min":8.9849999999999994,"5min":8.9849999999999994,"15min":8.9849999999999994},"max":{"1min":33.156999999999996,"5min":97.561999999999998,"15min":97.561999999999998},"last":16.126000000000001},{"interface":"front","average":{"1min":23.125,"5min":21.024000000000001,"15min":18.771999999999998},"min":{"1min":12.019,"5min":8.7219999999999995,"15min":8.7219999999999995},"max":{"1min":36.332000000000001,"5min":101.762,"15min":101.762},"last":15.502000000000001}]}]},{"osd":0,"up_from":16,"seq":68719476831,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343220,"statfs":{"total":96636764160,"available":96607457280,"internally_reserved":0,"allocated":1634304,"data_stored":838986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":13.518000000000001,"5min":11.452,"15min":15.608000000000001},"min":{"1min":5.6340000000000003,"5min":3.2360000000000002,"15min":3.2360000000000002},"max":{"1min":54.155000000000001,"5min":54.155000000000001,"15min":116.593},"last":7.984},{"interface":"front","average":{"1min":13.859,"5min":12.935,"15min":16.576000000000001},"min":{"1min":5.5389999999999997,"5min":4.9039999999999999,"15min":4.9039999999999999},"max":{"1min":68.778999999999996,"5min":68.778999999999996,"15min":117.896},"last":11.999000000000001}]},{"osd":2,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":17.434000000000001,"5min":14.888,"15min":14.132999999999999},"min":{"1min":8.9870000000000001,"5min":5.694,"15min":5.694},"max":{"1min":72.131,"5min":72.131,"15min":72.131},"last":9.8000000000000007},{"interface":"front","average":{"1min":14.316000000000001,"5min":13.224,"15min":12.055},"min":{"1min":6.7590000000000003,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":59.317999999999998,"5min":59.317999999999998,"15min":59.317999999999998},"last":16.341000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":19.911999999999999,"5min":16.183,"15min":16.312999999999999},"min":{"1min":8.9600000000000009,"5min":6.1379999999999999,"15min":6.1379999999999999},"max":{"1min":75.439999999999998,"5min":75.439999999999998,"15min":75.439999999999998},"last":23.076000000000001},{"interface":"front","average":{"1min":19.082000000000001,"5min":16.047000000000001,"15min":16.204999999999998},"min":{"1min":7.593,"5min":7.5510000000000002,"15min":7.5510000000000002},"max":{"1min":96.688999999999993,"5min":96.688999999999993,"15min":96.688999999999993},"last":9.141}]},{"osd":4,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":14.302,"5min":12.134,"15min":17.890999999999998},"min":{"1min":6.8239999999999998,"5min":5.9349999999999996,"15min":5.9349999999999996},"max":{"1min":57.694000000000003,"5min":57.694000000000003,"15min":82.432000000000002},"last":20.741},{"interface":"front","average":{"1min":19.693000000000001,"5min":18.327000000000002,"15min":19.846},"min":{"1min":7.7469999999999999,"5min":7.7469999999999999,"15min":6.2629999999999999},"max":{"1min":76.078000000000003,"5min":76.078000000000003,"15min":81.239999999999995},"last":19.042000000000002}]},{"osd":5,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":19.27,"5min":16.798999999999999,"15min":17.507000000000001},"min":{"1min":8.1229999999999993,"5min":8.0099999999999998,"15min":8.0099999999999998},"max":{"1min":97.947000000000003,"5min":97.947000000000003,"15min":97.947000000000003},"last":24.241},{"interface":"front","average":{"1min":16.989000000000001,"5min":15.443,"15min":17.440000000000001},"min":{"1min":7.4770000000000003,"5min":7.1779999999999999,"15min":7.1779999999999999},"max":{"1min":69.915999999999997,"5min":69.915999999999997,"15min":72.307000000000002},"last":13.196999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":25.541,"5min":20.477,"15min":19.626000000000001},"min":{"1min":12.661,"5min":11.401,"15min":7.7560000000000002},"max":{"1min":82.775000000000006,"5min":82.775000000000006,"15min":82.775000000000006},"last":25.363},{"interface":"front","average":{"1min":21.75,"5min":17.940999999999999,"15min":18.914999999999999},"min":{"1min":9.7729999999999997,"5min":9.3520000000000003,"15min":9.3520000000000003},"max":{"1min":74.057000000000002,"5min":74.057000000000002,"15min":74.057000000000002},"last":21.91}]},{"osd":7,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":26.137,"5min":21.724,"15min":19.988},"min":{"1min":16.024000000000001,"5min":12.009,"15min":12.009},"max":{"1min":95.287999999999997,"5min":95.287999999999997,"15min":95.287999999999997},"last":26.545000000000002},{"interface":"front","average":{"1min":23.494,"5min":20.041,"15min":18.611000000000001},"min":{"1min":10.24,"5min":10.24,"15min":10.24},"max":{"1min":98.995999999999995,"5min":98.995999999999995,"15min":98.995999999999995},"last":27.462}]}]},{"osd":5,"up_from":15,"seq":64424509535,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28628,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343212,"statfs":{"total":96636764160,"available":96607449088,"internally_reserved":0,"allocated":1634304,"data_stored":838986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":14.44,"5min":9.6449999999999996,"15min":13.73},"min":{"1min":4.0430000000000001,"5min":3.3660000000000001,"15min":3.3660000000000001},"max":{"1min":106.521,"5min":106.521,"15min":208.291},"last":4.9500000000000002},{"interface":"front","average":{"1min":14.817,"5min":11.609,"15min":17.067},"min":{"1min":4.984,"5min":4.758,"15min":4.4299999999999997},"max":{"1min":107.342,"5min":107.342,"15min":199.94900000000001},"last":10.15}]},{"osd":1,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":11.631,"5min":13.298999999999999,"15min":45.225999999999999},"min":{"1min":6.7469999999999999,"5min":5.085,"15min":5.085},"max":{"1min":26.692,"5min":41.234999999999999,"15min":1211.6010000000001},"last":9.1539999999999999},{"interface":"front","average":{"1min":12.16,"5min":10.502000000000001,"15min":41.011000000000003},"min":{"1min":5.359,"5min":4.7199999999999998,"15min":4.7199999999999998},"max":{"1min":22.664000000000001,"5min":39.802999999999997,"15min":1189.3869999999999},"last":13.106999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":14.343,"5min":14.629,"15min":21.640999999999998},"min":{"1min":9.0389999999999997,"5min":6.5129999999999999,"15min":6.5129999999999999},"max":{"1min":23.422000000000001,"5min":68.617999999999995,"15min":283.46800000000002},"last":13.855},{"interface":"front","average":{"1min":19.576000000000001,"5min":18.463999999999999,"15min":25.539000000000001},"min":{"1min":12.851000000000001,"5min":8.173,"15min":8.173},"max":{"1min":40.737000000000002,"5min":69.513999999999996,"15min":296.47800000000001},"last":17.058}]},{"osd":3,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":11.484999999999999,"5min":13.811,"15min":17.414000000000001},"min":{"1min":7.3639999999999999,"5min":6.0300000000000002,"15min":6.0300000000000002},"max":{"1min":19.872,"5min":36.156999999999996,"15min":47.926000000000002},"last":10.849},{"interface":"front","average":{"1min":15.419,"5min":14.555999999999999,"15min":15.007999999999999},"min":{"1min":10.163,"5min":5.3979999999999997,"15min":5.3979999999999997},"max":{"1min":28.914999999999999,"5min":34.609000000000002,"15min":66.176000000000002},"last":15.17}]},{"osd":4,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":18.212,"5min":17.952000000000002,"15min":19.096},"min":{"1min":11.897,"5min":9.7210000000000001,"15min":9.7210000000000001},"max":{"1min":34.069000000000003,"5min":41.923999999999999,"15min":67.248000000000005},"last":15.784000000000001},{"interface":"front","average":{"1min":11.249000000000001,"5min":10.798,"15min":12.347},"min":{"1min":6.5279999999999996,"5min":5.3300000000000001,"15min":5.3300000000000001},"max":{"1min":26.850999999999999,"5min":43.101999999999997,"15min":77.266999999999996},"last":6.9100000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":15.667999999999999,"5min":13.644,"15min":15.140000000000001},"min":{"1min":7.984,"5min":6.1280000000000001,"15min":5.915},"max":{"1min":25.727,"5min":44.587000000000003,"15min":76.751999999999995},"last":11.791},{"interface":"front","average":{"1min":20.158999999999999,"5min":19.356999999999999,"15min":20.469000000000001},"min":{"1min":11.744999999999999,"5min":11.744999999999999,"15min":8.8390000000000004},"max":{"1min":36.883000000000003,"5min":45.366,"15min":57.101999999999997},"last":16.425999999999998}]},{"osd":7,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":18.687999999999999,"5min":16.294,"15min":14.904999999999999},"min":{"1min":11.298999999999999,"5min":7.5519999999999996,"15min":7.5519999999999996},"max":{"1min":32.219999999999999,"5min":32.219999999999999,"15min":32.219999999999999},"last":12.474},{"interface":"front","average":{"1min":14.401999999999999,"5min":12.872999999999999,"15min":12.215999999999999},"min":{"1min":7.133,"5min":7.133,"15min":7.133},"max":{"1min":29.376999999999999,"5min":29.376999999999999,"15min":29.376999999999999},"last":8.2439999999999998}]}]},{"osd":4,"up_from":15,"seq":64424509536,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1228800,"data_stored":440865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":8.0180000000000007,"5min":9.266,"15min":12.295999999999999},"min":{"1min":4.2789999999999999,"5min":3.7429999999999999,"15min":3.7429999999999999},"max":{"1min":13.465999999999999,"5min":41.076000000000001,"15min":53.298999999999999},"last":9.5990000000000002},{"interface":"front","average":{"1min":9.9160000000000004,"5min":10,"15min":10.609999999999999},"min":{"1min":5.1280000000000001,"5min":4.8140000000000001,"15min":3.9670000000000001},"max":{"1min":18.329000000000001,"5min":35.863,"15min":52.567},"last":7.3300000000000001}]},{"osd":1,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":12.202999999999999,"5min":11.432,"15min":13.298999999999999},"min":{"1min":6.0529999999999999,"5min":5.8390000000000004,"15min":5.835},"max":{"1min":21.186,"5min":22.509,"15min":49.183},"last":16.628},{"interface":"front","average":{"1min":13.43,"5min":12.129,"15min":14.662000000000001},"min":{"1min":5.9770000000000003,"5min":5.9770000000000003,"15min":5.96},"max":{"1min":23.596,"5min":24.081,"15min":45.412999999999997},"last":19.776}]},{"osd":2,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":12.398999999999999,"5min":12.423,"15min":19.385000000000002},"min":{"1min":6.609,"5min":5.9539999999999997,"15min":5.6390000000000002},"max":{"1min":22.904,"5min":44.649000000000001,"15min":85.394000000000005},"last":17.27},{"interface":"front","average":{"1min":14.196,"5min":15.087999999999999,"15min":18.899000000000001},"min":{"1min":7.9269999999999996,"5min":7.5209999999999999,"15min":6.3639999999999999},"max":{"1min":27.780000000000001,"5min":42.500999999999998,"15min":69.530000000000001},"last":8.5999999999999996}]},{"osd":3,"last update":"Wed Dec 6 18:34:17 2023","interfaces":[{"interface":"back","average":{"1min":16.875,"5min":15.66,"15min":14.851000000000001},"min":{"1min":8.6349999999999998,"5min":8.0250000000000004,"15min":7.008},"max":{"1min":35.418999999999997,"5min":37.561999999999998,"15min":37.561999999999998},"last":10.789},{"interface":"front","average":{"1min":18.140000000000001,"5min":14.212,"15min":14.936},"min":{"1min":8.0619999999999994,"5min":5.5549999999999997,"15min":5.5549999999999997},"max":{"1min":36.694000000000003,"5min":36.694000000000003,"15min":36.694000000000003},"last":18.012}]},{"osd":5,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":16.562000000000001,"5min":13.952999999999999,"15min":16.228999999999999},"min":{"1min":6.9370000000000003,"5min":5.4969999999999999,"15min":5.4969999999999999},"max":{"1min":27.283000000000001,"5min":53.840000000000003,"15min":78.798000000000002},"last":15.369},{"interface":"front","average":{"1min":16.690000000000001,"5min":17.414000000000001,"15min":18.402000000000001},"min":{"1min":10.425000000000001,"5min":8.0519999999999996,"15min":6.4450000000000003},"max":{"1min":29.184000000000001,"5min":52.781999999999996,"15min":55.814999999999998},"last":12.109}]},{"osd":6,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":19,"5min":18.105,"15min":18.045000000000002},"min":{"1min":9.3170000000000002,"5min":9.2100000000000009,"15min":7.7850000000000001},"max":{"1min":32.869,"5min":41.723999999999997,"15min":51.149000000000001},"last":11.468999999999999},{"interface":"front","average":{"1min":16.550000000000001,"5min":15.154999999999999,"15min":16.164999999999999},"min":{"1min":7.9359999999999999,"5min":7.9359999999999999,"15min":7.8700000000000001},"max":{"1min":29.792000000000002,"5min":34.929000000000002,"15min":40.426000000000002},"last":13.619999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:34:17 2023","interfaces":[{"interface":"back","average":{"1min":17.311,"5min":17.135000000000002,"15min":18.212},"min":{"1min":11.176,"5min":8.093,"15min":8.093},"max":{"1min":24.667000000000002,"5min":60.390000000000001,"15min":60.390000000000001},"last":15.988},{"interface":"front","average":{"1min":21.228999999999999,"5min":21.193999999999999,"15min":22.27},"min":{"1min":11.083,"5min":11.083,"15min":11.083},"max":{"1min":33.914999999999999,"5min":57.606999999999999,"15min":57.606999999999999},"last":12.869}]}]},{"osd":7,"up_from":61,"seq":261993005111,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33412,"kb_used_data":1204,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94338428,"statfs":{"total":96636764160,"available":96602550272,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4395,"internal_metadata":28634837},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.206,"5min":11.058999999999999,"15min":10.446999999999999},"min":{"1min":4.0739999999999998,"5min":4.0739999999999998,"15min":4.0739999999999998},"max":{"1min":126.286,"5min":126.286,"15min":126.286},"last":4.2300000000000004},{"interface":"front","average":{"1min":15.504,"5min":11.65,"15min":11.226000000000001},"min":{"1min":5.2320000000000002,"5min":4.2530000000000001,"15min":4.2530000000000001},"max":{"1min":127.35599999999999,"5min":127.35599999999999,"15min":127.35599999999999},"last":7.7160000000000002}]},{"osd":1,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":12.824999999999999,"5min":13.972,"15min":16.297000000000001},"min":{"1min":5.9130000000000003,"5min":4.6929999999999996,"15min":4.6929999999999996},"max":{"1min":24.047999999999998,"5min":64.765000000000001,"15min":64.765000000000001},"last":6.2510000000000003},{"interface":"front","average":{"1min":13.289999999999999,"5min":15.44,"15min":17.219999999999999},"min":{"1min":6.2969999999999997,"5min":6.2969999999999997,"15min":6.2969999999999997},"max":{"1min":24.402999999999999,"5min":65.994,"15min":65.994},"last":13.346}]},{"osd":2,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.021000000000001,"5min":16.204000000000001,"15min":17.411999999999999},"min":{"1min":7.9569999999999999,"5min":7.9569999999999999,"15min":7.9569999999999999},"max":{"1min":24.219000000000001,"5min":46.478000000000002,"15min":46.478000000000002},"last":13.962999999999999},{"interface":"front","average":{"1min":14.906000000000001,"5min":14.161,"15min":14.132999999999999},"min":{"1min":6.415,"5min":6.415,"15min":6.415},"max":{"1min":23.727,"5min":41.603999999999999,"15min":41.603999999999999},"last":11.414}]},{"osd":3,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":17.462,"5min":15.693,"15min":16.739000000000001},"min":{"1min":7.8879999999999999,"5min":6.8319999999999999,"15min":6.8319999999999999},"max":{"1min":47.823,"5min":60.463999999999999,"15min":60.463999999999999},"last":12.75},{"interface":"front","average":{"1min":20.838999999999999,"5min":18.907,"15min":19.867999999999999},"min":{"1min":9.8889999999999993,"5min":9.7370000000000001,"15min":9.7370000000000001},"max":{"1min":44.863,"5min":56.286000000000001,"15min":56.286000000000001},"last":17.911999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":20.128,"5min":20.585000000000001,"15min":21.081},"min":{"1min":10.489000000000001,"5min":8.4960000000000004,"15min":8.4960000000000004},"max":{"1min":32.167999999999999,"5min":57.716999999999999,"15min":57.716999999999999},"last":18.552},{"interface":"front","average":{"1min":16.364000000000001,"5min":17.821999999999999,"15min":18.356000000000002},"min":{"1min":7.2699999999999996,"5min":7.2699999999999996,"15min":7.2699999999999996},"max":{"1min":30.831,"5min":67.159000000000006,"15min":67.159000000000006},"last":15.808}]},{"osd":5,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.968999999999999,"5min":14.587999999999999,"15min":14.688000000000001},"min":{"1min":7.4219999999999997,"5min":6.3399999999999999,"15min":6.3399999999999999},"max":{"1min":27.07,"5min":44.884,"15min":44.884},"last":9.7699999999999996},{"interface":"front","average":{"1min":20.382999999999999,"5min":18.724,"15min":19.527999999999999},"min":{"1min":9.3520000000000003,"5min":9.3520000000000003,"15min":9.3520000000000003},"max":{"1min":92.616,"5min":92.616,"15min":92.616},"last":10.680999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":21.416,"5min":17.657,"15min":17.068999999999999},"min":{"1min":7.625,"5min":7.625,"15min":7.625},"max":{"1min":95.501000000000005,"5min":95.501000000000005,"15min":95.501000000000005},"last":19.923999999999999},{"interface":"front","average":{"1min":23.699999999999999,"5min":20.664999999999999,"15min":20.841000000000001},"min":{"1min":10.811999999999999,"5min":10.144,"15min":10.144},"max":{"1min":94.533000000000001,"5min":94.533000000000001,"15min":94.533000000000001},"last":16.48}]}]},{"osd":6,"up_from":14,"seq":60129542238,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":9.9879999999999995,"5min":9.4030000000000005,"15min":9.8219999999999992},"min":{"1min":4.3010000000000002,"5min":4.1260000000000003,"15min":3.8519999999999999},"max":{"1min":22.216000000000001,"5min":27.033000000000001,"15min":46.947000000000003},"last":12.391},{"interface":"front","average":{"1min":11.436,"5min":10.529,"15min":10.420999999999999},"min":{"1min":4.2919999999999998,"5min":4.0650000000000004,"15min":4.0650000000000004},"max":{"1min":18.338999999999999,"5min":33.152000000000001,"15min":45.953000000000003},"last":13.583}]},{"osd":1,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":10.019,"5min":9.423,"15min":11.574999999999999},"min":{"1min":5.6189999999999998,"5min":4.3760000000000003,"15min":4.3760000000000003},"max":{"1min":18.969000000000001,"5min":66.313000000000002,"15min":66.313000000000002},"last":8.0109999999999992},{"interface":"front","average":{"1min":12.763999999999999,"5min":12.657,"15min":13.935},"min":{"1min":6.875,"5min":5.9349999999999996,"15min":5.8540000000000001},"max":{"1min":21.193000000000001,"5min":67.533000000000001,"15min":67.533000000000001},"last":8.9749999999999996}]},{"osd":2,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.625999999999999,"5min":13.667,"15min":18.298999999999999},"min":{"1min":9.8510000000000009,"5min":6.0640000000000001,"15min":6.0640000000000001},"max":{"1min":41.106000000000002,"5min":46.051000000000002,"15min":200.88499999999999},"last":9.8770000000000007},{"interface":"front","average":{"1min":20.809999999999999,"5min":17.507000000000001,"15min":16.818000000000001},"min":{"1min":12.57,"5min":7.8129999999999997,"15min":7.4119999999999999},"max":{"1min":39.863,"5min":45.003999999999998,"15min":48.005000000000003},"last":10.448}]},{"osd":3,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.510000000000002,"5min":14.548999999999999,"15min":14.074},"min":{"1min":9.1600000000000001,"5min":5.9820000000000002,"15min":5.9820000000000002},"max":{"1min":26.93,"5min":36.283999999999999,"15min":36.283999999999999},"last":11.109},{"interface":"front","average":{"1min":19.988,"5min":16.667000000000002,"15min":16.466000000000001},"min":{"1min":11.983000000000001,"5min":7.75,"15min":7.6079999999999997},"max":{"1min":31.489999999999998,"5min":34.317,"15min":47.006999999999998},"last":16.024000000000001}]},{"osd":4,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.172999999999998,"5min":16.164000000000001,"15min":15.500999999999999},"min":{"1min":10.173999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"max":{"1min":27.928000000000001,"5min":49.170000000000002,"15min":49.170000000000002},"last":16.382999999999999},{"interface":"front","average":{"1min":19.071000000000002,"5min":17.736000000000001,"15min":17.052},"min":{"1min":12.409000000000001,"5min":8.9019999999999992,"15min":7.7169999999999996},"max":{"1min":32.399000000000001,"5min":38.573999999999998,"15min":48.988999999999997},"last":13.015000000000001}]},{"osd":5,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":20.074000000000002,"5min":18.266999999999999,"15min":18.411999999999999},"min":{"1min":13.186999999999999,"5min":7.8879999999999999,"15min":7.8879999999999999},"max":{"1min":30.518999999999998,"5min":49.390999999999998,"15min":57.308},"last":17.131},{"interface":"front","average":{"1min":11.132,"5min":10.885,"15min":13.561999999999999},"min":{"1min":5.5380000000000003,"5min":5.5380000000000003,"15min":5.5380000000000003},"max":{"1min":15.542999999999999,"5min":47.415999999999997,"15min":68.463999999999999},"last":5.7089999999999996}]},{"osd":7,"last update":"Wed Dec 6 18:34:18 2023","interfaces":[{"interface":"back","average":{"1min":18.114000000000001,"5min":16.631,"15min":15.811},"min":{"1min":9.1460000000000008,"5min":7.4390000000000001,"15min":7.4390000000000001},"max":{"1min":26.856000000000002,"5min":38.887999999999998,"15min":38.887999999999998},"last":14.321},{"interface":"front","average":{"1min":22.518999999999998,"5min":20.096,"15min":19.210000000000001},"min":{"1min":16.774999999999999,"5min":13.555999999999999,"15min":13.555999999999999},"max":{"1min":34.822000000000003,"5min":34.822000000000003,"15min":34.822000000000003},"last":18.202000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:34:28.402 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-06T18:34:28.403 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:34:29.058 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:34:29.058 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:34:29.071 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2023-12-06T18:34:28.292294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231776,"kb_used_data":10412,"kb_used_omap":15,"kb_used_meta":217008,"kb_avail":754742944,"statfs":{"total":773094113280,"available":772856774656,"internally_reserved":0,"allocated":10661888,"data_stored":4324567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15557,"internal_metadata":222217019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.707578"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.633082+0000","last_change":"2023-12-06T18:29:09.051293+0000","last_active":"2023-12-06T18:34:18.633082+0000","last_peered":"2023-12-06T18:34:18.633082+0000","last_clean":"2023-12-06T18:34:18.633082+0000","last_became_active":"2023-12-06T18:29:09.045331+0000","last_became_peered":"2023-12-06T18:29:09.045331+0000","last_unstale":"2023-12-06T18:34:18.633082+0000","last_undegraded":"2023-12-06T18:34:18.633082+0000","last_fullsized":"2023-12-06T18:34:18.633082+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T02:46:01.854508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":51,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:19.332340+0000","last_change":"2023-12-06T18:33:00.623558+0000","last_active":"2023-12-06T18:34:19.332340+0000","last_peered":"2023-12-06T18:34:19.332340+0000","last_clean":"2023-12-06T18:34:19.332340+0000","last_became_active":"2023-12-06T18:33:00.616353+0000","last_became_peered":"2023-12-06T18:33:00.616353+0000","last_unstale":"2023-12-06T18:34:19.332340+0000","last_undegraded":"2023-12-06T18:34:19.332340+0000","last_fullsized":"2023-12-06T18:34:19.332340+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T18:33:31.069235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.603990+0000","last_change":"2023-12-06T18:33:00.592340+0000","last_active":"2023-12-06T18:34:18.603990+0000","last_peered":"2023-12-06T18:34:18.603990+0000","last_clean":"2023-12-06T18:34:18.603990+0000","last_became_active":"2023-12-06T18:33:00.581037+0000","last_became_peered":"2023-12-06T18:33:00.581037+0000","last_unstale":"2023-12-06T18:34:18.603990+0000","last_undegraded":"2023-12-06T18:34:18.603990+0000","last_fullsized":"2023-12-06T18:34:18.603990+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T06:13:54.498147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":118,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.631290+0000","last_change":"2023-12-06T18:33:52.522397+0000","last_active":"2023-12-06T18:34:18.631290+0000","last_peered":"2023-12-06T18:34:18.631290+0000","last_clean":"2023-12-06T18:34:18.631290+0000","last_became_active":"2023-12-06T18:33:52.516242+0000","last_became_peered":"2023-12-06T18:33:52.516242+0000","last_unstale":"2023-12-06T18:34:18.631290+0000","last_undegraded":"2023-12-06T18:34:18.631290+0000","last_fullsized":"2023-12-06T18:34:18.631290+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T03:19:25.211436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.599973+0000","last_change":"2023-12-06T18:33:52.521292+0000","last_active":"2023-12-06T18:34:18.599973+0000","last_peered":"2023-12-06T18:34:18.599973+0000","last_clean":"2023-12-06T18:34:18.599973+0000","last_became_active":"2023-12-06T18:33:52.514832+0000","last_became_peered":"2023-12-06T18:33:52.514832+0000","last_unstale":"2023-12-06T18:34:18.599973+0000","last_undegraded":"2023-12-06T18:34:18.599973+0000","last_fullsized":"2023-12-06T18:34:18.599973+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T23:51:07.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":94,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-12-06T18:34:19.744131+0000","last_change":"2023-12-06T18:34:19.744131+0000","last_active":"2023-12-06T18:34:19.744131+0000","last_peered":"2023-12-06T18:34:19.744131+0000","last_clean":"2023-12-06T18:34:19.744131+0000","last_became_active":"2023-12-06T18:34:19.738308+0000","last_became_peered":"2023-12-06T18:34:19.738308+0000","last_unstale":"2023-12-06T18:34:19.744131+0000","last_undegraded":"2023-12-06T18:34:19.744131+0000","last_fullsized":"2023-12-06T18:34:19.744131+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T01:32:29.556210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.597827+0000","last_change":"2023-12-06T18:28:47.876556+0000","last_active":"2023-12-06T18:34:18.597827+0000","last_peered":"2023-12-06T18:34:18.597827+0000","last_clean":"2023-12-06T18:34:18.597827+0000","last_became_active":"2023-12-06T18:28:47.865169+0000","last_became_peered":"2023-12-06T18:28:47.865169+0000","last_unstale":"2023-12-06T18:34:18.597827+0000","last_undegraded":"2023-12-06T18:34:18.597827+0000","last_fullsized":"2023-12-06T18:34:18.597827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:16:40.095542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":72,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.991987+0000","last_change":"2023-12-06T18:31:57.187068+0000","last_active":"2023-12-06T18:34:18.991987+0000","last_peered":"2023-12-06T18:34:18.991987+0000","last_clean":"2023-12-06T18:34:18.991987+0000","last_became_active":"2023-12-06T18:31:57.175444+0000","last_became_peered":"2023-12-06T18:31:57.175444+0000","last_unstale":"2023-12-06T18:34:18.991987+0000","last_undegraded":"2023-12-06T18:34:18.991987+0000","last_fullsized":"2023-12-06T18:34:18.991987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T20:25:09.547812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"23'76","reported_seq":49,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:19.324726+0000","last_change":"2023-12-06T18:33:22.203137+0000","last_active":"2023-12-06T18:34:19.324726+0000","last_peered":"2023-12-06T18:34:19.324726+0000","last_clean":"2023-12-06T18:34:19.324726+0000","last_became_active":"2023-12-06T18:33:21.485790+0000","last_became_peered":"2023-12-06T18:33:21.485790+0000","last_unstale":"2023-12-06T18:34:19.324726+0000","last_undegraded":"2023-12-06T18:34:19.324726+0000","last_fullsized":"2023-12-06T18:34:19.324726+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:26:52.919636+0000","last_clean_scrub_stamp":"2023-12-06T18:26:52.919636+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T01:25:32.645799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1605632,"data_stored":1591360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411422,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":11.292,"5min":11.291,"15min":10.609999999999999},"min":{"1min":5.069,"5min":3.835,"15min":3.835},"max":{"1min":27.620000000000001,"5min":108.57599999999999,"15min":108.57599999999999},"last":6.984},{"interface":"front","average":{"1min":11.958,"5min":11.699,"15min":12.119999999999999},"min":{"1min":6.3239999999999998,"5min":4.8639999999999999,"15min":3.7200000000000002},"max":{"1min":24.844000000000001,"5min":128.09399999999999,"15min":128.09399999999999},"last":12.19}]},{"osd":1,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":11.891999999999999,"5min":12.403,"15min":12.693},"min":{"1min":6.5359999999999996,"5min":6.2539999999999996,"15min":6.2539999999999996},"max":{"1min":19.538,"5min":70.072000000000003,"15min":70.072000000000003},"last":14.590999999999999},{"interface":"front","average":{"1min":12.019,"5min":12.824,"15min":14.788},"min":{"1min":6.2080000000000002,"5min":6.2080000000000002,"15min":6.2080000000000002},"max":{"1min":30.16,"5min":30.16,"15min":32.920000000000002},"last":10.211}]},{"osd":2,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":15.49,"5min":15.119,"15min":14.845000000000001},"min":{"1min":9.4450000000000003,"5min":8.5079999999999991,"15min":7.5149999999999997},"max":{"1min":52.232999999999997,"5min":75.125,"15min":75.125},"last":18.074000000000002},{"interface":"front","average":{"1min":18.545000000000002,"5min":16.343,"15min":17.477},"min":{"1min":9.2599999999999998,"5min":7.5389999999999997,"15min":7.5389999999999997},"max":{"1min":54.323999999999998,"5min":69.403999999999996,"15min":69.403999999999996},"last":15.433999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:34:13 2023","interfaces":[{"interface":"back","average":{"1min":18.949999999999999,"5min":15.185,"15min":12.898999999999999},"min":{"1min":10.714,"5min":5.8879999999999999,"15min":5.8879999999999999},"max":{"1min":56.255000000000003,"5min":56.255000000000003,"15min":56.255000000000003},"last":19.283000000000001},{"interface":"front","average":{"1min":14.9,"5min":13.204000000000001,"15min":15.038},"min":{"1min":6.4189999999999996,"5min":6.3099999999999996,"15min":6.3099999999999996},"max":{"1min":54.616,"5min":71.718000000000004,"15min":71.718000000000004},"last":8.9879999999999995}]},{"osd":5,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":17.082000000000001,"5min":18.359999999999999,"15min":18.027999999999999},"min":{"1min":11.888,"5min":10.417999999999999,"15min":8.3179999999999996},"max":{"1min":24.445,"5min":75.751999999999995,"15min":75.751999999999995},"last":23.257000000000001},{"interface":"front","average":{"1min":16.565999999999999,"5min":16.934000000000001,"15min":17.869},"min":{"1min":8.5389999999999997,"5min":7.9269999999999996,"15min":7.9269999999999996},"max":{"1min":32.219000000000001,"5min":70.933000000000007,"15min":70.933000000000007},"last":16.760000000000002}]},{"osd":6,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":19.771000000000001,"5min":15.018000000000001,"15min":15.869999999999999},"min":{"1min":10.798,"5min":8.2050000000000001,"15min":7.0919999999999996},"max":{"1min":30.991,"5min":30.991,"15min":30.991},"last":20.698},{"interface":"front","average":{"1min":16.899999999999999,"5min":18.946999999999999,"15min":20.498000000000001},"min":{"1min":7.2210000000000001,"5min":7.2210000000000001,"15min":7.2210000000000001},"max":{"1min":30.404,"5min":74.224000000000004,"15min":74.224000000000004},"last":11.503}]},{"osd":7,"last update":"Wed Dec 6 18:34:13 2023","interfaces":[{"interface":"back","average":{"1min":20.172999999999998,"5min":19.478999999999999,"15min":21.131},"min":{"1min":12.255000000000001,"5min":9.1859999999999999,"15min":9.1859999999999999},"max":{"1min":30.175000000000001,"5min":122.93000000000001,"15min":122.93000000000001},"last":12.834},{"interface":"front","average":{"1min":22.155000000000001,"5min":22.471,"15min":23.312999999999999},"min":{"1min":15.08,"5min":13.196,"15min":13.196},"max":{"1min":34.466999999999999,"5min":124.473,"15min":124.473},"last":22.026}]}]},{"osd":2,"up_from":16,"seq":68719476829,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":14.244999999999999,"5min":11.606999999999999,"15min":11.34},"min":{"1min":5.234,"5min":4.1639999999999997,"15min":4.1639999999999997},"max":{"1min":47.884999999999998,"5min":47.884999999999998,"15min":47.884999999999998},"last":10.805999999999999},{"interface":"front","average":{"1min":12.324999999999999,"5min":12.346,"15min":11.509},"min":{"1min":7.1399999999999997,"5min":4.3280000000000003,"15min":4.3280000000000003},"max":{"1min":48.305999999999997,"5min":48.305999999999997,"15min":48.305999999999997},"last":9.0329999999999995}]},{"osd":1,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":14.621,"5min":12.172000000000001,"15min":11.682},"min":{"1min":6.8860000000000001,"5min":5.9139999999999997,"15min":5.9139999999999997},"max":{"1min":28.242999999999999,"5min":42.920999999999999,"15min":42.920999999999999},"last":11.391999999999999},{"interface":"front","average":{"1min":17.202999999999999,"5min":14.548,"15min":14.061999999999999},"min":{"1min":7.8840000000000003,"5min":6.1479999999999997,"15min":6.1479999999999997},"max":{"1min":29.478999999999999,"5min":34.607999999999997,"15min":34.607999999999997},"last":13.411}]},{"osd":3,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":16.474,"5min":15.353,"15min":17.821999999999999},"min":{"1min":9.0139999999999993,"5min":7.2549999999999999,"15min":6.1260000000000003},"max":{"1min":37.463000000000001,"5min":49.609000000000002,"15min":112.129},"last":17.719000000000001},{"interface":"front","average":{"1min":21.956,"5min":17.753,"15min":21.963999999999999},"min":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"max":{"1min":36.770000000000003,"5min":61.243000000000002,"15min":181.601},"last":14.285}]},{"osd":4,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":19.707999999999998,"5min":16.417999999999999,"15min":17.349},"min":{"1min":8.5250000000000004,"5min":7.8070000000000004,"15min":6.609},"max":{"1min":35.396999999999998,"5min":60.332000000000001,"15min":97.088999999999999},"last":25.579999999999998},{"interface":"front","average":{"1min":13.917999999999999,"5min":13.611000000000001,"15min":18.606999999999999},"min":{"1min":7.7640000000000002,"5min":6.0720000000000001,"15min":6.0720000000000001},"max":{"1min":24.061,"5min":43.198,"15min":98.069000000000003},"last":18.262}]},{"osd":5,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":19.113,"5min":18.113,"15min":18.097000000000001},"min":{"1min":10.743,"5min":8.2100000000000009,"15min":7.8170000000000002},"max":{"1min":26.042000000000002,"5min":57.628999999999998,"15min":57.628999999999998},"last":17.033999999999999},{"interface":"front","average":{"1min":15.692,"5min":15.933,"15min":15.824},"min":{"1min":10.292,"5min":8.5990000000000002,"15min":8.5990000000000002},"max":{"1min":39.875999999999998,"5min":47.749000000000002,"15min":47.749000000000002},"last":20.780999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":20.952999999999999,"5min":19.140000000000001,"15min":36.003999999999998},"min":{"1min":7.0419999999999998,"5min":7.0419999999999998,"15min":7.0419999999999998},"max":{"1min":47.667999999999999,"5min":64.034999999999997,"15min":498.38499999999999},"last":21.663},{"interface":"front","average":{"1min":24.446000000000002,"5min":20.491,"15min":37.847000000000001},"min":{"1min":12.461,"5min":9.7010000000000005,"15min":9.7010000000000005},"max":{"1min":44.993000000000002,"5min":54.920000000000002,"15min":496.17399999999998},"last":20.18}]},{"osd":7,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":21.704999999999998,"5min":19.547000000000001,"15min":19.131},"min":{"1min":13.574999999999999,"5min":10.064,"15min":10.064},"max":{"1min":35.945,"5min":49.456000000000003,"15min":49.456000000000003},"last":24.835000000000001},{"interface":"front","average":{"1min":21.277999999999999,"5min":20.684000000000001,"15min":20.332000000000001},"min":{"1min":12.81,"5min":12.288,"15min":12.288},"max":{"1min":41.965000000000003,"5min":41.965000000000003,"15min":41.965000000000003},"last":26.071000000000002}]}]},{"osd":1,"up_from":16,"seq":68719476830,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":12.32,"5min":11.260999999999999,"15min":18.989000000000001},"min":{"1min":6.0650000000000004,"5min":5.0890000000000004,"15min":5.0890000000000004},"max":{"1min":34.816000000000003,"5min":34.816000000000003,"15min":238.87100000000001},"last":8.7870000000000008},{"interface":"front","average":{"1min":11.336,"5min":10.768000000000001,"15min":34.408000000000001},"min":{"1min":5.0629999999999997,"5min":5.0629999999999997,"15min":4.4269999999999996},"max":{"1min":22.834,"5min":61.262,"15min":661.33600000000001},"last":9.4359999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.962999999999999,"5min":13.722,"15min":27.556999999999999},"min":{"1min":6.6189999999999998,"5min":6.6189999999999998,"15min":6.6189999999999998},"max":{"1min":57.93,"5min":93.168000000000006,"15min":384.315},"last":12.364000000000001},{"interface":"front","average":{"1min":22.902000000000001,"5min":19.120999999999999,"15min":39.317999999999998},"min":{"1min":12.282,"5min":10.724,"15min":5.7009999999999996},"max":{"1min":60.173000000000002,"5min":94.230000000000004,"15min":656.98699999999997},"last":21.670000000000002}]},{"osd":3,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":12.403,"5min":12.194000000000001,"15min":12.863},"min":{"1min":6.9180000000000001,"5min":5.9720000000000004,"15min":5.548},"max":{"1min":25.774999999999999,"5min":31.16,"15min":34.057000000000002},"last":10.497},{"interface":"front","average":{"1min":14.6,"5min":14.064,"15min":15.500999999999999},"min":{"1min":8.0180000000000007,"5min":6.5730000000000004,"15min":6.5730000000000004},"max":{"1min":31.417999999999999,"5min":65.537000000000006,"15min":65.537000000000006},"last":13.625999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.082000000000001,"5min":13.308,"15min":33.085999999999999},"min":{"1min":8.6059999999999999,"5min":6.7009999999999996,"15min":6.2930000000000001},"max":{"1min":33.427999999999997,"5min":33.427999999999997,"15min":447.67200000000003},"last":17.629999999999999},{"interface":"front","average":{"1min":21.420000000000002,"5min":18.047999999999998,"15min":39.893000000000001},"min":{"1min":7.7300000000000004,"5min":7.7300000000000004,"15min":7.0919999999999996},"max":{"1min":40.564,"5min":44.043999999999997,"15min":630.572},"last":11.143000000000001}]},{"osd":5,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.390000000000001,"5min":16.076000000000001,"15min":35.482999999999997},"min":{"1min":7.2549999999999999,"5min":7.0540000000000003,"15min":7.0540000000000003},"max":{"1min":38.991999999999997,"5min":68.013999999999996,"15min":389.55000000000001},"last":12.994},{"interface":"front","average":{"1min":22.140999999999998,"5min":18.395,"15min":45.957999999999998},"min":{"1min":11.099,"5min":7.5359999999999996,"15min":7.0410000000000004},"max":{"1min":41.655999999999999,"5min":41.655999999999999,"15min":658.48599999999999},"last":20.786000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.672000000000001,"5min":15.733000000000001,"15min":30.202000000000002},"min":{"1min":8.9760000000000009,"5min":7.5899999999999999,"15min":7.5899999999999999},"max":{"1min":32.039999999999999,"5min":54.383000000000003,"15min":439.56400000000002},"last":14.907999999999999},{"interface":"front","average":{"1min":23.643000000000001,"5min":20.5,"15min":32.639000000000003},"min":{"1min":13.058,"5min":10.641999999999999,"15min":10.641999999999999},"max":{"1min":43.052,"5min":55.423999999999999,"15min":386.71499999999997},"last":22.509}]},{"osd":7,"last update":"Wed Dec 6 18:34:16 2023","interfaces":[{"interface":"back","average":{"1min":18.527000000000001,"5min":18.222999999999999,"15min":18.600000000000001},"min":{"1min":8.9849999999999994,"5min":8.9849999999999994,"15min":8.9849999999999994},"max":{"1min":33.156999999999996,"5min":97.561999999999998,"15min":97.561999999999998},"last":16.126000000000001},{"interface":"front","average":{"1min":23.125,"5min":21.024000000000001,"15min":18.771999999999998},"min":{"1min":12.019,"5min":8.7219999999999995,"15min":8.7219999999999995},"max":{"1min":36.332000000000001,"5min":101.762,"15min":101.762},"last":15.502000000000001}]}]},{"osd":0,"up_from":16,"seq":68719476831,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343220,"statfs":{"total":96636764160,"available":96607457280,"internally_reserved":0,"allocated":1634304,"data_stored":838986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":13.518000000000001,"5min":11.452,"15min":15.608000000000001},"min":{"1min":5.6340000000000003,"5min":3.2360000000000002,"15min":3.2360000000000002},"max":{"1min":54.155000000000001,"5min":54.155000000000001,"15min":116.593},"last":7.984},{"interface":"front","average":{"1min":13.859,"5min":12.935,"15min":16.576000000000001},"min":{"1min":5.5389999999999997,"5min":4.9039999999999999,"15min":4.9039999999999999},"max":{"1min":68.778999999999996,"5min":68.778999999999996,"15min":117.896},"last":11.999000000000001}]},{"osd":2,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":17.434000000000001,"5min":14.888,"15min":14.132999999999999},"min":{"1min":8.9870000000000001,"5min":5.694,"15min":5.694},"max":{"1min":72.131,"5min":72.131,"15min":72.131},"last":9.8000000000000007},{"interface":"front","average":{"1min":14.316000000000001,"5min":13.224,"15min":12.055},"min":{"1min":6.7590000000000003,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":59.317999999999998,"5min":59.317999999999998,"15min":59.317999999999998},"last":16.341000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":19.911999999999999,"5min":16.183,"15min":16.312999999999999},"min":{"1min":8.9600000000000009,"5min":6.1379999999999999,"15min":6.1379999999999999},"max":{"1min":75.439999999999998,"5min":75.439999999999998,"15min":75.439999999999998},"last":23.076000000000001},{"interface":"front","average":{"1min":19.082000000000001,"5min":16.047000000000001,"15min":16.204999999999998},"min":{"1min":7.593,"5min":7.5510000000000002,"15min":7.5510000000000002},"max":{"1min":96.688999999999993,"5min":96.688999999999993,"15min":96.688999999999993},"last":9.141}]},{"osd":4,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":14.302,"5min":12.134,"15min":17.890999999999998},"min":{"1min":6.8239999999999998,"5min":5.9349999999999996,"15min":5.9349999999999996},"max":{"1min":57.694000000000003,"5min":57.694000000000003,"15min":82.432000000000002},"last":20.741},{"interface":"front","average":{"1min":19.693000000000001,"5min":18.327000000000002,"15min":19.846},"min":{"1min":7.7469999999999999,"5min":7.7469999999999999,"15min":6.2629999999999999},"max":{"1min":76.078000000000003,"5min":76.078000000000003,"15min":81.239999999999995},"last":19.042000000000002}]},{"osd":5,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":19.27,"5min":16.798999999999999,"15min":17.507000000000001},"min":{"1min":8.1229999999999993,"5min":8.0099999999999998,"15min":8.0099999999999998},"max":{"1min":97.947000000000003,"5min":97.947000000000003,"15min":97.947000000000003},"last":24.241},{"interface":"front","average":{"1min":16.989000000000001,"5min":15.443,"15min":17.440000000000001},"min":{"1min":7.4770000000000003,"5min":7.1779999999999999,"15min":7.1779999999999999},"max":{"1min":69.915999999999997,"5min":69.915999999999997,"15min":72.307000000000002},"last":13.196999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":25.541,"5min":20.477,"15min":19.626000000000001},"min":{"1min":12.661,"5min":11.401,"15min":7.7560000000000002},"max":{"1min":82.775000000000006,"5min":82.775000000000006,"15min":82.775000000000006},"last":25.363},{"interface":"front","average":{"1min":21.75,"5min":17.940999999999999,"15min":18.914999999999999},"min":{"1min":9.7729999999999997,"5min":9.3520000000000003,"15min":9.3520000000000003},"max":{"1min":74.057000000000002,"5min":74.057000000000002,"15min":74.057000000000002},"last":21.91}]},{"osd":7,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":26.137,"5min":21.724,"15min":19.988},"min":{"1min":16.024000000000001,"5min":12.009,"15min":12.009},"max":{"1min":95.287999999999997,"5min":95.287999999999997,"15min":95.287999999999997},"last":26.545000000000002},{"interface":"front","average":{"1min":23.494,"5min":20.041,"15min":18.611000000000001},"min":{"1min":10.24,"5min":10.24,"15min":10.24},"max":{"1min":98.995999999999995,"5min":98.995999999999995,"15min":98.995999999999995},"last":27.462}]}]},{"osd":5,"up_from":15,"seq":64424509535,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28628,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343212,"statfs":{"total":96636764160,"available":96607449088,"internally_reserved":0,"allocated":1634304,"data_stored":838986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":14.44,"5min":9.6449999999999996,"15min":13.73},"min":{"1min":4.0430000000000001,"5min":3.3660000000000001,"15min":3.3660000000000001},"max":{"1min":106.521,"5min":106.521,"15min":208.291},"last":4.9500000000000002},{"interface":"front","average":{"1min":14.817,"5min":11.609,"15min":17.067},"min":{"1min":4.984,"5min":4.758,"15min":4.4299999999999997},"max":{"1min":107.342,"5min":107.342,"15min":199.94900000000001},"last":10.15}]},{"osd":1,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":11.631,"5min":13.298999999999999,"15min":45.225999999999999},"min":{"1min":6.7469999999999999,"5min":5.085,"15min":5.085},"max":{"1min":26.692,"5min":41.234999999999999,"15min":1211.6010000000001},"last":9.1539999999999999},{"interface":"front","average":{"1min":12.16,"5min":10.502000000000001,"15min":41.011000000000003},"min":{"1min":5.359,"5min":4.7199999999999998,"15min":4.7199999999999998},"max":{"1min":22.664000000000001,"5min":39.802999999999997,"15min":1189.3869999999999},"last":13.106999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":14.343,"5min":14.629,"15min":21.640999999999998},"min":{"1min":9.0389999999999997,"5min":6.5129999999999999,"15min":6.5129999999999999},"max":{"1min":23.422000000000001,"5min":68.617999999999995,"15min":283.46800000000002},"last":13.855},{"interface":"front","average":{"1min":19.576000000000001,"5min":18.463999999999999,"15min":25.539000000000001},"min":{"1min":12.851000000000001,"5min":8.173,"15min":8.173},"max":{"1min":40.737000000000002,"5min":69.513999999999996,"15min":296.47800000000001},"last":17.058}]},{"osd":3,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":11.484999999999999,"5min":13.811,"15min":17.414000000000001},"min":{"1min":7.3639999999999999,"5min":6.0300000000000002,"15min":6.0300000000000002},"max":{"1min":19.872,"5min":36.156999999999996,"15min":47.926000000000002},"last":10.849},{"interface":"front","average":{"1min":15.419,"5min":14.555999999999999,"15min":15.007999999999999},"min":{"1min":10.163,"5min":5.3979999999999997,"15min":5.3979999999999997},"max":{"1min":28.914999999999999,"5min":34.609000000000002,"15min":66.176000000000002},"last":15.17}]},{"osd":4,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":18.212,"5min":17.952000000000002,"15min":19.096},"min":{"1min":11.897,"5min":9.7210000000000001,"15min":9.7210000000000001},"max":{"1min":34.069000000000003,"5min":41.923999999999999,"15min":67.248000000000005},"last":15.784000000000001},{"interface":"front","average":{"1min":11.249000000000001,"5min":10.798,"15min":12.347},"min":{"1min":6.5279999999999996,"5min":5.3300000000000001,"15min":5.3300000000000001},"max":{"1min":26.850999999999999,"5min":43.101999999999997,"15min":77.266999999999996},"last":6.9100000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":15.667999999999999,"5min":13.644,"15min":15.140000000000001},"min":{"1min":7.984,"5min":6.1280000000000001,"15min":5.915},"max":{"1min":25.727,"5min":44.587000000000003,"15min":76.751999999999995},"last":11.791},{"interface":"front","average":{"1min":20.158999999999999,"5min":19.356999999999999,"15min":20.469000000000001},"min":{"1min":11.744999999999999,"5min":11.744999999999999,"15min":8.8390000000000004},"max":{"1min":36.883000000000003,"5min":45.366,"15min":57.101999999999997},"last":16.425999999999998}]},{"osd":7,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":18.687999999999999,"5min":16.294,"15min":14.904999999999999},"min":{"1min":11.298999999999999,"5min":7.5519999999999996,"15min":7.5519999999999996},"max":{"1min":32.219999999999999,"5min":32.219999999999999,"15min":32.219999999999999},"last":12.474},{"interface":"front","average":{"1min":14.401999999999999,"5min":12.872999999999999,"15min":12.215999999999999},"min":{"1min":7.133,"5min":7.133,"15min":7.133},"max":{"1min":29.376999999999999,"5min":29.376999999999999,"15min":29.376999999999999},"last":8.2439999999999998}]}]},{"osd":4,"up_from":15,"seq":64424509536,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1228800,"data_stored":440865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":8.0180000000000007,"5min":9.266,"15min":12.295999999999999},"min":{"1min":4.2789999999999999,"5min":3.7429999999999999,"15min":3.7429999999999999},"max":{"1min":13.465999999999999,"5min":41.076000000000001,"15min":53.298999999999999},"last":9.5990000000000002},{"interface":"front","average":{"1min":9.9160000000000004,"5min":10,"15min":10.609999999999999},"min":{"1min":5.1280000000000001,"5min":4.8140000000000001,"15min":3.9670000000000001},"max":{"1min":18.329000000000001,"5min":35.863,"15min":52.567},"last":7.3300000000000001}]},{"osd":1,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":12.202999999999999,"5min":11.432,"15min":13.298999999999999},"min":{"1min":6.0529999999999999,"5min":5.8390000000000004,"15min":5.835},"max":{"1min":21.186,"5min":22.509,"15min":49.183},"last":16.628},{"interface":"front","average":{"1min":13.43,"5min":12.129,"15min":14.662000000000001},"min":{"1min":5.9770000000000003,"5min":5.9770000000000003,"15min":5.96},"max":{"1min":23.596,"5min":24.081,"15min":45.412999999999997},"last":19.776}]},{"osd":2,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":12.398999999999999,"5min":12.423,"15min":19.385000000000002},"min":{"1min":6.609,"5min":5.9539999999999997,"15min":5.6390000000000002},"max":{"1min":22.904,"5min":44.649000000000001,"15min":85.394000000000005},"last":17.27},{"interface":"front","average":{"1min":14.196,"5min":15.087999999999999,"15min":18.899000000000001},"min":{"1min":7.9269999999999996,"5min":7.5209999999999999,"15min":6.3639999999999999},"max":{"1min":27.780000000000001,"5min":42.500999999999998,"15min":69.530000000000001},"last":8.5999999999999996}]},{"osd":3,"last update":"Wed Dec 6 18:34:17 2023","interfaces":[{"interface":"back","average":{"1min":16.875,"5min":15.66,"15min":14.851000000000001},"min":{"1min":8.6349999999999998,"5min":8.0250000000000004,"15min":7.008},"max":{"1min":35.418999999999997,"5min":37.561999999999998,"15min":37.561999999999998},"last":10.789},{"interface":"front","average":{"1min":18.140000000000001,"5min":14.212,"15min":14.936},"min":{"1min":8.0619999999999994,"5min":5.5549999999999997,"15min":5.5549999999999997},"max":{"1min":36.694000000000003,"5min":36.694000000000003,"15min":36.694000000000003},"last":18.012}]},{"osd":5,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":16.562000000000001,"5min":13.952999999999999,"15min":16.228999999999999},"min":{"1min":6.9370000000000003,"5min":5.4969999999999999,"15min":5.4969999999999999},"max":{"1min":27.283000000000001,"5min":53.840000000000003,"15min":78.798000000000002},"last":15.369},{"interface":"front","average":{"1min":16.690000000000001,"5min":17.414000000000001,"15min":18.402000000000001},"min":{"1min":10.425000000000001,"5min":8.0519999999999996,"15min":6.4450000000000003},"max":{"1min":29.184000000000001,"5min":52.781999999999996,"15min":55.814999999999998},"last":12.109}]},{"osd":6,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":19,"5min":18.105,"15min":18.045000000000002},"min":{"1min":9.3170000000000002,"5min":9.2100000000000009,"15min":7.7850000000000001},"max":{"1min":32.869,"5min":41.723999999999997,"15min":51.149000000000001},"last":11.468999999999999},{"interface":"front","average":{"1min":16.550000000000001,"5min":15.154999999999999,"15min":16.164999999999999},"min":{"1min":7.9359999999999999,"5min":7.9359999999999999,"15min":7.8700000000000001},"max":{"1min":29.792000000000002,"5min":34.929000000000002,"15min":40.426000000000002},"last":13.619999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:34:17 2023","interfaces":[{"interface":"back","average":{"1min":17.311,"5min":17.135000000000002,"15min":18.212},"min":{"1min":11.176,"5min":8.093,"15min":8.093},"max":{"1min":24.667000000000002,"5min":60.390000000000001,"15min":60.390000000000001},"last":15.988},{"interface":"front","average":{"1min":21.228999999999999,"5min":21.193999999999999,"15min":22.27},"min":{"1min":11.083,"5min":11.083,"15min":11.083},"max":{"1min":33.914999999999999,"5min":57.606999999999999,"15min":57.606999999999999},"last":12.869}]}]},{"osd":7,"up_from":61,"seq":261993005111,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33412,"kb_used_data":1204,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94338428,"statfs":{"total":96636764160,"available":96602550272,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4395,"internal_metadata":28634837},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.206,"5min":11.058999999999999,"15min":10.446999999999999},"min":{"1min":4.0739999999999998,"5min":4.0739999999999998,"15min":4.0739999999999998},"max":{"1min":126.286,"5min":126.286,"15min":126.286},"last":4.2300000000000004},{"interface":"front","average":{"1min":15.504,"5min":11.65,"15min":11.226000000000001},"min":{"1min":5.2320000000000002,"5min":4.2530000000000001,"15min":4.2530000000000001},"max":{"1min":127.35599999999999,"5min":127.35599999999999,"15min":127.35599999999999},"last":7.7160000000000002}]},{"osd":1,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":12.824999999999999,"5min":13.972,"15min":16.297000000000001},"min":{"1min":5.9130000000000003,"5min":4.6929999999999996,"15min":4.6929999999999996},"max":{"1min":24.047999999999998,"5min":64.765000000000001,"15min":64.765000000000001},"last":6.2510000000000003},{"interface":"front","average":{"1min":13.289999999999999,"5min":15.44,"15min":17.219999999999999},"min":{"1min":6.2969999999999997,"5min":6.2969999999999997,"15min":6.2969999999999997},"max":{"1min":24.402999999999999,"5min":65.994,"15min":65.994},"last":13.346}]},{"osd":2,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.021000000000001,"5min":16.204000000000001,"15min":17.411999999999999},"min":{"1min":7.9569999999999999,"5min":7.9569999999999999,"15min":7.9569999999999999},"max":{"1min":24.219000000000001,"5min":46.478000000000002,"15min":46.478000000000002},"last":13.962999999999999},{"interface":"front","average":{"1min":14.906000000000001,"5min":14.161,"15min":14.132999999999999},"min":{"1min":6.415,"5min":6.415,"15min":6.415},"max":{"1min":23.727,"5min":41.603999999999999,"15min":41.603999999999999},"last":11.414}]},{"osd":3,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":17.462,"5min":15.693,"15min":16.739000000000001},"min":{"1min":7.8879999999999999,"5min":6.8319999999999999,"15min":6.8319999999999999},"max":{"1min":47.823,"5min":60.463999999999999,"15min":60.463999999999999},"last":12.75},{"interface":"front","average":{"1min":20.838999999999999,"5min":18.907,"15min":19.867999999999999},"min":{"1min":9.8889999999999993,"5min":9.7370000000000001,"15min":9.7370000000000001},"max":{"1min":44.863,"5min":56.286000000000001,"15min":56.286000000000001},"last":17.911999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":20.128,"5min":20.585000000000001,"15min":21.081},"min":{"1min":10.489000000000001,"5min":8.4960000000000004,"15min":8.4960000000000004},"max":{"1min":32.167999999999999,"5min":57.716999999999999,"15min":57.716999999999999},"last":18.552},{"interface":"front","average":{"1min":16.364000000000001,"5min":17.821999999999999,"15min":18.356000000000002},"min":{"1min":7.2699999999999996,"5min":7.2699999999999996,"15min":7.2699999999999996},"max":{"1min":30.831,"5min":67.159000000000006,"15min":67.159000000000006},"last":15.808}]},{"osd":5,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.968999999999999,"5min":14.587999999999999,"15min":14.688000000000001},"min":{"1min":7.4219999999999997,"5min":6.3399999999999999,"15min":6.3399999999999999},"max":{"1min":27.07,"5min":44.884,"15min":44.884},"last":9.7699999999999996},{"interface":"front","average":{"1min":20.382999999999999,"5min":18.724,"15min":19.527999999999999},"min":{"1min":9.3520000000000003,"5min":9.3520000000000003,"15min":9.3520000000000003},"max":{"1min":92.616,"5min":92.616,"15min":92.616},"last":10.680999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":21.416,"5min":17.657,"15min":17.068999999999999},"min":{"1min":7.625,"5min":7.625,"15min":7.625},"max":{"1min":95.501000000000005,"5min":95.501000000000005,"15min":95.501000000000005},"last":19.923999999999999},{"interface":"front","average":{"1min":23.699999999999999,"5min":20.664999999999999,"15min":20.841000000000001},"min":{"1min":10.811999999999999,"5min":10.144,"15min":10.144},"max":{"1min":94.533000000000001,"5min":94.533000000000001,"15min":94.533000000000001},"last":16.48}]}]},{"osd":6,"up_from":14,"seq":60129542238,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":9.9879999999999995,"5min":9.4030000000000005,"15min":9.8219999999999992},"min":{"1min":4.3010000000000002,"5min":4.1260000000000003,"15min":3.8519999999999999},"max":{"1min":22.216000000000001,"5min":27.033000000000001,"15min":46.947000000000003},"last":12.391},{"interface":"front","average":{"1min":11.436,"5min":10.529,"15min":10.420999999999999},"min":{"1min":4.2919999999999998,"5min":4.0650000000000004,"15min":4.0650000000000004},"max":{"1min":18.338999999999999,"5min":33.152000000000001,"15min":45.953000000000003},"last":13.583}]},{"osd":1,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":10.019,"5min":9.423,"15min":11.574999999999999},"min":{"1min":5.6189999999999998,"5min":4.3760000000000003,"15min":4.3760000000000003},"max":{"1min":18.969000000000001,"5min":66.313000000000002,"15min":66.313000000000002},"last":8.0109999999999992},{"interface":"front","average":{"1min":12.763999999999999,"5min":12.657,"15min":13.935},"min":{"1min":6.875,"5min":5.9349999999999996,"15min":5.8540000000000001},"max":{"1min":21.193000000000001,"5min":67.533000000000001,"15min":67.533000000000001},"last":8.9749999999999996}]},{"osd":2,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.625999999999999,"5min":13.667,"15min":18.298999999999999},"min":{"1min":9.8510000000000009,"5min":6.0640000000000001,"15min":6.0640000000000001},"max":{"1min":41.106000000000002,"5min":46.051000000000002,"15min":200.88499999999999},"last":9.8770000000000007},{"interface":"front","average":{"1min":20.809999999999999,"5min":17.507000000000001,"15min":16.818000000000001},"min":{"1min":12.57,"5min":7.8129999999999997,"15min":7.4119999999999999},"max":{"1min":39.863,"5min":45.003999999999998,"15min":48.005000000000003},"last":10.448}]},{"osd":3,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.510000000000002,"5min":14.548999999999999,"15min":14.074},"min":{"1min":9.1600000000000001,"5min":5.9820000000000002,"15min":5.9820000000000002},"max":{"1min":26.93,"5min":36.283999999999999,"15min":36.283999999999999},"last":11.109},{"interface":"front","average":{"1min":19.988,"5min":16.667000000000002,"15min":16.466000000000001},"min":{"1min":11.983000000000001,"5min":7.75,"15min":7.6079999999999997},"max":{"1min":31.489999999999998,"5min":34.317,"15min":47.006999999999998},"last":16.024000000000001}]},{"osd":4,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.172999999999998,"5min":16.164000000000001,"15min":15.500999999999999},"min":{"1min":10.173999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"max":{"1min":27.928000000000001,"5min":49.170000000000002,"15min":49.170000000000002},"last":16.382999999999999},{"interface":"front","average":{"1min":19.071000000000002,"5min":17.736000000000001,"15min":17.052},"min":{"1min":12.409000000000001,"5min":8.9019999999999992,"15min":7.7169999999999996},"max":{"1min":32.399000000000001,"5min":38.573999999999998,"15min":48.988999999999997},"last":13.015000000000001}]},{"osd":5,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":20.074000000000002,"5min":18.266999999999999,"15min":18.411999999999999},"min":{"1min":13.186999999999999,"5min":7.8879999999999999,"15min":7.8879999999999999},"max":{"1min":30.518999999999998,"5min":49.390999999999998,"15min":57.308},"last":17.131},{"interface":"front","average":{"1min":11.132,"5min":10.885,"15min":13.561999999999999},"min":{"1min":5.5380000000000003,"5min":5.5380000000000003,"15min":5.5380000000000003},"max":{"1min":15.542999999999999,"5min":47.415999999999997,"15min":68.463999999999999},"last":5.7089999999999996}]},{"osd":7,"last update":"Wed Dec 6 18:34:18 2023","interfaces":[{"interface":"back","average":{"1min":18.114000000000001,"5min":16.631,"15min":15.811},"min":{"1min":9.1460000000000008,"5min":7.4390000000000001,"15min":7.4390000000000001},"max":{"1min":26.856000000000002,"5min":38.887999999999998,"15min":38.887999999999998},"last":14.321},{"interface":"front","average":{"1min":22.518999999999998,"5min":20.096,"15min":19.210000000000001},"min":{"1min":16.774999999999999,"5min":13.555999999999999,"15min":13.555999999999999},"max":{"1min":34.822000000000003,"5min":34.822000000000003,"15min":34.822000000000003},"last":18.202000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:34:29.073 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-06T18:34:29.786 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:34:29.787 INFO:teuthology.orchestra.run.smithi061.stdout:{"epoch":144,"fsid":"497e3283-cc16-40ff-a72c-a7487474fb4b","created":"2023-12-06T18:26:07.122862+0000","modified":"2023-12-06T18:34:19.529910+0000","last_up_change":"2023-12-06T18:30:04.956811+0000","last_in_change":"2023-12-06T18:33:50.507883+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-06T18:26:52.648377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-12-06T18:27:01.712581+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5e6e17b9-e72e-4e34-a531-0830d659bfe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6805","nonce":281956997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6806","nonce":281956997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6808","nonce":281956997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6807","nonce":281956997}]},"public_addr":"172.21.15.61:6805/281956997","cluster_addr":"172.21.15.61:6806/281956997","heartbeat_back_addr":"172.21.15.61:6808/281956997","heartbeat_front_addr":"172.21.15.61:6807/281956997","state":["exists","up"]},{"osd":1,"uuid":"b78c49b0-360d-4351-b44e-65ccaaeb768f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6809","nonce":3306625622}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6810","nonce":3306625622}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6812","nonce":3306625622}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6811","nonce":3306625622}]},"public_addr":"172.21.15.61:6809/3306625622","cluster_addr":"172.21.15.61:6810/3306625622","heartbeat_back_addr":"172.21.15.61:6812/3306625622","heartbeat_front_addr":"172.21.15.61:6811/3306625622","state":["exists","up"]},{"osd":2,"uuid":"a2af3151-a4da-4c34-8fb6-78abf035785b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6801","nonce":3589270120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6802","nonce":3589270120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6804","nonce":3589270120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6803","nonce":3589270120}]},"public_addr":"172.21.15.61:6801/3589270120","cluster_addr":"172.21.15.61:6802/3589270120","heartbeat_back_addr":"172.21.15.61:6804/3589270120","heartbeat_front_addr":"172.21.15.61:6803/3589270120","state":["exists","up"]},{"osd":3,"uuid":"1c754ed6-c535-454f-b35b-00f62cd6ce5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6813","nonce":2849920842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6814","nonce":2849920842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6816","nonce":2849920842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.61:6815","nonce":2849920842}]},"public_addr":"172.21.15.61:6813/2849920842","cluster_addr":"172.21.15.61:6814/2849920842","heartbeat_back_addr":"172.21.15.61:6816/2849920842","heartbeat_front_addr":"172.21.15.61:6815/2849920842","state":["exists","up"]},{"osd":4,"uuid":"766572af-b9ca-4f38-927d-36f26101ba51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":2778770069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6801","nonce":2778770069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6803","nonce":2778770069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":2778770069}]},"public_addr":"172.21.15.152:6800/2778770069","cluster_addr":"172.21.15.152:6801/2778770069","heartbeat_back_addr":"172.21.15.152:6803/2778770069","heartbeat_front_addr":"172.21.15.152:6802/2778770069","state":["exists","up"]},{"osd":5,"uuid":"7a566b60-1543-41d7-894c-e2fbb63e48dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6809","nonce":263671335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":263671335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":263671335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":263671335}]},"public_addr":"172.21.15.152:6809/263671335","cluster_addr":"172.21.15.152:6810/263671335","heartbeat_back_addr":"172.21.15.152:6814/263671335","heartbeat_front_addr":"172.21.15.152:6812/263671335","state":["exists","up"]},{"osd":6,"uuid":"28cc2358-4ac4-40b2-91a4-20edfe625a73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":4245639079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6811","nonce":4245639079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6815","nonce":4245639079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6813","nonce":4245639079}]},"public_addr":"172.21.15.152:6808/4245639079","cluster_addr":"172.21.15.152:6811/4245639079","heartbeat_back_addr":"172.21.15.152:6815/4245639079","heartbeat_front_addr":"172.21.15.152:6813/4245639079","state":["exists","up"]},{"osd":7,"uuid":"8f79279b-d2d0-438e-b3c2-88df1241ff30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":37,"up_from":61,"up_thru":143,"down_at":38,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":3075390540}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6805","nonce":3075390540}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6807","nonce":3075390540}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":3075390540}]},"public_addr":"172.21.15.152:6804/3075390540","cluster_addr":"172.21.15.152:6805/3075390540","heartbeat_back_addr":"172.21.15.152:6807/3075390540","heartbeat_front_addr":"172.21.15.152:6806/3075390540","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:52.171904+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:52.428840+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:51.681954+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:53.687417+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:50.158682+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:50.443807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:49.750276+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-06T18:26:49.770987+0000","dead_epoch":38}],"pg_upmap":[{"pgid":"2.7","osds":[5,3]}],"pg_upmap_items":[{"pgid":"1.0","mappings":[{"from":1,"to":0}]},{"pgid":"2.3","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-06T18:34:30.800 INFO:tasks.ceph:Scrubbing osd.0 2023-12-06T18:34:30.801 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-12-06T18:34:31.007 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:34:31.007 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-06T18:34:31.008 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:34:31.021 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-12-06T18:34:31.641 INFO:teuthology.orchestra.run.smithi061.stderr:instructed osd(s) 0 to deep-scrub 2023-12-06T18:34:31.658 INFO:tasks.ceph:Scrubbing osd.1 2023-12-06T18:34:31.658 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-12-06T18:34:31.861 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:34:31.861 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-06T18:34:31.862 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:34:31.872 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-12-06T18:34:32.471 INFO:teuthology.orchestra.run.smithi061.stderr:instructed osd(s) 1 to deep-scrub 2023-12-06T18:34:32.489 INFO:tasks.ceph:Scrubbing osd.2 2023-12-06T18:34:32.489 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-12-06T18:34:32.687 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:34:32.688 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-06T18:34:32.688 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:34:32.698 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-12-06T18:34:33.295 INFO:teuthology.orchestra.run.smithi061.stderr:instructed osd(s) 2 to deep-scrub 2023-12-06T18:34:33.312 INFO:tasks.ceph:Scrubbing osd.3 2023-12-06T18:34:33.313 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-12-06T18:34:33.560 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:34:33.561 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-06T18:34:33.561 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:34:33.571 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-12-06T18:34:34.152 INFO:teuthology.orchestra.run.smithi061.stderr:instructed osd(s) 3 to deep-scrub 2023-12-06T18:34:34.165 INFO:tasks.ceph:Scrubbing osd.4 2023-12-06T18:34:34.165 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-12-06T18:34:34.368 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:34:34.369 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-06T18:34:34.369 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:34:34.379 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-12-06T18:34:35.001 INFO:teuthology.orchestra.run.smithi061.stderr:instructed osd(s) 4 to deep-scrub 2023-12-06T18:34:35.014 INFO:tasks.ceph:Scrubbing osd.5 2023-12-06T18:34:35.014 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-12-06T18:34:35.219 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:34:35.219 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-06T18:34:35.220 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:34:35.232 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-12-06T18:34:35.793 INFO:teuthology.orchestra.run.smithi061.stderr:instructed osd(s) 5 to deep-scrub 2023-12-06T18:34:35.807 INFO:tasks.ceph:Scrubbing osd.6 2023-12-06T18:34:35.807 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-12-06T18:34:36.017 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:34:36.017 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-06T18:34:36.017 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:34:36.028 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-12-06T18:34:36.611 INFO:teuthology.orchestra.run.smithi061.stderr:instructed osd(s) 6 to deep-scrub 2023-12-06T18:34:36.625 INFO:tasks.ceph:Scrubbing osd.7 2023-12-06T18:34:36.625 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-12-06T18:34:36.829 INFO:teuthology.orchestra.run.smithi061.stdout:{ 2023-12-06T18:34:36.829 INFO:teuthology.orchestra.run.smithi061.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-06T18:34:36.829 INFO:teuthology.orchestra.run.smithi061.stdout:} 2023-12-06T18:34:36.844 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-12-06T18:34:37.493 INFO:teuthology.orchestra.run.smithi061.stderr:instructed osd(s) 7 to deep-scrub 2023-12-06T18:34:37.506 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:34:38.101 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:34:38.102 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:34:38.114 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2023-12-06T18:34:36.293622+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231752,"kb_used_data":10412,"kb_used_omap":15,"kb_used_meta":217008,"kb_avail":754742968,"statfs":{"total":773094113280,"available":772856799232,"internally_reserved":0,"allocated":10661888,"data_stored":4324567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15557,"internal_metadata":222217019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":160,"apply_latency_ms":160,"commit_latency_ns":160000000,"apply_latency_ns":160000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001958"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.633082+0000","last_change":"2023-12-06T18:29:09.051293+0000","last_active":"2023-12-06T18:34:18.633082+0000","last_peered":"2023-12-06T18:34:18.633082+0000","last_clean":"2023-12-06T18:34:18.633082+0000","last_became_active":"2023-12-06T18:29:09.045331+0000","last_became_peered":"2023-12-06T18:29:09.045331+0000","last_unstale":"2023-12-06T18:34:18.633082+0000","last_undegraded":"2023-12-06T18:34:18.633082+0000","last_fullsized":"2023-12-06T18:34:18.633082+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T02:46:01.854508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":51,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:19.332340+0000","last_change":"2023-12-06T18:33:00.623558+0000","last_active":"2023-12-06T18:34:19.332340+0000","last_peered":"2023-12-06T18:34:19.332340+0000","last_clean":"2023-12-06T18:34:19.332340+0000","last_became_active":"2023-12-06T18:33:00.616353+0000","last_became_peered":"2023-12-06T18:33:00.616353+0000","last_unstale":"2023-12-06T18:34:19.332340+0000","last_undegraded":"2023-12-06T18:34:19.332340+0000","last_fullsized":"2023-12-06T18:34:19.332340+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T18:33:31.069235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.603990+0000","last_change":"2023-12-06T18:33:00.592340+0000","last_active":"2023-12-06T18:34:18.603990+0000","last_peered":"2023-12-06T18:34:18.603990+0000","last_clean":"2023-12-06T18:34:18.603990+0000","last_became_active":"2023-12-06T18:33:00.581037+0000","last_became_peered":"2023-12-06T18:33:00.581037+0000","last_unstale":"2023-12-06T18:34:18.603990+0000","last_undegraded":"2023-12-06T18:34:18.603990+0000","last_fullsized":"2023-12-06T18:34:18.603990+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T06:13:54.498147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":118,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.631290+0000","last_change":"2023-12-06T18:33:52.522397+0000","last_active":"2023-12-06T18:34:18.631290+0000","last_peered":"2023-12-06T18:34:18.631290+0000","last_clean":"2023-12-06T18:34:18.631290+0000","last_became_active":"2023-12-06T18:33:52.516242+0000","last_became_peered":"2023-12-06T18:33:52.516242+0000","last_unstale":"2023-12-06T18:34:18.631290+0000","last_undegraded":"2023-12-06T18:34:18.631290+0000","last_fullsized":"2023-12-06T18:34:18.631290+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T03:19:25.211436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.599973+0000","last_change":"2023-12-06T18:33:52.521292+0000","last_active":"2023-12-06T18:34:18.599973+0000","last_peered":"2023-12-06T18:34:18.599973+0000","last_clean":"2023-12-06T18:34:18.599973+0000","last_became_active":"2023-12-06T18:33:52.514832+0000","last_became_peered":"2023-12-06T18:33:52.514832+0000","last_unstale":"2023-12-06T18:34:18.599973+0000","last_undegraded":"2023-12-06T18:34:18.599973+0000","last_fullsized":"2023-12-06T18:34:18.599973+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T23:51:07.302977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":94,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-12-06T18:34:19.744131+0000","last_change":"2023-12-06T18:34:19.744131+0000","last_active":"2023-12-06T18:34:19.744131+0000","last_peered":"2023-12-06T18:34:19.744131+0000","last_clean":"2023-12-06T18:34:19.744131+0000","last_became_active":"2023-12-06T18:34:19.738308+0000","last_became_peered":"2023-12-06T18:34:19.738308+0000","last_unstale":"2023-12-06T18:34:19.744131+0000","last_undegraded":"2023-12-06T18:34:19.744131+0000","last_fullsized":"2023-12-06T18:34:19.744131+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T01:32:29.556210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":142,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-12-06T18:34:18.597827+0000","last_change":"2023-12-06T18:28:47.876556+0000","last_active":"2023-12-06T18:34:18.597827+0000","last_peered":"2023-12-06T18:34:18.597827+0000","last_clean":"2023-12-06T18:34:18.597827+0000","last_became_active":"2023-12-06T18:28:47.865169+0000","last_became_peered":"2023-12-06T18:28:47.865169+0000","last_unstale":"2023-12-06T18:34:18.597827+0000","last_undegraded":"2023-12-06T18:34:18.597827+0000","last_fullsized":"2023-12-06T18:34:18.597827+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:27:02.644707+0000","last_clean_scrub_stamp":"2023-12-06T18:27:02.644707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T04:16:40.095542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":81,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-12-06T18:34:35.290539+0000","last_change":"2023-12-06T18:34:35.297383+0000","last_active":"2023-12-06T18:34:35.290539+0000","last_peered":"2023-12-06T18:34:35.290539+0000","last_clean":"2023-12-06T18:34:35.290539+0000","last_became_active":"2023-12-06T18:31:57.175444+0000","last_became_peered":"2023-12-06T18:31:57.175444+0000","last_unstale":"2023-12-06T18:34:35.290539+0000","last_undegraded":"2023-12-06T18:34:35.290539+0000","last_fullsized":"2023-12-06T18:34:35.290539+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:34:35.286454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:34:35.286454+0000","last_clean_scrub_stamp":"2023-12-06T18:34:35.286454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T23:17:34.764028+0000","scrub_duration":0.32877500999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"144'78","reported_seq":60,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-12-06T18:34:33.102577+0000","last_change":"2023-12-06T18:34:33.110314+0000","last_active":"2023-12-06T18:34:33.102577+0000","last_peered":"2023-12-06T18:34:33.102577+0000","last_clean":"2023-12-06T18:34:33.102577+0000","last_became_active":"2023-12-06T18:33:21.485790+0000","last_became_peered":"2023-12-06T18:33:21.485790+0000","last_unstale":"2023-12-06T18:34:33.102577+0000","last_undegraded":"2023-12-06T18:34:33.102577+0000","last_fullsized":"2023-12-06T18:34:33.102577+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"144'78","last_scrub_stamp":"2023-12-06T18:34:33.098287+0000","last_deep_scrub":"144'78","last_deep_scrub_stamp":"2023-12-06T18:34:33.098287+0000","last_clean_scrub_stamp":"2023-12-06T18:34:33.098287+0000","objects_scrubbed":2,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:41:35.343464+0000","scrub_duration":0.67993020400000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1605632,"data_stored":1591360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411423,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":11.292,"5min":11.291,"15min":10.609999999999999},"min":{"1min":5.069,"5min":3.835,"15min":3.835},"max":{"1min":27.620000000000001,"5min":108.57599999999999,"15min":108.57599999999999},"last":6.8120000000000003},{"interface":"front","average":{"1min":11.958,"5min":11.699,"15min":12.119999999999999},"min":{"1min":6.3239999999999998,"5min":4.8639999999999999,"15min":3.7200000000000002},"max":{"1min":24.844000000000001,"5min":128.09399999999999,"15min":128.09399999999999},"last":15.728}]},{"osd":1,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":11.891999999999999,"5min":12.403,"15min":12.693},"min":{"1min":6.5359999999999996,"5min":6.2539999999999996,"15min":6.2539999999999996},"max":{"1min":19.538,"5min":70.072000000000003,"15min":70.072000000000003},"last":7.585},{"interface":"front","average":{"1min":12.019,"5min":12.824,"15min":14.788},"min":{"1min":6.2080000000000002,"5min":6.2080000000000002,"15min":6.2080000000000002},"max":{"1min":30.16,"5min":30.16,"15min":32.920000000000002},"last":10.654}]},{"osd":2,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":15.49,"5min":15.119,"15min":14.845000000000001},"min":{"1min":9.4450000000000003,"5min":8.5079999999999991,"15min":7.5149999999999997},"max":{"1min":52.232999999999997,"5min":75.125,"15min":75.125},"last":10.289},{"interface":"front","average":{"1min":18.545000000000002,"5min":16.343,"15min":17.477},"min":{"1min":9.2599999999999998,"5min":7.5389999999999997,"15min":7.5389999999999997},"max":{"1min":54.323999999999998,"5min":69.403999999999996,"15min":69.403999999999996},"last":12.646000000000001}]},{"osd":4,"last update":"Wed Dec 6 18:34:13 2023","interfaces":[{"interface":"back","average":{"1min":18.949999999999999,"5min":15.185,"15min":12.898999999999999},"min":{"1min":10.714,"5min":5.8879999999999999,"15min":5.8879999999999999},"max":{"1min":56.255000000000003,"5min":56.255000000000003,"15min":56.255000000000003},"last":11.318},{"interface":"front","average":{"1min":14.9,"5min":13.204000000000001,"15min":15.038},"min":{"1min":6.4189999999999996,"5min":6.3099999999999996,"15min":6.3099999999999996},"max":{"1min":54.616,"5min":71.718000000000004,"15min":71.718000000000004},"last":16.616}]},{"osd":5,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":17.082000000000001,"5min":18.359999999999999,"15min":18.027999999999999},"min":{"1min":11.888,"5min":10.417999999999999,"15min":8.3179999999999996},"max":{"1min":24.445,"5min":75.751999999999995,"15min":75.751999999999995},"last":13.324},{"interface":"front","average":{"1min":16.565999999999999,"5min":16.934000000000001,"15min":17.869},"min":{"1min":8.5389999999999997,"5min":7.9269999999999996,"15min":7.9269999999999996},"max":{"1min":32.219000000000001,"5min":70.933000000000007,"15min":70.933000000000007},"last":13.935}]},{"osd":6,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":19.771000000000001,"5min":15.018000000000001,"15min":15.869999999999999},"min":{"1min":10.798,"5min":8.2050000000000001,"15min":7.0919999999999996},"max":{"1min":30.991,"5min":30.991,"15min":30.991},"last":17.538},{"interface":"front","average":{"1min":16.899999999999999,"5min":18.946999999999999,"15min":20.498000000000001},"min":{"1min":7.2210000000000001,"5min":7.2210000000000001,"15min":7.2210000000000001},"max":{"1min":30.404,"5min":74.224000000000004,"15min":74.224000000000004},"last":8.1839999999999993}]},{"osd":7,"last update":"Wed Dec 6 18:34:13 2023","interfaces":[{"interface":"back","average":{"1min":20.172999999999998,"5min":19.478999999999999,"15min":21.131},"min":{"1min":12.255000000000001,"5min":9.1859999999999999,"15min":9.1859999999999999},"max":{"1min":30.175000000000001,"5min":122.93000000000001,"15min":122.93000000000001},"last":14.369999999999999},{"interface":"front","average":{"1min":22.155000000000001,"5min":22.471,"15min":23.312999999999999},"min":{"1min":15.08,"5min":13.196,"15min":13.196},"max":{"1min":34.466999999999999,"5min":124.473,"15min":124.473},"last":18.385999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476831,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":14.244999999999999,"5min":11.606999999999999,"15min":11.34},"min":{"1min":5.234,"5min":4.1639999999999997,"15min":4.1639999999999997},"max":{"1min":47.884999999999998,"5min":47.884999999999998,"15min":47.884999999999998},"last":7.3049999999999997},{"interface":"front","average":{"1min":12.324999999999999,"5min":12.346,"15min":11.509},"min":{"1min":7.1399999999999997,"5min":4.3280000000000003,"15min":4.3280000000000003},"max":{"1min":48.305999999999997,"5min":48.305999999999997,"15min":48.305999999999997},"last":8.1359999999999992}]},{"osd":1,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":14.621,"5min":12.172000000000001,"15min":11.682},"min":{"1min":6.8860000000000001,"5min":5.9139999999999997,"15min":5.9139999999999997},"max":{"1min":28.242999999999999,"5min":42.920999999999999,"15min":42.920999999999999},"last":70.873999999999995},{"interface":"front","average":{"1min":17.202999999999999,"5min":14.548,"15min":14.061999999999999},"min":{"1min":7.8840000000000003,"5min":6.1479999999999997,"15min":6.1479999999999997},"max":{"1min":29.478999999999999,"5min":34.607999999999997,"15min":34.607999999999997},"last":140.452}]},{"osd":3,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":16.474,"5min":15.353,"15min":17.821999999999999},"min":{"1min":9.0139999999999993,"5min":7.2549999999999999,"15min":6.1260000000000003},"max":{"1min":37.463000000000001,"5min":49.609000000000002,"15min":112.129},"last":12.648},{"interface":"front","average":{"1min":21.956,"5min":17.753,"15min":21.963999999999999},"min":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"max":{"1min":36.770000000000003,"5min":61.243000000000002,"15min":181.601},"last":11.917}]},{"osd":4,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":19.707999999999998,"5min":16.417999999999999,"15min":17.349},"min":{"1min":8.5250000000000004,"5min":7.8070000000000004,"15min":6.609},"max":{"1min":35.396999999999998,"5min":60.332000000000001,"15min":97.088999999999999},"last":13.436999999999999},{"interface":"front","average":{"1min":13.917999999999999,"5min":13.611000000000001,"15min":18.606999999999999},"min":{"1min":7.7640000000000002,"5min":6.0720000000000001,"15min":6.0720000000000001},"max":{"1min":24.061,"5min":43.198,"15min":98.069000000000003},"last":14.686}]},{"osd":5,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":19.113,"5min":18.113,"15min":18.097000000000001},"min":{"1min":10.743,"5min":8.2100000000000009,"15min":7.8170000000000002},"max":{"1min":26.042000000000002,"5min":57.628999999999998,"15min":57.628999999999998},"last":15.291},{"interface":"front","average":{"1min":15.692,"5min":15.933,"15min":15.824},"min":{"1min":10.292,"5min":8.5990000000000002,"15min":8.5990000000000002},"max":{"1min":39.875999999999998,"5min":47.749000000000002,"15min":47.749000000000002},"last":14.042}]},{"osd":6,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":20.952999999999999,"5min":19.140000000000001,"15min":36.003999999999998},"min":{"1min":7.0419999999999998,"5min":7.0419999999999998,"15min":7.0419999999999998},"max":{"1min":47.667999999999999,"5min":64.034999999999997,"15min":498.38499999999999},"last":15.890000000000001},{"interface":"front","average":{"1min":24.446000000000002,"5min":20.491,"15min":37.847000000000001},"min":{"1min":12.461,"5min":9.7010000000000005,"15min":9.7010000000000005},"max":{"1min":44.993000000000002,"5min":54.920000000000002,"15min":496.17399999999998},"last":20.099}]},{"osd":7,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":21.704999999999998,"5min":19.547000000000001,"15min":19.131},"min":{"1min":13.574999999999999,"5min":10.064,"15min":10.064},"max":{"1min":35.945,"5min":49.456000000000003,"15min":49.456000000000003},"last":20.832000000000001},{"interface":"front","average":{"1min":21.277999999999999,"5min":20.684000000000001,"15min":20.332000000000001},"min":{"1min":12.81,"5min":12.288,"15min":12.288},"max":{"1min":41.965000000000003,"5min":41.965000000000003,"15min":41.965000000000003},"last":18.861999999999998}]}]},{"osd":1,"up_from":16,"seq":68719476832,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":73,"apply_latency_ms":73,"commit_latency_ns":73000000,"apply_latency_ns":73000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":12.32,"5min":11.260999999999999,"15min":18.989000000000001},"min":{"1min":6.0650000000000004,"5min":5.0890000000000004,"15min":5.0890000000000004},"max":{"1min":34.816000000000003,"5min":34.816000000000003,"15min":238.87100000000001},"last":56.762999999999998},{"interface":"front","average":{"1min":11.336,"5min":10.768000000000001,"15min":34.408000000000001},"min":{"1min":5.0629999999999997,"5min":5.0629999999999997,"15min":4.4269999999999996},"max":{"1min":22.834,"5min":61.262,"15min":661.33600000000001},"last":57.759}]},{"osd":2,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.962999999999999,"5min":13.722,"15min":27.556999999999999},"min":{"1min":6.6189999999999998,"5min":6.6189999999999998,"15min":6.6189999999999998},"max":{"1min":57.93,"5min":93.168000000000006,"15min":384.315},"last":8.827},{"interface":"front","average":{"1min":22.902000000000001,"5min":19.120999999999999,"15min":39.317999999999998},"min":{"1min":12.282,"5min":10.724,"15min":5.7009999999999996},"max":{"1min":60.173000000000002,"5min":94.230000000000004,"15min":656.98699999999997},"last":18.375}]},{"osd":3,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":12.403,"5min":12.194000000000001,"15min":12.863},"min":{"1min":6.9180000000000001,"5min":5.9720000000000004,"15min":5.548},"max":{"1min":25.774999999999999,"5min":31.16,"15min":34.057000000000002},"last":12.537000000000001},{"interface":"front","average":{"1min":14.6,"5min":14.064,"15min":15.500999999999999},"min":{"1min":8.0180000000000007,"5min":6.5730000000000004,"15min":6.5730000000000004},"max":{"1min":31.417999999999999,"5min":65.537000000000006,"15min":65.537000000000006},"last":13.15}]},{"osd":4,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.082000000000001,"5min":13.308,"15min":33.085999999999999},"min":{"1min":8.6059999999999999,"5min":6.7009999999999996,"15min":6.2930000000000001},"max":{"1min":33.427999999999997,"5min":33.427999999999997,"15min":447.67200000000003},"last":15.069000000000001},{"interface":"front","average":{"1min":21.420000000000002,"5min":18.047999999999998,"15min":39.893000000000001},"min":{"1min":7.7300000000000004,"5min":7.7300000000000004,"15min":7.0919999999999996},"max":{"1min":40.564,"5min":44.043999999999997,"15min":630.572},"last":9.2769999999999992}]},{"osd":5,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.390000000000001,"5min":16.076000000000001,"15min":35.482999999999997},"min":{"1min":7.2549999999999999,"5min":7.0540000000000003,"15min":7.0540000000000003},"max":{"1min":38.991999999999997,"5min":68.013999999999996,"15min":389.55000000000001},"last":8.077},{"interface":"front","average":{"1min":22.140999999999998,"5min":18.395,"15min":45.957999999999998},"min":{"1min":11.099,"5min":7.5359999999999996,"15min":7.0410000000000004},"max":{"1min":41.655999999999999,"5min":41.655999999999999,"15min":658.48599999999999},"last":17.393999999999998}]},{"osd":6,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.672000000000001,"5min":15.733000000000001,"15min":30.202000000000002},"min":{"1min":8.9760000000000009,"5min":7.5899999999999999,"15min":7.5899999999999999},"max":{"1min":32.039999999999999,"5min":54.383000000000003,"15min":439.56400000000002},"last":9.9299999999999997},{"interface":"front","average":{"1min":23.643000000000001,"5min":20.5,"15min":32.639000000000003},"min":{"1min":13.058,"5min":10.641999999999999,"15min":10.641999999999999},"max":{"1min":43.052,"5min":55.423999999999999,"15min":386.71499999999997},"last":16.408000000000001}]},{"osd":7,"last update":"Wed Dec 6 18:34:16 2023","interfaces":[{"interface":"back","average":{"1min":18.527000000000001,"5min":18.222999999999999,"15min":18.600000000000001},"min":{"1min":8.9849999999999994,"5min":8.9849999999999994,"15min":8.9849999999999994},"max":{"1min":33.156999999999996,"5min":97.561999999999998,"15min":97.561999999999998},"last":10.614000000000001},{"interface":"front","average":{"1min":23.125,"5min":21.024000000000001,"15min":18.771999999999998},"min":{"1min":12.019,"5min":8.7219999999999995,"15min":8.7219999999999995},"max":{"1min":36.332000000000001,"5min":101.762,"15min":101.762},"last":19.989999999999998}]}]},{"osd":0,"up_from":16,"seq":68719476833,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343228,"statfs":{"total":96636764160,"available":96607465472,"internally_reserved":0,"allocated":1634304,"data_stored":838986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":71,"apply_latency_ms":71,"commit_latency_ns":71000000,"apply_latency_ns":71000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":13.518000000000001,"5min":11.452,"15min":15.608000000000001},"min":{"1min":5.6340000000000003,"5min":3.2360000000000002,"15min":3.2360000000000002},"max":{"1min":54.155000000000001,"5min":54.155000000000001,"15min":116.593},"last":5.1029999999999998},{"interface":"front","average":{"1min":13.859,"5min":12.935,"15min":16.576000000000001},"min":{"1min":5.5389999999999997,"5min":4.9039999999999999,"15min":4.9039999999999999},"max":{"1min":68.778999999999996,"5min":68.778999999999996,"15min":117.896},"last":7.3810000000000002}]},{"osd":2,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":17.434000000000001,"5min":14.888,"15min":14.132999999999999},"min":{"1min":8.9870000000000001,"5min":5.694,"15min":5.694},"max":{"1min":72.131,"5min":72.131,"15min":72.131},"last":13.055},{"interface":"front","average":{"1min":14.316000000000001,"5min":13.224,"15min":12.055},"min":{"1min":6.7590000000000003,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":59.317999999999998,"5min":59.317999999999998,"15min":59.317999999999998},"last":6.524}]},{"osd":3,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":19.911999999999999,"5min":16.183,"15min":16.312999999999999},"min":{"1min":8.9600000000000009,"5min":6.1379999999999999,"15min":6.1379999999999999},"max":{"1min":75.439999999999998,"5min":75.439999999999998,"15min":75.439999999999998},"last":14.747999999999999},{"interface":"front","average":{"1min":19.082000000000001,"5min":16.047000000000001,"15min":16.204999999999998},"min":{"1min":7.593,"5min":7.5510000000000002,"15min":7.5510000000000002},"max":{"1min":96.688999999999993,"5min":96.688999999999993,"15min":96.688999999999993},"last":11.581}]},{"osd":4,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":14.302,"5min":12.134,"15min":17.890999999999998},"min":{"1min":6.8239999999999998,"5min":5.9349999999999996,"15min":5.9349999999999996},"max":{"1min":57.694000000000003,"5min":57.694000000000003,"15min":82.432000000000002},"last":16.701000000000001},{"interface":"front","average":{"1min":19.693000000000001,"5min":18.327000000000002,"15min":19.846},"min":{"1min":7.7469999999999999,"5min":7.7469999999999999,"15min":6.2629999999999999},"max":{"1min":76.078000000000003,"5min":76.078000000000003,"15min":81.239999999999995},"last":10.548}]},{"osd":5,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":19.27,"5min":16.798999999999999,"15min":17.507000000000001},"min":{"1min":8.1229999999999993,"5min":8.0099999999999998,"15min":8.0099999999999998},"max":{"1min":97.947000000000003,"5min":97.947000000000003,"15min":97.947000000000003},"last":17.404},{"interface":"front","average":{"1min":16.989000000000001,"5min":15.443,"15min":17.440000000000001},"min":{"1min":7.4770000000000003,"5min":7.1779999999999999,"15min":7.1779999999999999},"max":{"1min":69.915999999999997,"5min":69.915999999999997,"15min":72.307000000000002},"last":20.620000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":25.541,"5min":20.477,"15min":19.626000000000001},"min":{"1min":12.661,"5min":11.401,"15min":7.7560000000000002},"max":{"1min":82.775000000000006,"5min":82.775000000000006,"15min":82.775000000000006},"last":18.527999999999999},{"interface":"front","average":{"1min":21.75,"5min":17.940999999999999,"15min":18.914999999999999},"min":{"1min":9.7729999999999997,"5min":9.3520000000000003,"15min":9.3520000000000003},"max":{"1min":74.057000000000002,"5min":74.057000000000002,"15min":74.057000000000002},"last":13.73}]},{"osd":7,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":26.137,"5min":21.724,"15min":19.988},"min":{"1min":16.024000000000001,"5min":12.009,"15min":12.009},"max":{"1min":95.287999999999997,"5min":95.287999999999997,"15min":95.287999999999997},"last":19.556999999999999},{"interface":"front","average":{"1min":23.494,"5min":20.041,"15min":18.611000000000001},"min":{"1min":10.24,"5min":10.24,"15min":10.24},"max":{"1min":98.995999999999995,"5min":98.995999999999995,"15min":98.995999999999995},"last":15.390000000000001}]}]},{"osd":5,"up_from":15,"seq":64424509537,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343228,"statfs":{"total":96636764160,"available":96607465472,"internally_reserved":0,"allocated":1634304,"data_stored":838986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":14.44,"5min":9.6449999999999996,"15min":13.73},"min":{"1min":4.0430000000000001,"5min":3.3660000000000001,"15min":3.3660000000000001},"max":{"1min":106.521,"5min":106.521,"15min":208.291},"last":5.6639999999999997},{"interface":"front","average":{"1min":14.817,"5min":11.609,"15min":17.067},"min":{"1min":4.984,"5min":4.758,"15min":4.4299999999999997},"max":{"1min":107.342,"5min":107.342,"15min":199.94900000000001},"last":11.127000000000001}]},{"osd":1,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":11.631,"5min":13.298999999999999,"15min":45.225999999999999},"min":{"1min":6.7469999999999999,"5min":5.085,"15min":5.085},"max":{"1min":26.692,"5min":41.234999999999999,"15min":1211.6010000000001},"last":12.446999999999999},{"interface":"front","average":{"1min":12.16,"5min":10.502000000000001,"15min":41.011000000000003},"min":{"1min":5.359,"5min":4.7199999999999998,"15min":4.7199999999999998},"max":{"1min":22.664000000000001,"5min":39.802999999999997,"15min":1189.3869999999999},"last":8.9049999999999994}]},{"osd":2,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":14.343,"5min":14.629,"15min":21.640999999999998},"min":{"1min":9.0389999999999997,"5min":6.5129999999999999,"15min":6.5129999999999999},"max":{"1min":23.422000000000001,"5min":68.617999999999995,"15min":283.46800000000002},"last":13.037000000000001},{"interface":"front","average":{"1min":19.576000000000001,"5min":18.463999999999999,"15min":25.539000000000001},"min":{"1min":12.851000000000001,"5min":8.173,"15min":8.173},"max":{"1min":40.737000000000002,"5min":69.513999999999996,"15min":296.47800000000001},"last":15.102}]},{"osd":3,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":11.484999999999999,"5min":13.811,"15min":17.414000000000001},"min":{"1min":7.3639999999999999,"5min":6.0300000000000002,"15min":6.0300000000000002},"max":{"1min":19.872,"5min":36.156999999999996,"15min":47.926000000000002},"last":11.821},{"interface":"front","average":{"1min":15.419,"5min":14.555999999999999,"15min":15.007999999999999},"min":{"1min":10.163,"5min":5.3979999999999997,"15min":5.3979999999999997},"max":{"1min":28.914999999999999,"5min":34.609000000000002,"15min":66.176000000000002},"last":13.621}]},{"osd":4,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":18.212,"5min":17.952000000000002,"15min":19.096},"min":{"1min":11.897,"5min":9.7210000000000001,"15min":9.7210000000000001},"max":{"1min":34.069000000000003,"5min":41.923999999999999,"15min":67.248000000000005},"last":17.623999999999999},{"interface":"front","average":{"1min":11.249000000000001,"5min":10.798,"15min":12.347},"min":{"1min":6.5279999999999996,"5min":5.3300000000000001,"15min":5.3300000000000001},"max":{"1min":26.850999999999999,"5min":43.101999999999997,"15min":77.266999999999996},"last":6.883}]},{"osd":6,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":15.667999999999999,"5min":13.644,"15min":15.140000000000001},"min":{"1min":7.984,"5min":6.1280000000000001,"15min":5.915},"max":{"1min":25.727,"5min":44.587000000000003,"15min":76.751999999999995},"last":15.978999999999999},{"interface":"front","average":{"1min":20.158999999999999,"5min":19.356999999999999,"15min":20.469000000000001},"min":{"1min":11.744999999999999,"5min":11.744999999999999,"15min":8.8390000000000004},"max":{"1min":36.883000000000003,"5min":45.366,"15min":57.101999999999997},"last":18.379000000000001}]},{"osd":7,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":18.687999999999999,"5min":16.294,"15min":14.904999999999999},"min":{"1min":11.298999999999999,"5min":7.5519999999999996,"15min":7.5519999999999996},"max":{"1min":32.219999999999999,"5min":32.219999999999999,"15min":32.219999999999999},"last":16.768999999999998},{"interface":"front","average":{"1min":14.401999999999999,"5min":12.872999999999999,"15min":12.215999999999999},"min":{"1min":7.133,"5min":7.133,"15min":7.133},"max":{"1min":29.376999999999999,"5min":29.376999999999999,"15min":29.376999999999999},"last":9.4870000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509538,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1228800,"data_stored":440865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":8.0180000000000007,"5min":9.266,"15min":12.295999999999999},"min":{"1min":4.2789999999999999,"5min":3.7429999999999999,"15min":3.7429999999999999},"max":{"1min":13.465999999999999,"5min":41.076000000000001,"15min":53.298999999999999},"last":73.757999999999996},{"interface":"front","average":{"1min":9.9160000000000004,"5min":10,"15min":10.609999999999999},"min":{"1min":5.1280000000000001,"5min":4.8140000000000001,"15min":3.9670000000000001},"max":{"1min":18.329000000000001,"5min":35.863,"15min":52.567},"last":67.418999999999997}]},{"osd":1,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":12.202999999999999,"5min":11.432,"15min":13.298999999999999},"min":{"1min":6.0529999999999999,"5min":5.8390000000000004,"15min":5.835},"max":{"1min":21.186,"5min":22.509,"15min":49.183},"last":7.1239999999999997},{"interface":"front","average":{"1min":13.43,"5min":12.129,"15min":14.662000000000001},"min":{"1min":5.9770000000000003,"5min":5.9770000000000003,"15min":5.96},"max":{"1min":23.596,"5min":24.081,"15min":45.412999999999997},"last":8.2360000000000007}]},{"osd":2,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":12.398999999999999,"5min":12.423,"15min":19.385000000000002},"min":{"1min":6.609,"5min":5.9539999999999997,"15min":5.6390000000000002},"max":{"1min":22.904,"5min":44.649000000000001,"15min":85.394000000000005},"last":9.5350000000000001},{"interface":"front","average":{"1min":14.196,"5min":15.087999999999999,"15min":18.899000000000001},"min":{"1min":7.9269999999999996,"5min":7.5209999999999999,"15min":6.3639999999999999},"max":{"1min":27.780000000000001,"5min":42.500999999999998,"15min":69.530000000000001},"last":10.484}]},{"osd":3,"last update":"Wed Dec 6 18:34:17 2023","interfaces":[{"interface":"back","average":{"1min":16.875,"5min":15.66,"15min":14.851000000000001},"min":{"1min":8.6349999999999998,"5min":8.0250000000000004,"15min":7.008},"max":{"1min":35.418999999999997,"5min":37.561999999999998,"15min":37.561999999999998},"last":14.808},{"interface":"front","average":{"1min":18.140000000000001,"5min":14.212,"15min":14.936},"min":{"1min":8.0619999999999994,"5min":5.5549999999999997,"15min":5.5549999999999997},"max":{"1min":36.694000000000003,"5min":36.694000000000003,"15min":36.694000000000003},"last":13.151999999999999}]},{"osd":5,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":16.562000000000001,"5min":13.952999999999999,"15min":16.228999999999999},"min":{"1min":6.9370000000000003,"5min":5.4969999999999999,"15min":5.4969999999999999},"max":{"1min":27.283000000000001,"5min":53.840000000000003,"15min":78.798000000000002},"last":11.087},{"interface":"front","average":{"1min":16.690000000000001,"5min":17.414000000000001,"15min":18.402000000000001},"min":{"1min":10.425000000000001,"5min":8.0519999999999996,"15min":6.4450000000000003},"max":{"1min":29.184000000000001,"5min":52.781999999999996,"15min":55.814999999999998},"last":16.187999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":19,"5min":18.105,"15min":18.045000000000002},"min":{"1min":9.3170000000000002,"5min":9.2100000000000009,"15min":7.7850000000000001},"max":{"1min":32.869,"5min":41.723999999999997,"15min":51.149000000000001},"last":17.794},{"interface":"front","average":{"1min":16.550000000000001,"5min":15.154999999999999,"15min":16.164999999999999},"min":{"1min":7.9359999999999999,"5min":7.9359999999999999,"15min":7.8700000000000001},"max":{"1min":29.792000000000002,"5min":34.929000000000002,"15min":40.426000000000002},"last":12.209}]},{"osd":7,"last update":"Wed Dec 6 18:34:17 2023","interfaces":[{"interface":"back","average":{"1min":17.311,"5min":17.135000000000002,"15min":18.212},"min":{"1min":11.176,"5min":8.093,"15min":8.093},"max":{"1min":24.667000000000002,"5min":60.390000000000001,"15min":60.390000000000001},"last":14.115},{"interface":"front","average":{"1min":21.228999999999999,"5min":21.193999999999999,"15min":22.27},"min":{"1min":11.083,"5min":11.083,"15min":11.083},"max":{"1min":33.914999999999999,"5min":57.606999999999999,"15min":57.606999999999999},"last":16.995000000000001}]}]},{"osd":7,"up_from":61,"seq":261993005113,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33412,"kb_used_data":1204,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94338428,"statfs":{"total":96636764160,"available":96602550272,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4395,"internal_metadata":28634837},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.206,"5min":11.058999999999999,"15min":10.446999999999999},"min":{"1min":4.0739999999999998,"5min":4.0739999999999998,"15min":4.0739999999999998},"max":{"1min":126.286,"5min":126.286,"15min":126.286},"last":15.601000000000001},{"interface":"front","average":{"1min":15.504,"5min":11.65,"15min":11.226000000000001},"min":{"1min":5.2320000000000002,"5min":4.2530000000000001,"15min":4.2530000000000001},"max":{"1min":127.35599999999999,"5min":127.35599999999999,"15min":127.35599999999999},"last":18.350000000000001}]},{"osd":1,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":12.824999999999999,"5min":13.972,"15min":16.297000000000001},"min":{"1min":5.9130000000000003,"5min":4.6929999999999996,"15min":4.6929999999999996},"max":{"1min":24.047999999999998,"5min":64.765000000000001,"15min":64.765000000000001},"last":8.7560000000000002},{"interface":"front","average":{"1min":13.289999999999999,"5min":15.44,"15min":17.219999999999999},"min":{"1min":6.2969999999999997,"5min":6.2969999999999997,"15min":6.2969999999999997},"max":{"1min":24.402999999999999,"5min":65.994,"15min":65.994},"last":4.6689999999999996}]},{"osd":2,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.021000000000001,"5min":16.204000000000001,"15min":17.411999999999999},"min":{"1min":7.9569999999999999,"5min":7.9569999999999999,"15min":7.9569999999999999},"max":{"1min":24.219000000000001,"5min":46.478000000000002,"15min":46.478000000000002},"last":11.797000000000001},{"interface":"front","average":{"1min":14.906000000000001,"5min":14.161,"15min":14.132999999999999},"min":{"1min":6.415,"5min":6.415,"15min":6.415},"max":{"1min":23.727,"5min":41.603999999999999,"15min":41.603999999999999},"last":7.7889999999999997}]},{"osd":3,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":17.462,"5min":15.693,"15min":16.739000000000001},"min":{"1min":7.8879999999999999,"5min":6.8319999999999999,"15min":6.8319999999999999},"max":{"1min":47.823,"5min":60.463999999999999,"15min":60.463999999999999},"last":10.701000000000001},{"interface":"front","average":{"1min":20.838999999999999,"5min":18.907,"15min":19.867999999999999},"min":{"1min":9.8889999999999993,"5min":9.7370000000000001,"15min":9.7370000000000001},"max":{"1min":44.863,"5min":56.286000000000001,"15min":56.286000000000001},"last":9.9689999999999994}]},{"osd":4,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":20.128,"5min":20.585000000000001,"15min":21.081},"min":{"1min":10.489000000000001,"5min":8.4960000000000004,"15min":8.4960000000000004},"max":{"1min":32.167999999999999,"5min":57.716999999999999,"15min":57.716999999999999},"last":12.422000000000001},{"interface":"front","average":{"1min":16.364000000000001,"5min":17.821999999999999,"15min":18.356000000000002},"min":{"1min":7.2699999999999996,"5min":7.2699999999999996,"15min":7.2699999999999996},"max":{"1min":30.831,"5min":67.159000000000006,"15min":67.159000000000006},"last":14.880000000000001}]},{"osd":5,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.968999999999999,"5min":14.587999999999999,"15min":14.688000000000001},"min":{"1min":7.4219999999999997,"5min":6.3399999999999999,"15min":6.3399999999999999},"max":{"1min":27.07,"5min":44.884,"15min":44.884},"last":12.98},{"interface":"front","average":{"1min":20.382999999999999,"5min":18.724,"15min":19.527999999999999},"min":{"1min":9.3520000000000003,"5min":9.3520000000000003,"15min":9.3520000000000003},"max":{"1min":92.616,"5min":92.616,"15min":92.616},"last":13.689}]},{"osd":6,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":21.416,"5min":17.657,"15min":17.068999999999999},"min":{"1min":7.625,"5min":7.625,"15min":7.625},"max":{"1min":95.501000000000005,"5min":95.501000000000005,"15min":95.501000000000005},"last":16.303999999999998},{"interface":"front","average":{"1min":23.699999999999999,"5min":20.664999999999999,"15min":20.841000000000001},"min":{"1min":10.811999999999999,"5min":10.144,"15min":10.144},"max":{"1min":94.533000000000001,"5min":94.533000000000001,"15min":94.533000000000001},"last":14.287000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542239,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":9.9879999999999995,"5min":9.4030000000000005,"15min":9.8219999999999992},"min":{"1min":4.3010000000000002,"5min":4.1260000000000003,"15min":3.8519999999999999},"max":{"1min":22.216000000000001,"5min":27.033000000000001,"15min":46.947000000000003},"last":7.226},{"interface":"front","average":{"1min":11.436,"5min":10.529,"15min":10.420999999999999},"min":{"1min":4.2919999999999998,"5min":4.0650000000000004,"15min":4.0650000000000004},"max":{"1min":18.338999999999999,"5min":33.152000000000001,"15min":45.953000000000003},"last":9.3360000000000003}]},{"osd":1,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":10.019,"5min":9.423,"15min":11.574999999999999},"min":{"1min":5.6189999999999998,"5min":4.3760000000000003,"15min":4.3760000000000003},"max":{"1min":18.969000000000001,"5min":66.313000000000002,"15min":66.313000000000002},"last":10.52},{"interface":"front","average":{"1min":12.763999999999999,"5min":12.657,"15min":13.935},"min":{"1min":6.875,"5min":5.9349999999999996,"15min":5.8540000000000001},"max":{"1min":21.193000000000001,"5min":67.533000000000001,"15min":67.533000000000001},"last":16.408000000000001}]},{"osd":2,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.625999999999999,"5min":13.667,"15min":18.298999999999999},"min":{"1min":9.8510000000000009,"5min":6.0640000000000001,"15min":6.0640000000000001},"max":{"1min":41.106000000000002,"5min":46.051000000000002,"15min":200.88499999999999},"last":9.9589999999999996},{"interface":"front","average":{"1min":20.809999999999999,"5min":17.507000000000001,"15min":16.818000000000001},"min":{"1min":12.57,"5min":7.8129999999999997,"15min":7.4119999999999999},"max":{"1min":39.863,"5min":45.003999999999998,"15min":48.005000000000003},"last":11.196999999999999}]},{"osd":3,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.510000000000002,"5min":14.548999999999999,"15min":14.074},"min":{"1min":9.1600000000000001,"5min":5.9820000000000002,"15min":5.9820000000000002},"max":{"1min":26.93,"5min":36.283999999999999,"15min":36.283999999999999},"last":8.0839999999999996},{"interface":"front","average":{"1min":19.988,"5min":16.667000000000002,"15min":16.466000000000001},"min":{"1min":11.983000000000001,"5min":7.75,"15min":7.6079999999999997},"max":{"1min":31.489999999999998,"5min":34.317,"15min":47.006999999999998},"last":12.398}]},{"osd":4,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.172999999999998,"5min":16.164000000000001,"15min":15.500999999999999},"min":{"1min":10.173999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"max":{"1min":27.928000000000001,"5min":49.170000000000002,"15min":49.170000000000002},"last":17.277999999999999},{"interface":"front","average":{"1min":19.071000000000002,"5min":17.736000000000001,"15min":17.052},"min":{"1min":12.409000000000001,"5min":8.9019999999999992,"15min":7.7169999999999996},"max":{"1min":32.399000000000001,"5min":38.573999999999998,"15min":48.988999999999997},"last":12.991}]},{"osd":5,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":20.074000000000002,"5min":18.266999999999999,"15min":18.411999999999999},"min":{"1min":13.186999999999999,"5min":7.8879999999999999,"15min":7.8879999999999999},"max":{"1min":30.518999999999998,"5min":49.390999999999998,"15min":57.308},"last":11.811999999999999},{"interface":"front","average":{"1min":11.132,"5min":10.885,"15min":13.561999999999999},"min":{"1min":5.5380000000000003,"5min":5.5380000000000003,"15min":5.5380000000000003},"max":{"1min":15.542999999999999,"5min":47.415999999999997,"15min":68.463999999999999},"last":6.5060000000000002}]},{"osd":7,"last update":"Wed Dec 6 18:34:18 2023","interfaces":[{"interface":"back","average":{"1min":18.114000000000001,"5min":16.631,"15min":15.811},"min":{"1min":9.1460000000000008,"5min":7.4390000000000001,"15min":7.4390000000000001},"max":{"1min":26.856000000000002,"5min":38.887999999999998,"15min":38.887999999999998},"last":15.429},{"interface":"front","average":{"1min":22.518999999999998,"5min":20.096,"15min":19.210000000000001},"min":{"1min":16.774999999999999,"5min":13.555999999999999,"15min":13.555999999999999},"max":{"1min":34.822000000000003,"5min":34.822000000000003,"15min":34.822000000000003},"last":14.789999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:34:38.120 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-12-06T18:27:02.644707+0000 time.struct_time(tm_year=2023, tm_mon=12, tm_mday=6, tm_hour=18, tm_min=27, tm_sec=2, tm_wday=2, tm_yday=340, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=12, tm_mday=6, tm_hour=18, tm_min=34, tm_sec=29, tm_wday=2, tm_yday=340, tm_isdst=0) 2023-12-06T18:34:38.120 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-12-06T18:27:02.644707+0000 time.struct_time(tm_year=2023, tm_mon=12, tm_mday=6, tm_hour=18, tm_min=27, tm_sec=2, tm_wday=2, tm_yday=340, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=12, tm_mday=6, tm_hour=18, tm_min=34, tm_sec=29, tm_wday=2, tm_yday=340, tm_isdst=0) 2023-12-06T18:34:38.120 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-12-06T18:27:02.644707+0000 time.struct_time(tm_year=2023, tm_mon=12, tm_mday=6, tm_hour=18, tm_min=27, tm_sec=2, tm_wday=2, tm_yday=340, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=12, tm_mday=6, tm_hour=18, tm_min=34, tm_sec=29, tm_wday=2, tm_yday=340, tm_isdst=0) 2023-12-06T18:34:38.120 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-12-06T18:27:02.644707+0000 time.struct_time(tm_year=2023, tm_mon=12, tm_mday=6, tm_hour=18, tm_min=27, tm_sec=2, tm_wday=2, tm_yday=340, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=12, tm_mday=6, tm_hour=18, tm_min=34, tm_sec=29, tm_wday=2, tm_yday=340, tm_isdst=0) 2023-12-06T18:34:38.120 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-12-06T18:27:02.644707+0000 time.struct_time(tm_year=2023, tm_mon=12, tm_mday=6, tm_hour=18, tm_min=27, tm_sec=2, tm_wday=2, tm_yday=340, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=12, tm_mday=6, tm_hour=18, tm_min=34, tm_sec=29, tm_wday=2, tm_yday=340, tm_isdst=0) 2023-12-06T18:34:38.121 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-12-06T18:27:02.644707+0000 time.struct_time(tm_year=2023, tm_mon=12, tm_mday=6, tm_hour=18, tm_min=27, tm_sec=2, tm_wday=2, tm_yday=340, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=12, tm_mday=6, tm_hour=18, tm_min=34, tm_sec=29, tm_wday=2, tm_yday=340, tm_isdst=0) 2023-12-06T18:34:38.121 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-12-06T18:27:02.644707+0000 time.struct_time(tm_year=2023, tm_mon=12, tm_mday=6, tm_hour=18, tm_min=27, tm_sec=2, tm_wday=2, tm_yday=340, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=12, tm_mday=6, tm_hour=18, tm_min=34, tm_sec=29, tm_wday=2, tm_yday=340, tm_isdst=0) 2023-12-06T18:34:38.121 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-12-06T18:34:47.460 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:03:26.905 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:47.465 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:03:26.910 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:47.480 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:03:26.925 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:47.510 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:02:21.101 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:47.512 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:02:21.104 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:47.515 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:15.765 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:47.517 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:15.767 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:47.526 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:02:21.118 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:47.537 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:15.787 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:34:58.122 DEBUG:teuthology.orchestra.run.smithi061:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-06T18:34:58.732 INFO:teuthology.orchestra.run.smithi061.stdout: 2023-12-06T18:34:58.732 INFO:teuthology.orchestra.run.smithi061.stderr:dumped all 2023-12-06T18:34:58.744 INFO:teuthology.orchestra.run.smithi061.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2023-12-06T18:34:58.298397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231728,"kb_used_data":10412,"kb_used_omap":15,"kb_used_meta":217008,"kb_avail":754742992,"statfs":{"total":773094113280,"available":772856823808,"internally_reserved":0,"allocated":10661888,"data_stored":4324567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15557,"internal_metadata":222217019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002706"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":105,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-12-06T18:34:38.424979+0000","last_change":"2023-12-06T18:34:38.425669+0000","last_active":"2023-12-06T18:34:38.424979+0000","last_peered":"2023-12-06T18:34:38.424979+0000","last_clean":"2023-12-06T18:34:38.424979+0000","last_became_active":"2023-12-06T18:29:09.045331+0000","last_became_peered":"2023-12-06T18:29:09.045331+0000","last_unstale":"2023-12-06T18:34:38.424979+0000","last_undegraded":"2023-12-06T18:34:38.424979+0000","last_fullsized":"2023-12-06T18:34:38.424979+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:34:38.423631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:34:38.423631+0000","last_clean_scrub_stamp":"2023-12-06T18:34:38.423631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T00:14:20.920206+0000","scrub_duration":0.087867662999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":60,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-12-06T18:34:36.494176+0000","last_change":"2023-12-06T18:34:36.503783+0000","last_active":"2023-12-06T18:34:36.494176+0000","last_peered":"2023-12-06T18:34:36.494176+0000","last_clean":"2023-12-06T18:34:36.494176+0000","last_became_active":"2023-12-06T18:33:00.616353+0000","last_became_peered":"2023-12-06T18:33:00.616353+0000","last_unstale":"2023-12-06T18:34:36.494176+0000","last_undegraded":"2023-12-06T18:34:36.494176+0000","last_fullsized":"2023-12-06T18:34:36.494176+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:34:36.488516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:34:36.488516+0000","last_clean_scrub_stamp":"2023-12-06T18:34:36.488516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T19:11:39.689072+0000","scrub_duration":0.30687761299999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":151,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-12-06T18:34:47.753942+0000","last_change":"2023-12-06T18:34:47.754819+0000","last_active":"2023-12-06T18:34:47.753942+0000","last_peered":"2023-12-06T18:34:47.753942+0000","last_clean":"2023-12-06T18:34:47.753942+0000","last_became_active":"2023-12-06T18:33:00.581037+0000","last_became_peered":"2023-12-06T18:33:00.581037+0000","last_unstale":"2023-12-06T18:34:47.753942+0000","last_undegraded":"2023-12-06T18:34:47.753942+0000","last_fullsized":"2023-12-06T18:34:47.753942+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:34:47.751692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:34:47.751692+0000","last_clean_scrub_stamp":"2023-12-06T18:34:47.751692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T01:33:58.587604+0000","scrub_duration":0.085265149999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":127,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-12-06T18:34:36.644368+0000","last_change":"2023-12-06T18:34:36.648756+0000","last_active":"2023-12-06T18:34:36.644368+0000","last_peered":"2023-12-06T18:34:36.644368+0000","last_clean":"2023-12-06T18:34:36.644368+0000","last_became_active":"2023-12-06T18:33:52.516242+0000","last_became_peered":"2023-12-06T18:33:52.516242+0000","last_unstale":"2023-12-06T18:34:36.644368+0000","last_undegraded":"2023-12-06T18:34:36.644368+0000","last_fullsized":"2023-12-06T18:34:36.644368+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:34:36.640505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:34:36.640505+0000","last_clean_scrub_stamp":"2023-12-06T18:34:36.640505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T19:57:54.153855+0000","scrub_duration":0.34153093699999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":151,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-12-06T18:34:36.831717+0000","last_change":"2023-12-06T18:34:36.838686+0000","last_active":"2023-12-06T18:34:36.831717+0000","last_peered":"2023-12-06T18:34:36.831717+0000","last_clean":"2023-12-06T18:34:36.831717+0000","last_became_active":"2023-12-06T18:33:52.514832+0000","last_became_peered":"2023-12-06T18:33:52.514832+0000","last_unstale":"2023-12-06T18:34:36.831717+0000","last_undegraded":"2023-12-06T18:34:36.831717+0000","last_fullsized":"2023-12-06T18:34:36.831717+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:34:36.816591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:34:36.816591+0000","last_clean_scrub_stamp":"2023-12-06T18:34:36.816591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:32:52.085689+0000","scrub_duration":0.251764235,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":103,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-12-06T18:34:39.462859+0000","last_change":"2023-12-06T18:34:39.469083+0000","last_active":"2023-12-06T18:34:39.462859+0000","last_peered":"2023-12-06T18:34:39.462859+0000","last_clean":"2023-12-06T18:34:39.462859+0000","last_became_active":"2023-12-06T18:34:19.738308+0000","last_became_peered":"2023-12-06T18:34:19.738308+0000","last_unstale":"2023-12-06T18:34:39.462859+0000","last_undegraded":"2023-12-06T18:34:39.462859+0000","last_fullsized":"2023-12-06T18:34:39.462859+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:34:39.459100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:34:39.459100+0000","last_clean_scrub_stamp":"2023-12-06T18:34:39.459100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T21:05:32.820198+0000","scrub_duration":0.24044795999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":151,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-12-06T18:34:39.673703+0000","last_change":"2023-12-06T18:34:39.674452+0000","last_active":"2023-12-06T18:34:39.673703+0000","last_peered":"2023-12-06T18:34:39.673703+0000","last_clean":"2023-12-06T18:34:39.673703+0000","last_became_active":"2023-12-06T18:28:47.865169+0000","last_became_peered":"2023-12-06T18:28:47.865169+0000","last_unstale":"2023-12-06T18:34:39.673703+0000","last_undegraded":"2023-12-06T18:34:39.673703+0000","last_fullsized":"2023-12-06T18:34:39.673703+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:34:39.672306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:34:39.672306+0000","last_clean_scrub_stamp":"2023-12-06T18:34:39.672306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T23:49:06.172638+0000","scrub_duration":0.029152041,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":81,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-12-06T18:34:35.290539+0000","last_change":"2023-12-06T18:34:35.297383+0000","last_active":"2023-12-06T18:34:35.290539+0000","last_peered":"2023-12-06T18:34:35.290539+0000","last_clean":"2023-12-06T18:34:35.290539+0000","last_became_active":"2023-12-06T18:31:57.175444+0000","last_became_peered":"2023-12-06T18:31:57.175444+0000","last_unstale":"2023-12-06T18:34:35.290539+0000","last_undegraded":"2023-12-06T18:34:35.290539+0000","last_fullsized":"2023-12-06T18:34:35.290539+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-06T18:34:35.286454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-06T18:34:35.286454+0000","last_clean_scrub_stamp":"2023-12-06T18:34:35.286454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-07T23:17:34.764028+0000","scrub_duration":0.32877500999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"144'78","reported_seq":60,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-12-06T18:34:33.102577+0000","last_change":"2023-12-06T18:34:33.110314+0000","last_active":"2023-12-06T18:34:33.102577+0000","last_peered":"2023-12-06T18:34:33.102577+0000","last_clean":"2023-12-06T18:34:33.102577+0000","last_became_active":"2023-12-06T18:33:21.485790+0000","last_became_peered":"2023-12-06T18:33:21.485790+0000","last_unstale":"2023-12-06T18:34:33.102577+0000","last_undegraded":"2023-12-06T18:34:33.102577+0000","last_fullsized":"2023-12-06T18:34:33.102577+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"144'78","last_scrub_stamp":"2023-12-06T18:34:33.098287+0000","last_deep_scrub":"144'78","last_deep_scrub_stamp":"2023-12-06T18:34:33.098287+0000","last_clean_scrub_stamp":"2023-12-06T18:34:33.098287+0000","objects_scrubbed":2,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-08T05:41:35.343464+0000","scrub_duration":0.67993020400000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":82,"num_write":113,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1193520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1605632,"data_stored":1591360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":2,"acting":2,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":18,"seq":77309411428,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":11.292,"5min":11.291,"15min":10.609999999999999},"min":{"1min":5.069,"5min":3.835,"15min":3.835},"max":{"1min":27.620000000000001,"5min":108.57599999999999,"15min":108.57599999999999},"last":10.129},{"interface":"front","average":{"1min":11.958,"5min":11.699,"15min":12.119999999999999},"min":{"1min":6.3239999999999998,"5min":4.8639999999999999,"15min":3.7200000000000002},"max":{"1min":24.844000000000001,"5min":128.09399999999999,"15min":128.09399999999999},"last":7.3479999999999999}]},{"osd":1,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":11.891999999999999,"5min":12.403,"15min":12.693},"min":{"1min":6.5359999999999996,"5min":6.2539999999999996,"15min":6.2539999999999996},"max":{"1min":19.538,"5min":70.072000000000003,"15min":70.072000000000003},"last":12.298999999999999},{"interface":"front","average":{"1min":12.019,"5min":12.824,"15min":14.788},"min":{"1min":6.2080000000000002,"5min":6.2080000000000002,"15min":6.2080000000000002},"max":{"1min":30.16,"5min":30.16,"15min":32.920000000000002},"last":9.4860000000000007}]},{"osd":2,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":15.49,"5min":15.119,"15min":14.845000000000001},"min":{"1min":9.4450000000000003,"5min":8.5079999999999991,"15min":7.5149999999999997},"max":{"1min":52.232999999999997,"5min":75.125,"15min":75.125},"last":14.241},{"interface":"front","average":{"1min":18.545000000000002,"5min":16.343,"15min":17.477},"min":{"1min":9.2599999999999998,"5min":7.5389999999999997,"15min":7.5389999999999997},"max":{"1min":54.323999999999998,"5min":69.403999999999996,"15min":69.403999999999996},"last":10.801}]},{"osd":4,"last update":"Wed Dec 6 18:34:13 2023","interfaces":[{"interface":"back","average":{"1min":18.949999999999999,"5min":15.185,"15min":12.898999999999999},"min":{"1min":10.714,"5min":5.8879999999999999,"15min":5.8879999999999999},"max":{"1min":56.255000000000003,"5min":56.255000000000003,"15min":56.255000000000003},"last":12.959},{"interface":"front","average":{"1min":14.9,"5min":13.204000000000001,"15min":15.038},"min":{"1min":6.4189999999999996,"5min":6.3099999999999996,"15min":6.3099999999999996},"max":{"1min":54.616,"5min":71.718000000000004,"15min":71.718000000000004},"last":15.031000000000001}]},{"osd":5,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":17.082000000000001,"5min":18.359999999999999,"15min":18.027999999999999},"min":{"1min":11.888,"5min":10.417999999999999,"15min":8.3179999999999996},"max":{"1min":24.445,"5min":75.751999999999995,"15min":75.751999999999995},"last":13.589},{"interface":"front","average":{"1min":16.565999999999999,"5min":16.934000000000001,"15min":17.869},"min":{"1min":8.5389999999999997,"5min":7.9269999999999996,"15min":7.9269999999999996},"max":{"1min":32.219000000000001,"5min":70.933000000000007,"15min":70.933000000000007},"last":11.539}]},{"osd":6,"last update":"Wed Dec 6 18:34:09 2023","interfaces":[{"interface":"back","average":{"1min":19.771000000000001,"5min":15.018000000000001,"15min":15.869999999999999},"min":{"1min":10.798,"5min":8.2050000000000001,"15min":7.0919999999999996},"max":{"1min":30.991,"5min":30.991,"15min":30.991},"last":19.292999999999999},{"interface":"front","average":{"1min":16.899999999999999,"5min":18.946999999999999,"15min":20.498000000000001},"min":{"1min":7.2210000000000001,"5min":7.2210000000000001,"15min":7.2210000000000001},"max":{"1min":30.404,"5min":74.224000000000004,"15min":74.224000000000004},"last":8.5649999999999995}]},{"osd":7,"last update":"Wed Dec 6 18:34:13 2023","interfaces":[{"interface":"back","average":{"1min":20.172999999999998,"5min":19.478999999999999,"15min":21.131},"min":{"1min":12.255000000000001,"5min":9.1859999999999999,"15min":9.1859999999999999},"max":{"1min":30.175000000000001,"5min":122.93000000000001,"15min":122.93000000000001},"last":15.695},{"interface":"front","average":{"1min":22.155000000000001,"5min":22.471,"15min":23.312999999999999},"min":{"1min":15.08,"5min":13.196,"15min":13.196},"max":{"1min":34.466999999999999,"5min":124.473,"15min":124.473},"last":16.594999999999999}]}]},{"osd":2,"up_from":16,"seq":68719476835,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28224,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343616,"statfs":{"total":96636764160,"available":96607862784,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":14.244999999999999,"5min":11.606999999999999,"15min":11.34},"min":{"1min":5.234,"5min":4.1639999999999997,"15min":4.1639999999999997},"max":{"1min":47.884999999999998,"5min":47.884999999999998,"15min":47.884999999999998},"last":7.2930000000000001},{"interface":"front","average":{"1min":12.324999999999999,"5min":12.346,"15min":11.509},"min":{"1min":7.1399999999999997,"5min":4.3280000000000003,"15min":4.3280000000000003},"max":{"1min":48.305999999999997,"5min":48.305999999999997,"15min":48.305999999999997},"last":9.1150000000000002}]},{"osd":1,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":14.621,"5min":12.172000000000001,"15min":11.682},"min":{"1min":6.8860000000000001,"5min":5.9139999999999997,"15min":5.9139999999999997},"max":{"1min":28.242999999999999,"5min":42.920999999999999,"15min":42.920999999999999},"last":4.9379999999999997},{"interface":"front","average":{"1min":17.202999999999999,"5min":14.548,"15min":14.061999999999999},"min":{"1min":7.8840000000000003,"5min":6.1479999999999997,"15min":6.1479999999999997},"max":{"1min":29.478999999999999,"5min":34.607999999999997,"15min":34.607999999999997},"last":9.7420000000000009}]},{"osd":3,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":16.474,"5min":15.353,"15min":17.821999999999999},"min":{"1min":9.0139999999999993,"5min":7.2549999999999999,"15min":6.1260000000000003},"max":{"1min":37.463000000000001,"5min":49.609000000000002,"15min":112.129},"last":8.5139999999999993},{"interface":"front","average":{"1min":21.956,"5min":17.753,"15min":21.963999999999999},"min":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"max":{"1min":36.770000000000003,"5min":61.243000000000002,"15min":181.601},"last":15.805999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":19.707999999999998,"5min":16.417999999999999,"15min":17.349},"min":{"1min":8.5250000000000004,"5min":7.8070000000000004,"15min":6.609},"max":{"1min":35.396999999999998,"5min":60.332000000000001,"15min":97.088999999999999},"last":14.946999999999999},{"interface":"front","average":{"1min":13.917999999999999,"5min":13.611000000000001,"15min":18.606999999999999},"min":{"1min":7.7640000000000002,"5min":6.0720000000000001,"15min":6.0720000000000001},"max":{"1min":24.061,"5min":43.198,"15min":98.069000000000003},"last":12.942}]},{"osd":5,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":19.113,"5min":18.113,"15min":18.097000000000001},"min":{"1min":10.743,"5min":8.2100000000000009,"15min":7.8170000000000002},"max":{"1min":26.042000000000002,"5min":57.628999999999998,"15min":57.628999999999998},"last":11.778},{"interface":"front","average":{"1min":15.692,"5min":15.933,"15min":15.824},"min":{"1min":10.292,"5min":8.5990000000000002,"15min":8.5990000000000002},"max":{"1min":39.875999999999998,"5min":47.749000000000002,"15min":47.749000000000002},"last":16.766999999999999}]},{"osd":6,"last update":"Wed Dec 6 18:34:08 2023","interfaces":[{"interface":"back","average":{"1min":20.952999999999999,"5min":19.140000000000001,"15min":36.003999999999998},"min":{"1min":7.0419999999999998,"5min":7.0419999999999998,"15min":7.0419999999999998},"max":{"1min":47.667999999999999,"5min":64.034999999999997,"15min":498.38499999999999},"last":17.710000000000001},{"interface":"front","average":{"1min":24.446000000000002,"5min":20.491,"15min":37.847000000000001},"min":{"1min":12.461,"5min":9.7010000000000005,"15min":9.7010000000000005},"max":{"1min":44.993000000000002,"5min":54.920000000000002,"15min":496.17399999999998},"last":12.369999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":21.704999999999998,"5min":19.547000000000001,"15min":19.131},"min":{"1min":13.574999999999999,"5min":10.064,"15min":10.064},"max":{"1min":35.945,"5min":49.456000000000003,"15min":49.456000000000003},"last":13.525},{"interface":"front","average":{"1min":21.277999999999999,"5min":20.684000000000001,"15min":20.332000000000001},"min":{"1min":12.81,"5min":12.288,"15min":12.288},"max":{"1min":41.965000000000003,"5min":41.965000000000003,"15min":41.965000000000003},"last":18.335000000000001}]}]},{"osd":1,"up_from":16,"seq":68719476836,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":12.32,"5min":11.260999999999999,"15min":18.989000000000001},"min":{"1min":6.0650000000000004,"5min":5.0890000000000004,"15min":5.0890000000000004},"max":{"1min":34.816000000000003,"5min":34.816000000000003,"15min":238.87100000000001},"last":10.053000000000001},{"interface":"front","average":{"1min":11.336,"5min":10.768000000000001,"15min":34.408000000000001},"min":{"1min":5.0629999999999997,"5min":5.0629999999999997,"15min":4.4269999999999996},"max":{"1min":22.834,"5min":61.262,"15min":661.33600000000001},"last":10.484}]},{"osd":2,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.962999999999999,"5min":13.722,"15min":27.556999999999999},"min":{"1min":6.6189999999999998,"5min":6.6189999999999998,"15min":6.6189999999999998},"max":{"1min":57.93,"5min":93.168000000000006,"15min":384.315},"last":11.294},{"interface":"front","average":{"1min":22.902000000000001,"5min":19.120999999999999,"15min":39.317999999999998},"min":{"1min":12.282,"5min":10.724,"15min":5.7009999999999996},"max":{"1min":60.173000000000002,"5min":94.230000000000004,"15min":656.98699999999997},"last":18.545999999999999}]},{"osd":3,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":12.403,"5min":12.194000000000001,"15min":12.863},"min":{"1min":6.9180000000000001,"5min":5.9720000000000004,"15min":5.548},"max":{"1min":25.774999999999999,"5min":31.16,"15min":34.057000000000002},"last":7.6459999999999999},{"interface":"front","average":{"1min":14.6,"5min":14.064,"15min":15.500999999999999},"min":{"1min":8.0180000000000007,"5min":6.5730000000000004,"15min":6.5730000000000004},"max":{"1min":31.417999999999999,"5min":65.537000000000006,"15min":65.537000000000006},"last":12.747}]},{"osd":4,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.082000000000001,"5min":13.308,"15min":33.085999999999999},"min":{"1min":8.6059999999999999,"5min":6.7009999999999996,"15min":6.2930000000000001},"max":{"1min":33.427999999999997,"5min":33.427999999999997,"15min":447.67200000000003},"last":14.257999999999999},{"interface":"front","average":{"1min":21.420000000000002,"5min":18.047999999999998,"15min":39.893000000000001},"min":{"1min":7.7300000000000004,"5min":7.7300000000000004,"15min":7.0919999999999996},"max":{"1min":40.564,"5min":44.043999999999997,"15min":630.572},"last":9.7050000000000001}]},{"osd":5,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.390000000000001,"5min":16.076000000000001,"15min":35.482999999999997},"min":{"1min":7.2549999999999999,"5min":7.0540000000000003,"15min":7.0540000000000003},"max":{"1min":38.991999999999997,"5min":68.013999999999996,"15min":389.55000000000001},"last":15.682},{"interface":"front","average":{"1min":22.140999999999998,"5min":18.395,"15min":45.957999999999998},"min":{"1min":11.099,"5min":7.5359999999999996,"15min":7.0410000000000004},"max":{"1min":41.655999999999999,"5min":41.655999999999999,"15min":658.48599999999999},"last":20.318000000000001}]},{"osd":6,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.672000000000001,"5min":15.733000000000001,"15min":30.202000000000002},"min":{"1min":8.9760000000000009,"5min":7.5899999999999999,"15min":7.5899999999999999},"max":{"1min":32.039999999999999,"5min":54.383000000000003,"15min":439.56400000000002},"last":13.507999999999999},{"interface":"front","average":{"1min":23.643000000000001,"5min":20.5,"15min":32.639000000000003},"min":{"1min":13.058,"5min":10.641999999999999,"15min":10.641999999999999},"max":{"1min":43.052,"5min":55.423999999999999,"15min":386.71499999999997},"last":19.448}]},{"osd":7,"last update":"Wed Dec 6 18:34:16 2023","interfaces":[{"interface":"back","average":{"1min":18.527000000000001,"5min":18.222999999999999,"15min":18.600000000000001},"min":{"1min":8.9849999999999994,"5min":8.9849999999999994,"15min":8.9849999999999994},"max":{"1min":33.156999999999996,"5min":97.561999999999998,"15min":97.561999999999998},"last":16.440999999999999},{"interface":"front","average":{"1min":23.125,"5min":21.024000000000001,"15min":18.771999999999998},"min":{"1min":12.019,"5min":8.7219999999999995,"15min":8.7219999999999995},"max":{"1min":36.332000000000001,"5min":101.762,"15min":101.762},"last":17.102}]}]},{"osd":0,"up_from":16,"seq":68719476837,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343228,"statfs":{"total":96636764160,"available":96607465472,"internally_reserved":0,"allocated":1634304,"data_stored":838986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":13.518000000000001,"5min":11.452,"15min":15.608000000000001},"min":{"1min":5.6340000000000003,"5min":3.2360000000000002,"15min":3.2360000000000002},"max":{"1min":54.155000000000001,"5min":54.155000000000001,"15min":116.593},"last":6.9669999999999996},{"interface":"front","average":{"1min":13.859,"5min":12.935,"15min":16.576000000000001},"min":{"1min":5.5389999999999997,"5min":4.9039999999999999,"15min":4.9039999999999999},"max":{"1min":68.778999999999996,"5min":68.778999999999996,"15min":117.896},"last":12.981999999999999}]},{"osd":2,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":17.434000000000001,"5min":14.888,"15min":14.132999999999999},"min":{"1min":8.9870000000000001,"5min":5.694,"15min":5.694},"max":{"1min":72.131,"5min":72.131,"15min":72.131},"last":20.864000000000001},{"interface":"front","average":{"1min":14.316000000000001,"5min":13.224,"15min":12.055},"min":{"1min":6.7590000000000003,"5min":6.2880000000000003,"15min":6.2880000000000003},"max":{"1min":59.317999999999998,"5min":59.317999999999998,"15min":59.317999999999998},"last":12.009}]},{"osd":3,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":19.911999999999999,"5min":16.183,"15min":16.312999999999999},"min":{"1min":8.9600000000000009,"5min":6.1379999999999999,"15min":6.1379999999999999},"max":{"1min":75.439999999999998,"5min":75.439999999999998,"15min":75.439999999999998},"last":9.1940000000000008},{"interface":"front","average":{"1min":19.082000000000001,"5min":16.047000000000001,"15min":16.204999999999998},"min":{"1min":7.593,"5min":7.5510000000000002,"15min":7.5510000000000002},"max":{"1min":96.688999999999993,"5min":96.688999999999993,"15min":96.688999999999993},"last":17.396000000000001}]},{"osd":4,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":14.302,"5min":12.134,"15min":17.890999999999998},"min":{"1min":6.8239999999999998,"5min":5.9349999999999996,"15min":5.9349999999999996},"max":{"1min":57.694000000000003,"5min":57.694000000000003,"15min":82.432000000000002},"last":21.747},{"interface":"front","average":{"1min":19.693000000000001,"5min":18.327000000000002,"15min":19.846},"min":{"1min":7.7469999999999999,"5min":7.7469999999999999,"15min":6.2629999999999999},"max":{"1min":76.078000000000003,"5min":76.078000000000003,"15min":81.239999999999995},"last":14.069000000000001}]},{"osd":5,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":19.27,"5min":16.798999999999999,"15min":17.507000000000001},"min":{"1min":8.1229999999999993,"5min":8.0099999999999998,"15min":8.0099999999999998},"max":{"1min":97.947000000000003,"5min":97.947000000000003,"15min":97.947000000000003},"last":16.141999999999999},{"interface":"front","average":{"1min":16.989000000000001,"5min":15.443,"15min":17.440000000000001},"min":{"1min":7.4770000000000003,"5min":7.1779999999999999,"15min":7.1779999999999999},"max":{"1min":69.915999999999997,"5min":69.915999999999997,"15min":72.307000000000002},"last":10.081}]},{"osd":6,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":25.541,"5min":20.477,"15min":19.626000000000001},"min":{"1min":12.661,"5min":11.401,"15min":7.7560000000000002},"max":{"1min":82.775000000000006,"5min":82.775000000000006,"15min":82.775000000000006},"last":10.771000000000001},{"interface":"front","average":{"1min":21.75,"5min":17.940999999999999,"15min":18.914999999999999},"min":{"1min":9.7729999999999997,"5min":9.3520000000000003,"15min":9.3520000000000003},"max":{"1min":74.057000000000002,"5min":74.057000000000002,"15min":74.057000000000002},"last":19.317}]},{"osd":7,"last update":"Wed Dec 6 18:34:12 2023","interfaces":[{"interface":"back","average":{"1min":26.137,"5min":21.724,"15min":19.988},"min":{"1min":16.024000000000001,"5min":12.009,"15min":12.009},"max":{"1min":95.287999999999997,"5min":95.287999999999997,"15min":95.287999999999997},"last":14.81},{"interface":"front","average":{"1min":23.494,"5min":20.041,"15min":18.611000000000001},"min":{"1min":10.24,"5min":10.24,"15min":10.24},"max":{"1min":98.995999999999995,"5min":98.995999999999995,"15min":98.995999999999995},"last":18.356999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509541,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343228,"statfs":{"total":96636764160,"available":96607465472,"internally_reserved":0,"allocated":1634304,"data_stored":838986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":14.44,"5min":9.6449999999999996,"15min":13.73},"min":{"1min":4.0430000000000001,"5min":3.3660000000000001,"15min":3.3660000000000001},"max":{"1min":106.521,"5min":106.521,"15min":208.291},"last":7.9379999999999997},{"interface":"front","average":{"1min":14.817,"5min":11.609,"15min":17.067},"min":{"1min":4.984,"5min":4.758,"15min":4.4299999999999997},"max":{"1min":107.342,"5min":107.342,"15min":199.94900000000001},"last":14.367000000000001}]},{"osd":1,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":11.631,"5min":13.298999999999999,"15min":45.225999999999999},"min":{"1min":6.7469999999999999,"5min":5.085,"15min":5.085},"max":{"1min":26.692,"5min":41.234999999999999,"15min":1211.6010000000001},"last":12.923999999999999},{"interface":"front","average":{"1min":12.16,"5min":10.502000000000001,"15min":41.011000000000003},"min":{"1min":5.359,"5min":4.7199999999999998,"15min":4.7199999999999998},"max":{"1min":22.664000000000001,"5min":39.802999999999997,"15min":1189.3869999999999},"last":6.641}]},{"osd":2,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":14.343,"5min":14.629,"15min":21.640999999999998},"min":{"1min":9.0389999999999997,"5min":6.5129999999999999,"15min":6.5129999999999999},"max":{"1min":23.422000000000001,"5min":68.617999999999995,"15min":283.46800000000002},"last":12.289999999999999},{"interface":"front","average":{"1min":19.576000000000001,"5min":18.463999999999999,"15min":25.539000000000001},"min":{"1min":12.851000000000001,"5min":8.173,"15min":8.173},"max":{"1min":40.737000000000002,"5min":69.513999999999996,"15min":296.47800000000001},"last":17.681999999999999}]},{"osd":3,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":11.484999999999999,"5min":13.811,"15min":17.414000000000001},"min":{"1min":7.3639999999999999,"5min":6.0300000000000002,"15min":6.0300000000000002},"max":{"1min":19.872,"5min":36.156999999999996,"15min":47.926000000000002},"last":8.5890000000000004},{"interface":"front","average":{"1min":15.419,"5min":14.555999999999999,"15min":15.007999999999999},"min":{"1min":10.163,"5min":5.3979999999999997,"15min":5.3979999999999997},"max":{"1min":28.914999999999999,"5min":34.609000000000002,"15min":66.176000000000002},"last":11.609999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":18.212,"5min":17.952000000000002,"15min":19.096},"min":{"1min":11.897,"5min":9.7210000000000001,"15min":9.7210000000000001},"max":{"1min":34.069000000000003,"5min":41.923999999999999,"15min":67.248000000000005},"last":18.721},{"interface":"front","average":{"1min":11.249000000000001,"5min":10.798,"15min":12.347},"min":{"1min":6.5279999999999996,"5min":5.3300000000000001,"15min":5.3300000000000001},"max":{"1min":26.850999999999999,"5min":43.101999999999997,"15min":77.266999999999996},"last":10.885}]},{"osd":6,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":15.667999999999999,"5min":13.644,"15min":15.140000000000001},"min":{"1min":7.984,"5min":6.1280000000000001,"15min":5.915},"max":{"1min":25.727,"5min":44.587000000000003,"15min":76.751999999999995},"last":9.2249999999999996},{"interface":"front","average":{"1min":20.158999999999999,"5min":19.356999999999999,"15min":20.469000000000001},"min":{"1min":11.744999999999999,"5min":11.744999999999999,"15min":8.8390000000000004},"max":{"1min":36.883000000000003,"5min":45.366,"15min":57.101999999999997},"last":15.032999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:34:14 2023","interfaces":[{"interface":"back","average":{"1min":18.687999999999999,"5min":16.294,"15min":14.904999999999999},"min":{"1min":11.298999999999999,"5min":7.5519999999999996,"15min":7.5519999999999996},"max":{"1min":32.219999999999999,"5min":32.219999999999999,"15min":32.219999999999999},"last":16.858000000000001},{"interface":"front","average":{"1min":14.401999999999999,"5min":12.872999999999999,"15min":12.215999999999999},"min":{"1min":7.133,"5min":7.133,"15min":7.133},"max":{"1min":29.376999999999999,"5min":29.376999999999999,"15min":29.376999999999999},"last":13.638}]}]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1228800,"data_stored":440865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":8.0180000000000007,"5min":9.266,"15min":12.295999999999999},"min":{"1min":4.2789999999999999,"5min":3.7429999999999999,"15min":3.7429999999999999},"max":{"1min":13.465999999999999,"5min":41.076000000000001,"15min":53.298999999999999},"last":4.4729999999999999},{"interface":"front","average":{"1min":9.9160000000000004,"5min":10,"15min":10.609999999999999},"min":{"1min":5.1280000000000001,"5min":4.8140000000000001,"15min":3.9670000000000001},"max":{"1min":18.329000000000001,"5min":35.863,"15min":52.567},"last":10.371}]},{"osd":1,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":12.202999999999999,"5min":11.432,"15min":13.298999999999999},"min":{"1min":6.0529999999999999,"5min":5.8390000000000004,"15min":5.835},"max":{"1min":21.186,"5min":22.509,"15min":49.183},"last":14.901999999999999},{"interface":"front","average":{"1min":13.43,"5min":12.129,"15min":14.662000000000001},"min":{"1min":5.9770000000000003,"5min":5.9770000000000003,"15min":5.96},"max":{"1min":23.596,"5min":24.081,"15min":45.412999999999997},"last":11.057}]},{"osd":2,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":12.398999999999999,"5min":12.423,"15min":19.385000000000002},"min":{"1min":6.609,"5min":5.9539999999999997,"15min":5.6390000000000002},"max":{"1min":22.904,"5min":44.649000000000001,"15min":85.394000000000005},"last":16.321999999999999},{"interface":"front","average":{"1min":14.196,"5min":15.087999999999999,"15min":18.899000000000001},"min":{"1min":7.9269999999999996,"5min":7.5209999999999999,"15min":6.3639999999999999},"max":{"1min":27.780000000000001,"5min":42.500999999999998,"15min":69.530000000000001},"last":9.4640000000000004}]},{"osd":3,"last update":"Wed Dec 6 18:34:17 2023","interfaces":[{"interface":"back","average":{"1min":16.875,"5min":15.66,"15min":14.851000000000001},"min":{"1min":8.6349999999999998,"5min":8.0250000000000004,"15min":7.008},"max":{"1min":35.418999999999997,"5min":37.561999999999998,"15min":37.561999999999998},"last":14.18},{"interface":"front","average":{"1min":18.140000000000001,"5min":14.212,"15min":14.936},"min":{"1min":8.0619999999999994,"5min":5.5549999999999997,"15min":5.5549999999999997},"max":{"1min":36.694000000000003,"5min":36.694000000000003,"15min":36.694000000000003},"last":7.7130000000000001}]},{"osd":5,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":16.562000000000001,"5min":13.952999999999999,"15min":16.228999999999999},"min":{"1min":6.9370000000000003,"5min":5.4969999999999999,"15min":5.4969999999999999},"max":{"1min":27.283000000000001,"5min":53.840000000000003,"15min":78.798000000000002},"last":17.706},{"interface":"front","average":{"1min":16.690000000000001,"5min":17.414000000000001,"15min":18.402000000000001},"min":{"1min":10.425000000000001,"5min":8.0519999999999996,"15min":6.4450000000000003},"max":{"1min":29.184000000000001,"5min":52.781999999999996,"15min":55.814999999999998},"last":12.949}]},{"osd":6,"last update":"Wed Dec 6 18:34:06 2023","interfaces":[{"interface":"back","average":{"1min":19,"5min":18.105,"15min":18.045000000000002},"min":{"1min":9.3170000000000002,"5min":9.2100000000000009,"15min":7.7850000000000001},"max":{"1min":32.869,"5min":41.723999999999997,"15min":51.149000000000001},"last":18.579000000000001},{"interface":"front","average":{"1min":16.550000000000001,"5min":15.154999999999999,"15min":16.164999999999999},"min":{"1min":7.9359999999999999,"5min":7.9359999999999999,"15min":7.8700000000000001},"max":{"1min":29.792000000000002,"5min":34.929000000000002,"15min":40.426000000000002},"last":12.25}]},{"osd":7,"last update":"Wed Dec 6 18:34:17 2023","interfaces":[{"interface":"back","average":{"1min":17.311,"5min":17.135000000000002,"15min":18.212},"min":{"1min":11.176,"5min":8.093,"15min":8.093},"max":{"1min":24.667000000000002,"5min":60.390000000000001,"15min":60.390000000000001},"last":19.579999999999998},{"interface":"front","average":{"1min":21.228999999999999,"5min":21.193999999999999,"15min":22.27},"min":{"1min":11.083,"5min":11.083,"15min":11.083},"max":{"1min":33.914999999999999,"5min":57.606999999999999,"15min":57.606999999999999},"last":15.678000000000001}]}]},{"osd":7,"up_from":61,"seq":261993005117,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33404,"kb_used_data":1204,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94338436,"statfs":{"total":96636764160,"available":96602558464,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4395,"internal_metadata":28634837},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.206,"5min":11.058999999999999,"15min":10.446999999999999},"min":{"1min":4.0739999999999998,"5min":4.0739999999999998,"15min":4.0739999999999998},"max":{"1min":126.286,"5min":126.286,"15min":126.286},"last":14.686999999999999},{"interface":"front","average":{"1min":15.504,"5min":11.65,"15min":11.226000000000001},"min":{"1min":5.2320000000000002,"5min":4.2530000000000001,"15min":4.2530000000000001},"max":{"1min":127.35599999999999,"5min":127.35599999999999,"15min":127.35599999999999},"last":9.7910000000000004}]},{"osd":1,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":12.824999999999999,"5min":13.972,"15min":16.297000000000001},"min":{"1min":5.9130000000000003,"5min":4.6929999999999996,"15min":4.6929999999999996},"max":{"1min":24.047999999999998,"5min":64.765000000000001,"15min":64.765000000000001},"last":7.2030000000000003},{"interface":"front","average":{"1min":13.289999999999999,"5min":15.44,"15min":17.219999999999999},"min":{"1min":6.2969999999999997,"5min":6.2969999999999997,"15min":6.2969999999999997},"max":{"1min":24.402999999999999,"5min":65.994,"15min":65.994},"last":11.731}]},{"osd":2,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.021000000000001,"5min":16.204000000000001,"15min":17.411999999999999},"min":{"1min":7.9569999999999999,"5min":7.9569999999999999,"15min":7.9569999999999999},"max":{"1min":24.219000000000001,"5min":46.478000000000002,"15min":46.478000000000002},"last":12.435},{"interface":"front","average":{"1min":14.906000000000001,"5min":14.161,"15min":14.132999999999999},"min":{"1min":6.415,"5min":6.415,"15min":6.415},"max":{"1min":23.727,"5min":41.603999999999999,"15min":41.603999999999999},"last":9.1050000000000004}]},{"osd":3,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":17.462,"5min":15.693,"15min":16.739000000000001},"min":{"1min":7.8879999999999999,"5min":6.8319999999999999,"15min":6.8319999999999999},"max":{"1min":47.823,"5min":60.463999999999999,"15min":60.463999999999999},"last":7.8789999999999996},{"interface":"front","average":{"1min":20.838999999999999,"5min":18.907,"15min":19.867999999999999},"min":{"1min":9.8889999999999993,"5min":9.7370000000000001,"15min":9.7370000000000001},"max":{"1min":44.863,"5min":56.286000000000001,"15min":56.286000000000001},"last":17.462}]},{"osd":4,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":20.128,"5min":20.585000000000001,"15min":21.081},"min":{"1min":10.489000000000001,"5min":8.4960000000000004,"15min":8.4960000000000004},"max":{"1min":32.167999999999999,"5min":57.716999999999999,"15min":57.716999999999999},"last":18.306999999999999},{"interface":"front","average":{"1min":16.364000000000001,"5min":17.821999999999999,"15min":18.356000000000002},"min":{"1min":7.2699999999999996,"5min":7.2699999999999996,"15min":7.2699999999999996},"max":{"1min":30.831,"5min":67.159000000000006,"15min":67.159000000000006},"last":8.7829999999999995}]},{"osd":5,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":15.968999999999999,"5min":14.587999999999999,"15min":14.688000000000001},"min":{"1min":7.4219999999999997,"5min":6.3399999999999999,"15min":6.3399999999999999},"max":{"1min":27.07,"5min":44.884,"15min":44.884},"last":11.093999999999999},{"interface":"front","average":{"1min":20.382999999999999,"5min":18.724,"15min":19.527999999999999},"min":{"1min":9.3520000000000003,"5min":9.3520000000000003,"15min":9.3520000000000003},"max":{"1min":92.616,"5min":92.616,"15min":92.616},"last":16.52}]},{"osd":6,"last update":"Wed Dec 6 18:34:15 2023","interfaces":[{"interface":"back","average":{"1min":21.416,"5min":17.657,"15min":17.068999999999999},"min":{"1min":7.625,"5min":7.625,"15min":7.625},"max":{"1min":95.501000000000005,"5min":95.501000000000005,"15min":95.501000000000005},"last":10.499000000000001},{"interface":"front","average":{"1min":23.699999999999999,"5min":20.664999999999999,"15min":20.841000000000001},"min":{"1min":10.811999999999999,"5min":10.144,"15min":10.144},"max":{"1min":94.533000000000001,"5min":94.533000000000001,"15min":94.533000000000001},"last":15.682}]}]},{"osd":6,"up_from":14,"seq":60129542244,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1232896,"data_stored":441146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":9.9879999999999995,"5min":9.4030000000000005,"15min":9.8219999999999992},"min":{"1min":4.3010000000000002,"5min":4.1260000000000003,"15min":3.8519999999999999},"max":{"1min":22.216000000000001,"5min":27.033000000000001,"15min":46.947000000000003},"last":8.8030000000000008},{"interface":"front","average":{"1min":11.436,"5min":10.529,"15min":10.420999999999999},"min":{"1min":4.2919999999999998,"5min":4.0650000000000004,"15min":4.0650000000000004},"max":{"1min":18.338999999999999,"5min":33.152000000000001,"15min":45.953000000000003},"last":4.7309999999999999}]},{"osd":1,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":10.019,"5min":9.423,"15min":11.574999999999999},"min":{"1min":5.6189999999999998,"5min":4.3760000000000003,"15min":4.3760000000000003},"max":{"1min":18.969000000000001,"5min":66.313000000000002,"15min":66.313000000000002},"last":9.8940000000000001},{"interface":"front","average":{"1min":12.763999999999999,"5min":12.657,"15min":13.935},"min":{"1min":6.875,"5min":5.9349999999999996,"15min":5.8540000000000001},"max":{"1min":21.193000000000001,"5min":67.533000000000001,"15min":67.533000000000001},"last":11.367000000000001}]},{"osd":2,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":15.625999999999999,"5min":13.667,"15min":18.298999999999999},"min":{"1min":9.8510000000000009,"5min":6.0640000000000001,"15min":6.0640000000000001},"max":{"1min":41.106000000000002,"5min":46.051000000000002,"15min":200.88499999999999},"last":15.74},{"interface":"front","average":{"1min":20.809999999999999,"5min":17.507000000000001,"15min":16.818000000000001},"min":{"1min":12.57,"5min":7.8129999999999997,"15min":7.4119999999999999},"max":{"1min":39.863,"5min":45.003999999999998,"15min":48.005000000000003},"last":16.591000000000001}]},{"osd":3,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.510000000000002,"5min":14.548999999999999,"15min":14.074},"min":{"1min":9.1600000000000001,"5min":5.9820000000000002,"15min":5.9820000000000002},"max":{"1min":26.93,"5min":36.283999999999999,"15min":36.283999999999999},"last":14.157},{"interface":"front","average":{"1min":19.988,"5min":16.667000000000002,"15min":16.466000000000001},"min":{"1min":11.983000000000001,"5min":7.75,"15min":7.6079999999999997},"max":{"1min":31.489999999999998,"5min":34.317,"15min":47.006999999999998},"last":17.434999999999999}]},{"osd":4,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":17.172999999999998,"5min":16.164000000000001,"15min":15.500999999999999},"min":{"1min":10.173999999999999,"5min":6.1859999999999999,"15min":6.1859999999999999},"max":{"1min":27.928000000000001,"5min":49.170000000000002,"15min":49.170000000000002},"last":12.097},{"interface":"front","average":{"1min":19.071000000000002,"5min":17.736000000000001,"15min":17.052},"min":{"1min":12.409000000000001,"5min":8.9019999999999992,"15min":7.7169999999999996},"max":{"1min":32.399000000000001,"5min":38.573999999999998,"15min":48.988999999999997},"last":18.378}]},{"osd":5,"last update":"Wed Dec 6 18:34:07 2023","interfaces":[{"interface":"back","average":{"1min":20.074000000000002,"5min":18.266999999999999,"15min":18.411999999999999},"min":{"1min":13.186999999999999,"5min":7.8879999999999999,"15min":7.8879999999999999},"max":{"1min":30.518999999999998,"5min":49.390999999999998,"15min":57.308},"last":19.199999999999999},{"interface":"front","average":{"1min":11.132,"5min":10.885,"15min":13.561999999999999},"min":{"1min":5.5380000000000003,"5min":5.5380000000000003,"15min":5.5380000000000003},"max":{"1min":15.542999999999999,"5min":47.415999999999997,"15min":68.463999999999999},"last":10.625999999999999}]},{"osd":7,"last update":"Wed Dec 6 18:34:18 2023","interfaces":[{"interface":"back","average":{"1min":18.114000000000001,"5min":16.631,"15min":15.811},"min":{"1min":9.1460000000000008,"5min":7.4390000000000001,"15min":7.4390000000000001},"max":{"1min":26.856000000000002,"5min":38.887999999999998,"15min":38.887999999999998},"last":14.891999999999999},{"interface":"front","average":{"1min":22.518999999999998,"5min":20.096,"15min":19.210000000000001},"min":{"1min":16.774999999999999,"5min":13.555999999999999,"15min":13.555999999999999},"max":{"1min":34.822000000000003,"5min":34.822000000000003,"15min":34.822000000000003},"last":20.062999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-06T18:34:58.746 DEBUG:teuthology.orchestra.run.smithi061:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-12-06T18:34:59.470 INFO:teuthology.misc:Shutting down mds daemons... 2023-12-06T18:34:59.470 INFO:teuthology.misc:Shutting down osd daemons... 2023-12-06T18:34:59.470 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-12-06T18:34:59.470 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:34:59.481 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:34:59.468+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 87249) UID: 0 2023-12-06T18:34:59.481 INFO:tasks.ceph.osd.0.smithi061.stderr:2023-12-06T18:34:59.468+0000 b47e640 -1 osd.0 144 *** Got signal Terminated *** 2023-12-06T18:35:05.572 INFO:tasks.ceph.osd.0:Stopped 2023-12-06T18:35:05.572 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-12-06T18:35:05.573 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:35:05.582 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:35:05.570+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 87251) UID: 0 2023-12-06T18:35:05.582 INFO:tasks.ceph.osd.1.smithi061.stderr:2023-12-06T18:35:05.571+0000 b47e640 -1 osd.1 146 *** Got signal Terminated *** 2023-12-06T18:35:06.366 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2023-12-06T18:35:07.516 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:03:46.961 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:07.520 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:03:46.965 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:07.535 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:03:46.980 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:07.566 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:02:41.157 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:07.567 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:02:41.159 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:07.567 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:35.818 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:07.571 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:35.821 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:07.582 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:02:41.173 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:07.586 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:35.836 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:11.674 INFO:tasks.ceph.osd.1:Stopped 2023-12-06T18:35:11.675 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-12-06T18:35:11.675 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:35:11.684 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:35:11.673+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 87253) UID: 0 2023-12-06T18:35:11.684 INFO:tasks.ceph.osd.2.smithi061.stderr:2023-12-06T18:35:11.673+0000 b47e640 -1 osd.2 148 *** Got signal Terminated *** 2023-12-06T18:35:12.476 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-12-06T18:35:12.476 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2023-12-06T18:35:17.777 INFO:tasks.ceph.osd.2:Stopped 2023-12-06T18:35:17.777 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-12-06T18:35:17.777 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:35:17.786 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:35:17.775+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 3 (PID: 87254) UID: 0 2023-12-06T18:35:17.788 INFO:tasks.ceph.osd.3.smithi061.stderr:2023-12-06T18:35:17.775+0000 b47e640 -1 osd.3 149 *** Got signal Terminated *** 2023-12-06T18:35:18.485 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-12-06T18:35:18.485 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2023-12-06T18:35:23.879 INFO:tasks.ceph.osd.3:Stopped 2023-12-06T18:35:23.879 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-12-06T18:35:23.879 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:35:23.890 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:35:23.877+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 4 (PID: 87105) UID: 0 2023-12-06T18:35:23.890 INFO:tasks.ceph.osd.4.smithi152.stderr:2023-12-06T18:35:23.877+0000 b47e640 -1 osd.4 154 *** Got signal Terminated *** 2023-12-06T18:35:24.392 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-12-06T18:35:24.392 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2023-12-06T18:35:27.742 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:04:07.186 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:27.745 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:04:07.190 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:27.763 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:04:07.208 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:27.787 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:03:01.378 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:27.790 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:03:01.382 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:27.792 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:56.042 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:27.799 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:56.049 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:27.810 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:03:01.401 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:27.822 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:01:56.072 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:30.300 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-12-06T18:35:47.695 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:04:27.140 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:47.697 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:04:27.142 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:47.717 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:04:27.162 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:47.740 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:03:21.331 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:47.745 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:03:21.337 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:47.751 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:02:16.002 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:47.753 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:02:16.004 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:47.759 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:03:21.350 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:35:47.767 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:02:16.017 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:07.732 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:04:47.176 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:07.735 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:04:47.180 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:07.748 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:04:47.193 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:07.780 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:02:36.030 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:07.783 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:03:41.374 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:07.784 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:02:36.034 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:07.785 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:03:41.377 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:07.800 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:02:36.050 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:07.802 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:03:41.394 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:12.613 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:04:52.055 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:12.614 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:04:52.059 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:12.627 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:04:52.073 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:12.666 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:02:40.917 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:12.667 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:03:46.259 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:12.667 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:02:40.918 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:12.669 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:03:46.260 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:12.682 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:02:40.932 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:12.682 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:03:46.274 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:27.811 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:05:07.256 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:27.814 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:05:07.259 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:27.830 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:05:07.275 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:27.864 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:02:56.115 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:27.868 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:02:56.118 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:27.871 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:04:01.463 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:27.873 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:04:01.465 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:27.883 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:02:56.133 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:27.889 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:04:01.481 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:47.866 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:05:27.311 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:47.871 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:05:27.316 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:47.889 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:05:27.334 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:47.916 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:04:21.507 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:47.918 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:04:21.510 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:47.921 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:03:16.172 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:47.924 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:03:16.174 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:47.931 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:04:21.523 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:36:47.938 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:03:16.188 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:07.918 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:05:47.363 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:07.923 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:05:47.368 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:07.940 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:05:47.385 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:07.973 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:03:36.221 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:07.974 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:04:41.565 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:07.976 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:03:36.226 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:07.976 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:04:41.568 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:07.990 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:03:36.240 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:07.991 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:04:41.583 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:27.974 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:06:07.418 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:27.978 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:06:07.423 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:27.995 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:06:07.440 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:28.022 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:05:01.614 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:28.025 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:05:01.617 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:28.026 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:03:56.276 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:28.031 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:03:56.281 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:28.039 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:05:01.630 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:28.045 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:03:56.295 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:38.112 INFO:tasks.ceph.osd.4:Stopped 2023-12-06T18:37:38.112 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-12-06T18:37:38.113 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:37:38.121 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:37:38.110+0000 b47e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 5 (PID: 87106) UID: 0 2023-12-06T18:37:38.121 INFO:tasks.ceph.osd.5.smithi152.stderr:2023-12-06T18:37:38.111+0000 b47e640 -1 osd.5 156 *** Got signal Terminated *** 2023-12-06T18:37:48.037 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:06:27.482 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:48.042 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:06:27.487 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:48.064 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:06:27.509 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:48.087 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:05:21.678 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:48.092 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:05:21.684 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:48.109 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:05:21.700 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:48.117 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:04:16.367 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:48.118 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:04:16.368 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:37:48.137 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:04:16.387 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:08.224 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:06:47.668 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:08.227 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:06:47.672 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:08.240 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:06:47.685 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:08.273 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:05:41.864 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:08.274 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:05:41.866 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:08.276 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:04:36.526 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:08.277 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:04:36.527 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:08.292 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:05:41.879 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:08.296 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:04:36.546 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:28.164 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:07:07.608 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:28.168 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:07:07.613 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:28.181 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:07:07.626 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:28.207 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:06:01.799 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:28.209 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:06:01.801 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:28.222 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:04:56.472 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:28.223 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:04:56.474 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:28.224 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:06:01.815 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:28.236 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:04:56.487 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:48.224 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:07:27.668 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:48.225 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:07:27.670 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:48.238 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:07:27.684 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:48.267 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:06:21.859 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:48.271 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:06:21.862 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:48.273 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:05:16.523 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:48.282 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:05:16.533 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:48.285 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:06:21.876 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:38:48.298 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:05:16.548 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:08.278 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:07:47.723 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:08.282 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:07:47.727 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:08.304 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:07:47.749 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:08.325 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:06:41.916 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:08.327 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:06:41.918 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:08.338 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:05:36.588 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:08.341 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:06:41.933 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:08.356 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:05:36.607 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:08.374 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:05:36.625 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:28.281 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:08:07.726 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:28.334 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:08:07.779 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:28.350 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:08:07.795 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:28.384 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:07:01.975 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:28.385 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:05:56.635 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:28.386 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:07:01.977 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:28.390 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:05:56.640 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:28.398 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:07:01.990 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:28.406 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:05:56.656 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:28.812 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:07:02.403 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:28.814 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:07:02.406 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:28.826 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:07:02.417 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:28.878 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:08:08.323 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:28.879 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:08:08.324 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:28.881 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:05:57.131 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:28.882 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:05:57.132 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:28.894 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:08:08.339 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:28.895 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:05:57.145 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:48.385 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:08:27.830 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:48.387 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:08:27.833 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:48.401 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:08:27.846 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:48.432 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:06:16.683 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:48.438 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:07:22.030 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:48.441 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:07:22.033 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:48.441 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:06:16.692 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:48.455 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:07:22.046 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:39:48.458 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:06:16.709 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:08.459 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:08:47.903 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:08.460 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:08:47.905 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:08.474 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:08:47.919 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:08.509 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:07:42.100 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:08.511 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:07:42.102 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:08.514 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:06:36.764 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:08.517 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:06:36.767 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:08.524 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:07:42.116 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:08.531 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:06:36.781 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:28.499 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:09:07.943 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:28.503 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:09:07.948 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:28.520 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:09:07.965 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:28.550 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:08:02.141 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:28.552 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:08:02.144 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:28.559 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:06:56.809 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:28.562 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:06:56.812 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:28.566 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:08:02.158 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:28.577 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:06:56.827 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:48.594 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:09:28.036 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:48.638 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:08:22.229 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:48.639 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:08:22.230 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:48.641 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:07:16.892 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:48.643 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:09:28.088 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:48.645 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:07:16.896 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:48.654 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:08:22.246 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:48.662 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:07:16.913 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:40:48.663 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:09:28.108 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:13.665 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:09:53.110 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:13.666 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:09:53.111 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:13.683 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:09:53.128 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:13.710 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:08:47.301 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:13.713 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:08:47.304 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:13.716 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:07:41.967 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:13.720 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:07:41.970 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:13.727 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:08:47.319 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:13.735 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:07:41.986 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:33.748 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:10:13.193 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:33.752 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:10:13.197 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:33.770 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:10:13.215 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:33.798 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:09:07.389 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:33.800 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:09:07.392 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:33.805 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:08:02.055 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:33.807 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:08:02.057 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:33.816 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:09:07.407 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:33.822 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:08:02.072 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:53.794 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:10:33.239 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:53.830 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:10:33.241 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:53.830 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:10:33.256 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:53.842 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:08:22.092 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:53.845 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:08:22.096 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:53.848 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:09:27.439 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:53.849 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:09:27.440 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:53.860 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:08:22.110 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:41:53.861 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:09:27.453 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:13.814 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:10:53.259 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:13.865 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:10:53.310 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:13.880 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:10:53.325 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:13.910 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:09:47.501 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:13.916 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:08:42.166 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:13.917 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:08:42.168 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:13.925 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:09:47.516 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:13.938 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:08:42.188 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:13.940 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:09:47.532 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:18.888 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:10:58.333 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:18.891 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:10:58.336 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:18.902 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:10:58.347 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:18.932 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:09:52.523 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:18.933 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:09:52.525 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:18.940 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:08:47.190 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:18.941 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:08:47.191 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:18.944 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:09:52.535 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:18.956 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:08:47.206 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:33.921 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:11:13.366 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:33.923 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:11:13.369 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:33.939 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:11:13.384 113618== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:33.970 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:10:07.561 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:33.972 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:10:07.563 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:33.985 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:10:07.576 111036== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:33.985 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:09:02.236 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:33.989 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:09:02.239 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:34.004 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:09:02.254 128355== Warning: unimplemented fcntl command: 1036 2023-12-06T18:42:37.185 INFO:teuthology.misc:Shutting down mgr daemons... 2023-12-06T18:42:37.186 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-12-06T18:42:37.186 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:42:37.187 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:42:37.183+0000 7f470cab7640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mgr -f --cluster ceph -i y (PID: 86956) UID: 0 2023-12-06T18:42:37.187 INFO:tasks.ceph.mgr.y.smithi061.stderr:2023-12-06T18:42:37.183+0000 7f470cab7640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-12-06T18:42:37.217 INFO:tasks.ceph.mgr.y:Stopped 2023-12-06T18:42:37.218 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-12-06T18:42:37.218 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:42:37.242 INFO:tasks.ceph.mgr.x:Stopped 2023-12-06T18:42:37.242 INFO:teuthology.misc:Shutting down mon daemons... 2023-12-06T18:42:37.242 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-12-06T18:42:37.242 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:42:37.272 INFO:tasks.ceph.mon.a.smithi061.stderr:2023-12-06T18:42:37.258+0000 9f54640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 113613) UID: 0 2023-12-06T18:42:37.273 INFO:tasks.ceph.mon.a.smithi061.stderr:2023-12-06T18:42:37.260+0000 9f54640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-12-06T18:42:37.791 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:11:17.236 113618== 2023-12-06T18:42:37.791 INFO:tasks.ceph.mon.a.smithi061.stderr:==00:00:11:17.236 113618== Exit program on first error (--exit-on-first-error=yes) 2023-12-06T18:42:37.804 INFO:tasks.ceph.mon.a.smithi061.stderr:daemon-helper: command failed with exit status 42 2023-12-06T18:42:41.783 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-12-06T18:42:43.343 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-12-06T18:42:43.344 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1449, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/misc.py", line 1173, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi061 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a" 2023-12-06T18:42:43.346 INFO:tasks.ceph.mon.a:Stopped 2023-12-06T18:42:43.346 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-12-06T18:42:43.346 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:42:43.362 INFO:tasks.ceph.mon.c.smithi061.stderr:2023-12-06T18:42:43.347+0000 9f54640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 128350) UID: 0 2023-12-06T18:42:43.362 INFO:tasks.ceph.mon.c.smithi061.stderr:2023-12-06T18:42:43.349+0000 9f54640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-12-06T18:42:44.052 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:09:12.302 128355== 2023-12-06T18:42:44.052 INFO:tasks.ceph.mon.c.smithi061.stderr:==00:00:09:12.302 128355== Exit program on first error (--exit-on-first-error=yes) 2023-12-06T18:42:44.065 INFO:tasks.ceph.mon.c.smithi061.stderr:daemon-helper: command failed with exit status 42 2023-12-06T18:42:47.187 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-12-06T18:42:47.187 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-12-06T18:42:49.448 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-12-06T18:42:49.448 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1449, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/misc.py", line 1173, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi061 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c" 2023-12-06T18:42:49.449 INFO:tasks.ceph.mon.c:Stopped 2023-12-06T18:42:49.449 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-12-06T18:42:49.449 INFO:teuthology.orchestra.run:waiting for 300 2023-12-06T18:42:49.463 INFO:tasks.ceph.mon.b.smithi152.stderr:2023-12-06T18:42:49.450+0000 9f54640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 111034) UID: 0 2023-12-06T18:42:49.464 INFO:tasks.ceph.mon.b.smithi152.stderr:2023-12-06T18:42:49.451+0000 9f54640 -1 mon.b@1(probing) e1 *** Got Signal Terminated *** 2023-12-06T18:42:49.989 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:10:23.580 111036== 2023-12-06T18:42:49.989 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:10:23.580 111036== Exit program on first error (--exit-on-first-error=yes) 2023-12-06T18:42:50.000 INFO:tasks.ceph.mon.b.smithi152.stderr:daemon-helper: command failed with exit status 42 2023-12-06T18:42:52.489 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-12-06T18:42:52.490 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-12-06T18:42:55.551 DEBUG:teuthology.orchestra.run:got remote process result: 42 2023-12-06T18:42:55.551 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1447, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1449, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/misc.py", line 1173, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi152 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b" 2023-12-06T18:42:55.551 INFO:tasks.ceph.mon.b:Stopped 2023-12-06T18:42:55.552 INFO:tasks.ceph:Checking cluster log for badness... 2023-12-06T18:42:55.552 DEBUG:teuthology.orchestra.run.smithi061:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(SLOW OPS\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | head -n 1 2023-12-06T18:42:55.597 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi061.front.sepia.ceph.com 2023-12-06T18:42:55.597 DEBUG:teuthology.orchestra.run.smithi061:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-12-06T18:42:55.759 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi061.front.sepia.ceph.com 2023-12-06T18:42:55.760 DEBUG:teuthology.orchestra.run.smithi061:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-12-06T18:42:55.820 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi061.front.sepia.ceph.com 2023-12-06T18:42:55.820 DEBUG:teuthology.orchestra.run.smithi061:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-12-06T18:42:55.879 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi061.front.sepia.ceph.com 2023-12-06T18:42:55.879 DEBUG:teuthology.orchestra.run.smithi061:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-12-06T18:42:55.954 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi152.front.sepia.ceph.com 2023-12-06T18:42:55.954 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-12-06T18:42:56.074 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi152.front.sepia.ceph.com 2023-12-06T18:42:56.074 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-12-06T18:42:56.164 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi152.front.sepia.ceph.com 2023-12-06T18:42:56.164 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:56.221 INFO:teuthology.orchestra.run.smithi152.stderr:umount: /var/lib/ceph/osd/ceph-6: target is busy. 2023-12-06T18:42:56.222 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-06T18:42:56.223 DEBUG:teuthology.orchestra.run.smithi152:> sudo PATH=/usr/sbin:$PATH lsof ; ps auxf 2023-12-06T18:42:56.634 INFO:teuthology.orchestra.run.smithi152.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2023-12-06T18:42:56.635 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.635 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.635 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root txt REG 8,1 102120 20353 /usr/lib/systemd/systemd 2023-12-06T18:42:56.635 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 21219 1167 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2023-12-06T18:42:56.635 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 597073 1143 /etc/selinux/targeted/contexts/files/file_contexts.bin 2023-12-06T18:42:56.635 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.635 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.635 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.635 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.635 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.635 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.635 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 28552 9804 /usr/lib64/libattr.so.1.1.2501 2023-12-06T18:42:56.636 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.636 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.636 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-12-06T18:42:56.636 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.636 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.636 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.636 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.636 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.636 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.636 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 3645472 49253 /usr/lib64/systemd/libsystemd-shared-252.so 2023-12-06T18:42:56.636 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.636 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-12-06T18:42:56.636 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.636 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.636 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 107424 10180 /usr/lib64/libkmod.so.2.3.6 2023-12-06T18:42:56.637 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.637 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 2174088 49251 /usr/lib64/systemd/libsystemd-core-252.so 2023-12-06T18:42:56.637 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 41120 9806 /usr/lib64/libacl.so.1.1.2301 2023-12-06T18:42:56.637 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-12-06T18:42:56.637 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 127680 9972 /usr/lib64/libseccomp.so.2.5.2 2023-12-06T18:42:56.637 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.637 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.637 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root mem REG 0,19 0 19 /sys/fs/selinux/status 2023-12-06T18:42:56.637 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.637 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.637 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.637 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 3w CHR 1,11 0t0 10 /dev/kmsg 2023-12-06T18:42:56.637 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 4u a_inode 0,14 0 13316 [eventpoll:5,6,8,9,11,14,15,16,17,18,44,45,46,47,50,53,60,63,65,67,76,91,97,107,109,110,111,112,117,118,122,127...] 2023-12-06T18:42:56.637 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 5u a_inode 0,14 0 13316 [signalfd] 2023-12-06T18:42:56.638 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 6r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.638 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 7r DIR 0,25 0 1 /sys/fs/cgroup 2023-12-06T18:42:56.638 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 8u a_inode 0,14 0 13316 [timerfd] 2023-12-06T18:42:56.638 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 9u a_inode 0,14 0 13316 [eventpoll:10,13] 2023-12-06T18:42:56.638 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 10r REG 0,20 0 13422 /proc/1/mountinfo 2023-12-06T18:42:56.638 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 11r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.638 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 12u netlink 0t0 19632 AUDIT 2023-12-06T18:42:56.638 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 13r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.638 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 14r REG 0,20 0 4026532076 /proc/swaps 2023-12-06T18:42:56.638 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 15u netlink 0t0 19624 KOBJECT_UEVENT 2023-12-06T18:42:56.638 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 16u unix 0xffff8a57582c1540 0t0 19626 /run/systemd/private type=STREAM (LISTEN) 2023-12-06T18:42:56.638 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 17u a_inode 0,14 0 13316 [timerfd] 2023-12-06T18:42:56.638 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 18r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.638 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 19r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.638 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 20u a_inode 0,14 0 13316 bpf-prog 2023-12-06T18:42:56.639 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 21u a_inode 0,14 0 13316 bpf-prog 2023-12-06T18:42:56.639 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 22u a_inode 0,14 0 13316 bpf-prog 2023-12-06T18:42:56.639 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 23u a_inode 0,14 0 13316 bpf-prog 2023-12-06T18:42:56.639 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 24u a_inode 0,14 0 13316 bpf-prog 2023-12-06T18:42:56.639 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 25u unix 0xffff8a5747ceaec0 0t0 289 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.639 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 26u a_inode 0,14 0 13316 bpf-prog 2023-12-06T18:42:56.639 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 27r CHR 10,235 0t0 111 /dev/autofs 2023-12-06T18:42:56.639 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 28u a_inode 0,14 0 13316 bpf-prog 2023-12-06T18:42:56.639 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 29u a_inode 0,14 0 13316 bpf-prog 2023-12-06T18:42:56.639 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 30u a_inode 0,14 0 13316 bpf-prog 2023-12-06T18:42:56.639 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 31u a_inode 0,14 0 13316 bpf-prog 2023-12-06T18:42:56.639 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 44u unix 0xffff8a575806d0c0 0t0 13430 /run/systemd/io.system.ManagedOOM type=STREAM (LISTEN) 2023-12-06T18:42:56.639 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 45u unix 0xffff8a5758069980 0t0 13429 /run/systemd/userdb/io.systemd.DynamicUser type=STREAM (LISTEN) 2023-12-06T18:42:56.639 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 46u unix 0xffff8a575813a640 0t0 13424 /run/systemd/notify type=DGRAM (CONNECTED) 2023-12-06T18:42:56.639 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 47u unix 0xffff8a575813a200 0t0 13425 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.640 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 48u unix 0xffff8a575813dd80 0t0 13426 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.640 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 50u a_inode 0,14 0 13316 [timerfd] 2023-12-06T18:42:56.640 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 53u CHR 10,242 0t0 165 /dev/rfkill 2023-12-06T18:42:56.640 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 60u unix 0xffff8a5748414840 0t0 17776 type=STREAM (CONNECTED) 2023-12-06T18:42:56.640 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 63u unix 0xffff8a5748412640 0t0 17770 /run/.heim_org.h5l.kcm-socket type=STREAM (LISTEN) 2023-12-06T18:42:56.640 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 65u FIFO 0,24 0t0 441 /run/dmeventd-server 2023-12-06T18:42:56.640 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 67u FIFO 0,24 0t0 442 /run/dmeventd-client 2023-12-06T18:42:56.640 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 68u FIFO 0,24 0t0 447 /run/initctl 2023-12-06T18:42:56.640 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 69u unix 0xffff8a575813ee80 0t0 13439 /run/systemd/journal/socket type=DGRAM (CONNECTED) 2023-12-06T18:42:56.640 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 72u unix 0xffff8a5758068880 0t0 13441 /run/systemd/journal/stdout type=STREAM (LISTEN) 2023-12-06T18:42:56.640 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 76r FIFO 0,13 0t0 19636 pipe 2023-12-06T18:42:56.640 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 77u unix 0xffff8a5747ce9540 0t0 250 /run/udev/control type=SEQPACKET (LISTEN) 2023-12-06T18:42:56.640 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 79u unix 0xffff8a5747ce3300 0t0 238 /run/rpcbind.sock type=STREAM (LISTEN) 2023-12-06T18:42:56.640 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 80u IPv4 17661 0t0 TCP *:sunrpc (LISTEN) 2023-12-06T18:42:56.640 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 81u IPv4 13809 0t0 UDP *:sunrpc 2023-12-06T18:42:56.641 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 82u IPv6 13815 0t0 TCP *:sunrpc (LISTEN) 2023-12-06T18:42:56.641 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 83u IPv6 19646 0t0 UDP *:sunrpc 2023-12-06T18:42:56.641 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 90u unix 0xffff8a575813d500 0t0 13437 /run/systemd/journal/dev-log type=DGRAM (CONNECTED) 2023-12-06T18:42:56.641 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 91u unix 0xffff8a5747cebfc0 0t0 248 /run/systemd/coredump type=SEQPACKET (LISTEN) 2023-12-06T18:42:56.641 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 93u unix 0xffff8a5748412200 0t0 17769 /run/dbus/system_bus_socket type=STREAM (LISTEN) 2023-12-06T18:42:56.641 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 96u netlink 0t0 252 KOBJECT_UEVENT 2023-12-06T18:42:56.641 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 97u unix 0xffff8a57582c1100 0t0 19639 /run/lvm/lvmpolld.socket type=STREAM (LISTEN) 2023-12-06T18:42:56.641 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 107u unix 0xffff8a575828f700 0t0 21919 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.641 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 109u unix 0xffff8a574739e1c0 0t0 21144 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.641 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 110u unix 0xffff8a5748416a40 0t0 13961 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.641 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 111u unix 0xffff8a5747ce72c0 0t0 13960 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.641 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 112u unix 0xffff8a5747ce2640 0t0 18892 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.641 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 117u unix 0xffff8a574739f2c0 0t0 22753 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.641 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 118u unix 0xffff8a5748557700 0t0 22748 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.641 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 121u unix 0xffff8a5747d8d940 0t0 21607 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.642 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 122u unix 0xffff8a574739cc80 0t0 22744 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.642 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 123u unix 0xffff8a574739fb40 0t0 21599 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.642 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 124u unix 0xffff8a5747ce3fc0 0t0 21598 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.642 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 125u unix 0xffff8a57582c0000 0t0 21022 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.642 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 126u unix 0xffff8a5747ce0880 0t0 16637 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.642 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 1 root 127u unix 0xffff8a5748410880 0t0 161573 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.642 INFO:teuthology.orchestra.run.smithi152.stdout:kthreadd 2 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.642 INFO:teuthology.orchestra.run.smithi152.stdout:kthreadd 2 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.642 INFO:teuthology.orchestra.run.smithi152.stdout:kthreadd 2 root txt unknown /proc/2/exe 2023-12-06T18:42:56.642 INFO:teuthology.orchestra.run.smithi152.stdout:rcu_gp 3 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.642 INFO:teuthology.orchestra.run.smithi152.stdout:rcu_gp 3 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.642 INFO:teuthology.orchestra.run.smithi152.stdout:rcu_gp 3 root txt unknown /proc/3/exe 2023-12-06T18:42:56.642 INFO:teuthology.orchestra.run.smithi152.stdout:rcu_par_g 4 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.642 INFO:teuthology.orchestra.run.smithi152.stdout:rcu_par_g 4 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.642 INFO:teuthology.orchestra.run.smithi152.stdout:rcu_par_g 4 root txt unknown /proc/4/exe 2023-12-06T18:42:56.643 INFO:teuthology.orchestra.run.smithi152.stdout:slub_flus 5 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.643 INFO:teuthology.orchestra.run.smithi152.stdout:slub_flus 5 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.643 INFO:teuthology.orchestra.run.smithi152.stdout:slub_flus 5 root txt unknown /proc/5/exe 2023-12-06T18:42:56.643 INFO:teuthology.orchestra.run.smithi152.stdout:netns 6 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.643 INFO:teuthology.orchestra.run.smithi152.stdout:netns 6 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.643 INFO:teuthology.orchestra.run.smithi152.stdout:netns 6 root txt unknown /proc/6/exe 2023-12-06T18:42:56.643 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/0 8 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.643 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/0 8 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.643 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/0 8 root txt unknown /proc/8/exe 2023-12-06T18:42:56.643 INFO:teuthology.orchestra.run.smithi152.stdout:mm_percpu 10 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.643 INFO:teuthology.orchestra.run.smithi152.stdout:mm_percpu 10 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.643 INFO:teuthology.orchestra.run.smithi152.stdout:mm_percpu 10 root txt unknown /proc/10/exe 2023-12-06T18:42:56.643 INFO:teuthology.orchestra.run.smithi152.stdout:rcu_tasks 12 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.643 INFO:teuthology.orchestra.run.smithi152.stdout:rcu_tasks 12 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.644 INFO:teuthology.orchestra.run.smithi152.stdout:rcu_tasks 12 root txt unknown /proc/12/exe 2023-12-06T18:42:56.644 INFO:teuthology.orchestra.run.smithi152.stdout:rcu_tasks 13 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.644 INFO:teuthology.orchestra.run.smithi152.stdout:rcu_tasks 13 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.644 INFO:teuthology.orchestra.run.smithi152.stdout:rcu_tasks 13 root txt unknown /proc/13/exe 2023-12-06T18:42:56.644 INFO:teuthology.orchestra.run.smithi152.stdout:rcu_tasks 14 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.644 INFO:teuthology.orchestra.run.smithi152.stdout:rcu_tasks 14 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.644 INFO:teuthology.orchestra.run.smithi152.stdout:rcu_tasks 14 root txt unknown /proc/14/exe 2023-12-06T18:42:56.644 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 15 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.644 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 15 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.644 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 15 root txt unknown /proc/15/exe 2023-12-06T18:42:56.644 INFO:teuthology.orchestra.run.smithi152.stdout:pr/ttyS1 16 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.644 INFO:teuthology.orchestra.run.smithi152.stdout:pr/ttyS1 16 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.644 INFO:teuthology.orchestra.run.smithi152.stdout:pr/ttyS1 16 root txt unknown /proc/16/exe 2023-12-06T18:42:56.644 INFO:teuthology.orchestra.run.smithi152.stdout:rcu_preem 17 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.645 INFO:teuthology.orchestra.run.smithi152.stdout:rcu_preem 17 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.645 INFO:teuthology.orchestra.run.smithi152.stdout:rcu_preem 17 root txt unknown /proc/17/exe 2023-12-06T18:42:56.645 INFO:teuthology.orchestra.run.smithi152.stdout:migration 18 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.645 INFO:teuthology.orchestra.run.smithi152.stdout:migration 18 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.645 INFO:teuthology.orchestra.run.smithi152.stdout:migration 18 root txt unknown /proc/18/exe 2023-12-06T18:42:56.645 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 19 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.645 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 19 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.645 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 19 root txt unknown /proc/19/exe 2023-12-06T18:42:56.645 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/0 21 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.645 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/0 21 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.645 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/0 21 root txt unknown /proc/21/exe 2023-12-06T18:42:56.645 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/1 22 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.645 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/1 22 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.645 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/1 22 root txt unknown /proc/22/exe 2023-12-06T18:42:56.645 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 23 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.646 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 23 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.646 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 23 root txt unknown /proc/23/exe 2023-12-06T18:42:56.646 INFO:teuthology.orchestra.run.smithi152.stdout:migration 24 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.646 INFO:teuthology.orchestra.run.smithi152.stdout:migration 24 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.646 INFO:teuthology.orchestra.run.smithi152.stdout:migration 24 root txt unknown /proc/24/exe 2023-12-06T18:42:56.646 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 25 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.646 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 25 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.646 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 25 root txt unknown /proc/25/exe 2023-12-06T18:42:56.646 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/1 27 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.646 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/1 27 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.646 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/1 27 root txt unknown /proc/27/exe 2023-12-06T18:42:56.646 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/2 28 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.646 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/2 28 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.646 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/2 28 root txt unknown /proc/28/exe 2023-12-06T18:42:56.646 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 29 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.647 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 29 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.647 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 29 root txt unknown /proc/29/exe 2023-12-06T18:42:56.647 INFO:teuthology.orchestra.run.smithi152.stdout:migration 30 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.647 INFO:teuthology.orchestra.run.smithi152.stdout:migration 30 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.647 INFO:teuthology.orchestra.run.smithi152.stdout:migration 30 root txt unknown /proc/30/exe 2023-12-06T18:42:56.647 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 31 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.647 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 31 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.647 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 31 root txt unknown /proc/31/exe 2023-12-06T18:42:56.647 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/2 33 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.647 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/2 33 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.647 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/2 33 root txt unknown /proc/33/exe 2023-12-06T18:42:56.647 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/3 34 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.647 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/3 34 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.647 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/3 34 root txt unknown /proc/34/exe 2023-12-06T18:42:56.647 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 35 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.648 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 35 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.648 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 35 root txt unknown /proc/35/exe 2023-12-06T18:42:56.648 INFO:teuthology.orchestra.run.smithi152.stdout:migration 36 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.648 INFO:teuthology.orchestra.run.smithi152.stdout:migration 36 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.648 INFO:teuthology.orchestra.run.smithi152.stdout:migration 36 root txt unknown /proc/36/exe 2023-12-06T18:42:56.648 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 37 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.648 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 37 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.648 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 37 root txt unknown /proc/37/exe 2023-12-06T18:42:56.648 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/3 39 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.648 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/3 39 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.648 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/3 39 root txt unknown /proc/39/exe 2023-12-06T18:42:56.648 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/4 40 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.648 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/4 40 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.648 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/4 40 root txt unknown /proc/40/exe 2023-12-06T18:42:56.649 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 41 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.649 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 41 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.649 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 41 root txt unknown /proc/41/exe 2023-12-06T18:42:56.649 INFO:teuthology.orchestra.run.smithi152.stdout:migration 42 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.649 INFO:teuthology.orchestra.run.smithi152.stdout:migration 42 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.649 INFO:teuthology.orchestra.run.smithi152.stdout:migration 42 root txt unknown /proc/42/exe 2023-12-06T18:42:56.649 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 43 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.649 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 43 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.649 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 43 root txt unknown /proc/43/exe 2023-12-06T18:42:56.649 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/4 45 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.649 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/4 45 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.649 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/4 45 root txt unknown /proc/45/exe 2023-12-06T18:42:56.649 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/5 46 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.649 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/5 46 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.649 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/5 46 root txt unknown /proc/46/exe 2023-12-06T18:42:56.649 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 47 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.650 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 47 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.650 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 47 root txt unknown /proc/47/exe 2023-12-06T18:42:56.650 INFO:teuthology.orchestra.run.smithi152.stdout:migration 48 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.650 INFO:teuthology.orchestra.run.smithi152.stdout:migration 48 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.650 INFO:teuthology.orchestra.run.smithi152.stdout:migration 48 root txt unknown /proc/48/exe 2023-12-06T18:42:56.650 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 49 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.650 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 49 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.650 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 49 root txt unknown /proc/49/exe 2023-12-06T18:42:56.650 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/5 50 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.650 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/5 50 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.650 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/5 50 root txt unknown /proc/50/exe 2023-12-06T18:42:56.650 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/5 51 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.650 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/5 51 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.650 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/5 51 root txt unknown /proc/51/exe 2023-12-06T18:42:56.650 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/6 52 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.651 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/6 52 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.651 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/6 52 root txt unknown /proc/52/exe 2023-12-06T18:42:56.651 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 53 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.651 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 53 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.651 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 53 root txt unknown /proc/53/exe 2023-12-06T18:42:56.651 INFO:teuthology.orchestra.run.smithi152.stdout:migration 54 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.651 INFO:teuthology.orchestra.run.smithi152.stdout:migration 54 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.651 INFO:teuthology.orchestra.run.smithi152.stdout:migration 54 root txt unknown /proc/54/exe 2023-12-06T18:42:56.651 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 55 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.651 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 55 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.651 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 55 root txt unknown /proc/55/exe 2023-12-06T18:42:56.651 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/6 57 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.651 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/6 57 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.651 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/6 57 root txt unknown /proc/57/exe 2023-12-06T18:42:56.651 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/7 58 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.651 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/7 58 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.652 INFO:teuthology.orchestra.run.smithi152.stdout:cpuhp/7 58 root txt unknown /proc/58/exe 2023-12-06T18:42:56.652 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 59 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.652 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 59 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.652 INFO:teuthology.orchestra.run.smithi152.stdout:idle_inje 59 root txt unknown /proc/59/exe 2023-12-06T18:42:56.652 INFO:teuthology.orchestra.run.smithi152.stdout:migration 60 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.652 INFO:teuthology.orchestra.run.smithi152.stdout:migration 60 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.652 INFO:teuthology.orchestra.run.smithi152.stdout:migration 60 root txt unknown /proc/60/exe 2023-12-06T18:42:56.652 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 61 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.652 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 61 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.652 INFO:teuthology.orchestra.run.smithi152.stdout:ksoftirqd 61 root txt unknown /proc/61/exe 2023-12-06T18:42:56.652 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/7 63 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.652 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/7 63 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.652 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/7 63 root txt unknown /proc/63/exe 2023-12-06T18:42:56.652 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 70 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.652 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 70 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.652 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 70 root txt unknown /proc/70/exe 2023-12-06T18:42:56.653 INFO:teuthology.orchestra.run.smithi152.stdout:kdevtmpfs 71 root cwd DIR 0,5 3500 1 / 2023-12-06T18:42:56.653 INFO:teuthology.orchestra.run.smithi152.stdout:kdevtmpfs 71 root rtd DIR 0,5 3500 1 / 2023-12-06T18:42:56.653 INFO:teuthology.orchestra.run.smithi152.stdout:kdevtmpfs 71 root txt unknown /proc/71/exe 2023-12-06T18:42:56.653 INFO:teuthology.orchestra.run.smithi152.stdout:inet_frag 72 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.653 INFO:teuthology.orchestra.run.smithi152.stdout:inet_frag 72 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.653 INFO:teuthology.orchestra.run.smithi152.stdout:inet_frag 72 root txt unknown /proc/72/exe 2023-12-06T18:42:56.653 INFO:teuthology.orchestra.run.smithi152.stdout:kauditd 73 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.653 INFO:teuthology.orchestra.run.smithi152.stdout:kauditd 73 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.653 INFO:teuthology.orchestra.run.smithi152.stdout:kauditd 73 root txt unknown /proc/73/exe 2023-12-06T18:42:56.653 INFO:teuthology.orchestra.run.smithi152.stdout:khungtask 75 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.654 INFO:teuthology.orchestra.run.smithi152.stdout:khungtask 75 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.654 INFO:teuthology.orchestra.run.smithi152.stdout:khungtask 75 root txt unknown /proc/75/exe 2023-12-06T18:42:56.654 INFO:teuthology.orchestra.run.smithi152.stdout:oom_reape 76 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.654 INFO:teuthology.orchestra.run.smithi152.stdout:oom_reape 76 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.654 INFO:teuthology.orchestra.run.smithi152.stdout:oom_reape 76 root txt unknown /proc/76/exe 2023-12-06T18:42:56.654 INFO:teuthology.orchestra.run.smithi152.stdout:writeback 77 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.654 INFO:teuthology.orchestra.run.smithi152.stdout:writeback 77 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.654 INFO:teuthology.orchestra.run.smithi152.stdout:writeback 77 root txt unknown /proc/77/exe 2023-12-06T18:42:56.654 INFO:teuthology.orchestra.run.smithi152.stdout:kcompactd 78 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.654 INFO:teuthology.orchestra.run.smithi152.stdout:kcompactd 78 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.654 INFO:teuthology.orchestra.run.smithi152.stdout:kcompactd 78 root txt unknown /proc/78/exe 2023-12-06T18:42:56.654 INFO:teuthology.orchestra.run.smithi152.stdout:ksmd 79 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.654 INFO:teuthology.orchestra.run.smithi152.stdout:ksmd 79 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.654 INFO:teuthology.orchestra.run.smithi152.stdout:ksmd 79 root txt unknown /proc/79/exe 2023-12-06T18:42:56.654 INFO:teuthology.orchestra.run.smithi152.stdout:khugepage 81 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.655 INFO:teuthology.orchestra.run.smithi152.stdout:khugepage 81 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.655 INFO:teuthology.orchestra.run.smithi152.stdout:khugepage 81 root txt unknown /proc/81/exe 2023-12-06T18:42:56.655 INFO:teuthology.orchestra.run.smithi152.stdout:cryptd 82 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.655 INFO:teuthology.orchestra.run.smithi152.stdout:cryptd 82 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.655 INFO:teuthology.orchestra.run.smithi152.stdout:cryptd 82 root txt unknown /proc/82/exe 2023-12-06T18:42:56.655 INFO:teuthology.orchestra.run.smithi152.stdout:kintegrit 83 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.655 INFO:teuthology.orchestra.run.smithi152.stdout:kintegrit 83 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.655 INFO:teuthology.orchestra.run.smithi152.stdout:kintegrit 83 root txt unknown /proc/83/exe 2023-12-06T18:42:56.655 INFO:teuthology.orchestra.run.smithi152.stdout:kblockd 84 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.655 INFO:teuthology.orchestra.run.smithi152.stdout:kblockd 84 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.655 INFO:teuthology.orchestra.run.smithi152.stdout:kblockd 84 root txt unknown /proc/84/exe 2023-12-06T18:42:56.655 INFO:teuthology.orchestra.run.smithi152.stdout:blkcg_pun 85 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.655 INFO:teuthology.orchestra.run.smithi152.stdout:blkcg_pun 85 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.655 INFO:teuthology.orchestra.run.smithi152.stdout:blkcg_pun 85 root txt unknown /proc/85/exe 2023-12-06T18:42:56.655 INFO:teuthology.orchestra.run.smithi152.stdout:tpm_dev_w 86 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.656 INFO:teuthology.orchestra.run.smithi152.stdout:tpm_dev_w 86 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.656 INFO:teuthology.orchestra.run.smithi152.stdout:tpm_dev_w 86 root txt unknown /proc/86/exe 2023-12-06T18:42:56.656 INFO:teuthology.orchestra.run.smithi152.stdout:md 87 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.656 INFO:teuthology.orchestra.run.smithi152.stdout:md 87 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.656 INFO:teuthology.orchestra.run.smithi152.stdout:md 87 root txt unknown /proc/87/exe 2023-12-06T18:42:56.656 INFO:teuthology.orchestra.run.smithi152.stdout:md_bitmap 88 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.656 INFO:teuthology.orchestra.run.smithi152.stdout:md_bitmap 88 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.656 INFO:teuthology.orchestra.run.smithi152.stdout:md_bitmap 88 root txt unknown /proc/88/exe 2023-12-06T18:42:56.656 INFO:teuthology.orchestra.run.smithi152.stdout:edac-poll 89 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.656 INFO:teuthology.orchestra.run.smithi152.stdout:edac-poll 89 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.656 INFO:teuthology.orchestra.run.smithi152.stdout:edac-poll 89 root txt unknown /proc/89/exe 2023-12-06T18:42:56.656 INFO:teuthology.orchestra.run.smithi152.stdout:watchdogd 90 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.656 INFO:teuthology.orchestra.run.smithi152.stdout:watchdogd 90 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.656 INFO:teuthology.orchestra.run.smithi152.stdout:watchdogd 90 root txt unknown /proc/90/exe 2023-12-06T18:42:56.656 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/5 93 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.657 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/5 93 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.657 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/5 93 root txt unknown /proc/93/exe 2023-12-06T18:42:56.657 INFO:teuthology.orchestra.run.smithi152.stdout:kswapd0 95 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.657 INFO:teuthology.orchestra.run.smithi152.stdout:kswapd0 95 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.657 INFO:teuthology.orchestra.run.smithi152.stdout:kswapd0 95 root txt unknown /proc/95/exe 2023-12-06T18:42:56.657 INFO:teuthology.orchestra.run.smithi152.stdout:kthrotld 101 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.657 INFO:teuthology.orchestra.run.smithi152.stdout:kthrotld 101 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.657 INFO:teuthology.orchestra.run.smithi152.stdout:kthrotld 101 root txt unknown /proc/101/exe 2023-12-06T18:42:56.657 INFO:teuthology.orchestra.run.smithi152.stdout:acpi_ther 109 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.657 INFO:teuthology.orchestra.run.smithi152.stdout:acpi_ther 109 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.657 INFO:teuthology.orchestra.run.smithi152.stdout:acpi_ther 109 root txt unknown /proc/109/exe 2023-12-06T18:42:56.657 INFO:teuthology.orchestra.run.smithi152.stdout:kmpath_rd 110 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.657 INFO:teuthology.orchestra.run.smithi152.stdout:kmpath_rd 110 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.657 INFO:teuthology.orchestra.run.smithi152.stdout:kmpath_rd 110 root txt unknown /proc/110/exe 2023-12-06T18:42:56.657 INFO:teuthology.orchestra.run.smithi152.stdout:kaluad 111 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.658 INFO:teuthology.orchestra.run.smithi152.stdout:kaluad 111 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.658 INFO:teuthology.orchestra.run.smithi152.stdout:kaluad 111 root txt unknown /proc/111/exe 2023-12-06T18:42:56.658 INFO:teuthology.orchestra.run.smithi152.stdout:mld 112 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.658 INFO:teuthology.orchestra.run.smithi152.stdout:mld 112 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.658 INFO:teuthology.orchestra.run.smithi152.stdout:mld 112 root txt unknown /proc/112/exe 2023-12-06T18:42:56.658 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/4 113 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.658 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/4 113 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.658 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/4 113 root txt unknown /proc/113/exe 2023-12-06T18:42:56.658 INFO:teuthology.orchestra.run.smithi152.stdout:ipv6_addr 114 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.658 INFO:teuthology.orchestra.run.smithi152.stdout:ipv6_addr 114 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.658 INFO:teuthology.orchestra.run.smithi152.stdout:ipv6_addr 114 root txt unknown /proc/114/exe 2023-12-06T18:42:56.658 INFO:teuthology.orchestra.run.smithi152.stdout:kstrp 123 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.658 INFO:teuthology.orchestra.run.smithi152.stdout:kstrp 123 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.659 INFO:teuthology.orchestra.run.smithi152.stdout:kstrp 123 root txt unknown /proc/123/exe 2023-12-06T18:42:56.659 INFO:teuthology.orchestra.run.smithi152.stdout:zswap-shr 129 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.659 INFO:teuthology.orchestra.run.smithi152.stdout:zswap-shr 129 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.659 INFO:teuthology.orchestra.run.smithi152.stdout:zswap-shr 129 root txt unknown /proc/129/exe 2023-12-06T18:42:56.659 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 238 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.659 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 238 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.659 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 238 root txt unknown /proc/238/exe 2023-12-06T18:42:56.659 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/0 288 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.659 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/0 288 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.659 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/0 288 root txt unknown /proc/288/exe 2023-12-06T18:42:56.659 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/2 289 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.659 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/2 289 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.659 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/2 289 root txt unknown /proc/289/exe 2023-12-06T18:42:56.659 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/7 330 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.659 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/7 330 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.659 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/7 330 root txt unknown /proc/330/exe 2023-12-06T18:42:56.660 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/6 357 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.660 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/6 357 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.660 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/6 357 root txt unknown /proc/357/exe 2023-12-06T18:42:56.660 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/1 377 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.660 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/1 377 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.660 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/1 377 root txt unknown /proc/377/exe 2023-12-06T18:42:56.660 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/3 411 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.660 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/3 411 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.660 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/3 411 root txt unknown /proc/411/exe 2023-12-06T18:42:56.660 INFO:teuthology.orchestra.run.smithi152.stdout:nvme-wq 427 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.660 INFO:teuthology.orchestra.run.smithi152.stdout:nvme-wq 427 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.660 INFO:teuthology.orchestra.run.smithi152.stdout:nvme-wq 427 root txt unknown /proc/427/exe 2023-12-06T18:42:56.660 INFO:teuthology.orchestra.run.smithi152.stdout:nvme-rese 428 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.660 INFO:teuthology.orchestra.run.smithi152.stdout:nvme-rese 428 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.660 INFO:teuthology.orchestra.run.smithi152.stdout:nvme-rese 428 root txt unknown /proc/428/exe 2023-12-06T18:42:56.661 INFO:teuthology.orchestra.run.smithi152.stdout:nvme-dele 429 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.661 INFO:teuthology.orchestra.run.smithi152.stdout:nvme-dele 429 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.661 INFO:teuthology.orchestra.run.smithi152.stdout:nvme-dele 429 root txt unknown /proc/429/exe 2023-12-06T18:42:56.661 INFO:teuthology.orchestra.run.smithi152.stdout:nvme-auth 430 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.661 INFO:teuthology.orchestra.run.smithi152.stdout:nvme-auth 430 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.661 INFO:teuthology.orchestra.run.smithi152.stdout:nvme-auth 430 root txt unknown /proc/430/exe 2023-12-06T18:42:56.661 INFO:teuthology.orchestra.run.smithi152.stdout:ata_sff 431 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.661 INFO:teuthology.orchestra.run.smithi152.stdout:ata_sff 431 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.661 INFO:teuthology.orchestra.run.smithi152.stdout:ata_sff 431 root txt unknown /proc/431/exe 2023-12-06T18:42:56.661 INFO:teuthology.orchestra.run.smithi152.stdout:ixgbe 434 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.661 INFO:teuthology.orchestra.run.smithi152.stdout:ixgbe 434 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.661 INFO:teuthology.orchestra.run.smithi152.stdout:ixgbe 434 root txt unknown /proc/434/exe 2023-12-06T18:42:56.661 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_0 443 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.661 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_0 443 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.661 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_0 443 root txt unknown /proc/443/exe 2023-12-06T18:42:56.661 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 444 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.662 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 444 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.662 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 444 root txt unknown /proc/444/exe 2023-12-06T18:42:56.662 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_1 445 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.662 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_1 445 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.662 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_1 445 root txt unknown /proc/445/exe 2023-12-06T18:42:56.662 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 446 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.662 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 446 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.662 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 446 root txt unknown /proc/446/exe 2023-12-06T18:42:56.662 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_2 447 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.662 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_2 447 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.662 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_2 447 root txt unknown /proc/447/exe 2023-12-06T18:42:56.662 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 448 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.662 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 448 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.662 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 448 root txt unknown /proc/448/exe 2023-12-06T18:42:56.663 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_3 449 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.663 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_3 449 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.663 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_3 449 root txt unknown /proc/449/exe 2023-12-06T18:42:56.663 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 450 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.663 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 450 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.663 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 450 root txt unknown /proc/450/exe 2023-12-06T18:42:56.663 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_4 456 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.663 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_4 456 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.663 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_4 456 root txt unknown /proc/456/exe 2023-12-06T18:42:56.663 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 457 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.663 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 457 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.663 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 457 root txt unknown /proc/457/exe 2023-12-06T18:42:56.663 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_5 458 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.663 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_5 458 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.664 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_5 458 root txt unknown /proc/458/exe 2023-12-06T18:42:56.664 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 459 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.664 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 459 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.664 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 459 root txt unknown /proc/459/exe 2023-12-06T18:42:56.664 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_6 460 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.664 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_6 460 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.664 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_6 460 root txt unknown /proc/460/exe 2023-12-06T18:42:56.664 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 461 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.664 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 461 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.664 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 461 root txt unknown /proc/461/exe 2023-12-06T18:42:56.664 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_7 462 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.664 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_7 462 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.664 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_7 462 root txt unknown /proc/462/exe 2023-12-06T18:42:56.664 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 463 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.664 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 463 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.664 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 463 root txt unknown /proc/463/exe 2023-12-06T18:42:56.665 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_8 464 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.665 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_8 464 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.665 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_8 464 root txt unknown /proc/464/exe 2023-12-06T18:42:56.665 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 465 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.665 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 465 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.665 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 465 root txt unknown /proc/465/exe 2023-12-06T18:42:56.665 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_9 466 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.665 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_9 466 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.665 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_eh_9 466 root txt unknown /proc/466/exe 2023-12-06T18:42:56.665 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 467 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.665 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 467 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.665 INFO:teuthology.orchestra.run.smithi152.stdout:scsi_tmf_ 467 root txt unknown /proc/467/exe 2023-12-06T18:42:56.665 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 468 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.665 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 468 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.665 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 468 root txt unknown /proc/468/exe 2023-12-06T18:42:56.666 INFO:teuthology.orchestra.run.smithi152.stdout:jbd2/sda1 490 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.666 INFO:teuthology.orchestra.run.smithi152.stdout:jbd2/sda1 490 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.666 INFO:teuthology.orchestra.run.smithi152.stdout:jbd2/sda1 490 root txt unknown /proc/490/exe 2023-12-06T18:42:56.666 INFO:teuthology.orchestra.run.smithi152.stdout:ext4-rsv- 491 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.666 INFO:teuthology.orchestra.run.smithi152.stdout:ext4-rsv- 491 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.666 INFO:teuthology.orchestra.run.smithi152.stdout:ext4-rsv- 491 root txt unknown /proc/491/exe 2023-12-06T18:42:56.666 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.666 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.666 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root txt REG 8,1 185496 20360 /usr/lib/systemd/systemd-journald 2023-12-06T18:42:56.666 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 0,24 8388608 58 /run/log/journal/15949e3c88704300882482eddfd7baae/system.journal 2023-12-06T18:42:56.666 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-12-06T18:42:56.666 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.666 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.666 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.667 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.667 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.667 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.667 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.667 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.667 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.667 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.667 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 127680 9972 /usr/lib64/libseccomp.so.2.5.2 2023-12-06T18:42:56.667 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-12-06T18:42:56.667 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-12-06T18:42:56.667 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.667 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.667 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 28552 9804 /usr/lib64/libattr.so.1.1.2501 2023-12-06T18:42:56.667 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.667 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 107424 10180 /usr/lib64/libkmod.so.2.3.6 2023-12-06T18:42:56.668 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.668 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.668 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.668 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 3645472 49253 /usr/lib64/systemd/libsystemd-shared-252.so 2023-12-06T18:42:56.668 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.668 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 41120 9806 /usr/lib64/libacl.so.1.1.2301 2023-12-06T18:42:56.668 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.668 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.668 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.668 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root mem REG 0,24 8 55 /run/systemd/journal/kernel-seqnum 2023-12-06T18:42:56.668 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.668 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 1w CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.668 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 2w CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.668 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 3u unix 0xffff8a575813ee80 0t0 13439 /run/systemd/journal/socket type=DGRAM (CONNECTED) 2023-12-06T18:42:56.669 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 4u unix 0xffff8a5758068880 0t0 13441 /run/systemd/journal/stdout type=STREAM (LISTEN) 2023-12-06T18:42:56.669 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 5u unix 0xffff8a575813d500 0t0 13437 /run/systemd/journal/dev-log type=DGRAM (CONNECTED) 2023-12-06T18:42:56.669 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 6w CHR 1,11 0t0 10 /dev/kmsg 2023-12-06T18:42:56.669 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 7u a_inode 0,14 0 13316 [eventpoll:3,4,5,8,9,10,11,12,13,15,17,18,19,20,22,23,24,25,27,28,29,30,31,32] 2023-12-06T18:42:56.669 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 8u CHR 1,11 0t0 10 /dev/kmsg 2023-12-06T18:42:56.669 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 9u unix 0xffff8a57582c1dc0 0t0 19670 /run/systemd/journal/io.systemd.journal type=STREAM (LISTEN) 2023-12-06T18:42:56.669 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 10r REG 0,20 0 16425 /proc/sys/kernel/hostname 2023-12-06T18:42:56.669 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 11u a_inode 0,14 0 13316 [signalfd] 2023-12-06T18:42:56.669 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 12u a_inode 0,14 0 13316 [signalfd] 2023-12-06T18:42:56.669 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 13u a_inode 0,14 0 13316 [signalfd] 2023-12-06T18:42:56.669 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 14u unix 0xffff8a57582cea40 0t0 19671 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.669 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 15u a_inode 0,14 0 13316 [timerfd] 2023-12-06T18:42:56.669 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 16u REG 0,24 8388608 58 /run/log/journal/15949e3c88704300882482eddfd7baae/system.journal 2023-12-06T18:42:56.669 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 17u unix 0xffff8a5747ce0880 0t0 16637 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.670 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 18u unix 0xffff8a5747ce3fc0 0t0 21598 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.670 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 19u unix 0xffff8a57582c0000 0t0 21022 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.670 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 20u unix 0xffff8a574739fb40 0t0 21599 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.670 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 22u unix 0xffff8a574739cc80 0t0 22744 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.670 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 23u unix 0xffff8a5747d8d940 0t0 21607 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.670 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 24u unix 0xffff8a5748557700 0t0 22748 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.670 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 25u unix 0xffff8a575828f700 0t0 21919 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.670 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 27u unix 0xffff8a5747ce2640 0t0 18892 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.670 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 28u unix 0xffff8a574739f2c0 0t0 22753 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.670 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 29u unix 0xffff8a5747ce72c0 0t0 13960 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.670 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 30u unix 0xffff8a5748416a40 0t0 13961 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.670 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 31u unix 0xffff8a5748410880 0t0 161573 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.671 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-j 566 root 32u unix 0xffff8a574739e1c0 0t0 21144 /run/systemd/journal/stdout type=STREAM (CONNECTED) 2023-12-06T18:42:56.671 INFO:teuthology.orchestra.run.smithi152.stdout:ext4lazyi 572 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.671 INFO:teuthology.orchestra.run.smithi152.stdout:ext4lazyi 572 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.671 INFO:teuthology.orchestra.run.smithi152.stdout:ext4lazyi 572 root txt unknown /proc/572/exe 2023-12-06T18:42:56.671 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.671 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.671 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root txt REG 8,1 597744 11595 /usr/bin/udevadm 2023-12-06T18:42:56.671 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 11730661 64128 /etc/udev/hwdb.bin 2023-12-06T18:42:56.671 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 597073 1143 /etc/selinux/targeted/contexts/files/file_contexts.bin 2023-12-06T18:42:56.671 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 567179 265240 /usr/lib/modules/5.14.0-390.el9.x86_64/modules.symbols.bin 2023-12-06T18:42:56.671 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 822085 265237 /usr/lib/modules/5.14.0-390.el9.x86_64/modules.alias.bin 2023-12-06T18:42:56.671 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.671 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.672 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.672 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.672 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 21219 1167 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2023-12-06T18:42:56.672 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 383474 265246 /usr/lib/modules/5.14.0-390.el9.x86_64/modules.dep.bin 2023-12-06T18:42:56.672 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.672 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-12-06T18:42:56.672 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.672 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.672 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-12-06T18:42:56.672 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.672 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.672 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.672 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.672 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.673 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.673 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 3645472 49253 /usr/lib64/systemd/libsystemd-shared-252.so 2023-12-06T18:42:56.673 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 28552 9804 /usr/lib64/libattr.so.1.1.2501 2023-12-06T18:42:56.673 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 127680 9972 /usr/lib64/libseccomp.so.2.5.2 2023-12-06T18:42:56.673 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-12-06T18:42:56.673 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.673 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.673 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.673 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.673 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 41120 9806 /usr/lib64/libacl.so.1.1.2301 2023-12-06T18:42:56.673 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.673 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.673 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 107424 10180 /usr/lib64/libkmod.so.2.3.6 2023-12-06T18:42:56.673 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 11618 265241 /usr/lib/modules/5.14.0-390.el9.x86_64/modules.builtin.bin 2023-12-06T18:42:56.673 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.673 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root mem REG 0,19 0 19 /sys/fs/selinux/status 2023-12-06T18:42:56.674 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.674 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root 1u unix 0xffff8a5747ce2a80 0t0 313 type=STREAM (CONNECTED) 2023-12-06T18:42:56.674 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root 2u unix 0xffff8a5747ce2a80 0t0 313 type=STREAM (CONNECTED) 2023-12-06T18:42:56.674 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root 3u unix 0xffff8a5747ce9540 0t0 250 /run/udev/control type=SEQPACKET (LISTEN) 2023-12-06T18:42:56.674 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root 4u netlink 0t0 252 KOBJECT_UEVENT 2023-12-06T18:42:56.674 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root 5u unix 0xffff8a5747d97b40 0t0 3305 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.674 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root 6u unix 0xffff8a5747d94840 0t0 3309 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.674 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root 7u unix 0xffff8a5747d961c0 0t0 3310 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.674 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root 8r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.674 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root 9u a_inode 0,14 0 13316 [eventpoll:3,4,6,8,10,11,14] 2023-12-06T18:42:56.674 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root 10u a_inode 0,14 0 13316 [signalfd] 2023-12-06T18:42:56.674 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root 11u a_inode 0,14 0 13316 [timerfd] 2023-12-06T18:42:56.674 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root 12r REG 8,1 11730661 64128 /etc/udev/hwdb.bin 2023-12-06T18:42:56.675 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-u 581 root 14u a_inode 0,14 0 13316 [timerfd] 2023-12-06T18:42:56.675 INFO:teuthology.orchestra.run.smithi152.stdout:ipmi-msgh 623 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.675 INFO:teuthology.orchestra.run.smithi152.stdout:ipmi-msgh 623 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.675 INFO:teuthology.orchestra.run.smithi152.stdout:ipmi-msgh 623 root txt unknown /proc/623/exe 2023-12-06T18:42:56.675 INFO:teuthology.orchestra.run.smithi152.stdout:kipmi0 633 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.675 INFO:teuthology.orchestra.run.smithi152.stdout:kipmi0 633 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.675 INFO:teuthology.orchestra.run.smithi152.stdout:kipmi0 633 root txt unknown /proc/633/exe 2023-12-06T18:42:56.675 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.675 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.675 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc txt REG 8,1 61200 14999 /usr/bin/rpcbind 2023-12-06T18:42:56.675 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.675 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.675 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.675 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.675 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.675 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.676 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.676 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.676 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.676 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.676 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.676 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.676 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.676 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.676 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.676 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.676 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.676 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.676 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.676 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.677 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 196112 11431 /usr/lib64/libtirpc.so.3.0.0 2023-12-06T18:42:56.677 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.677 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.677 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc 1u unix 0xffff8a57582c7700 0t0 19789 type=STREAM (CONNECTED) 2023-12-06T18:42:56.677 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc 2u unix 0xffff8a57582c7700 0t0 19789 type=STREAM (CONNECTED) 2023-12-06T18:42:56.677 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc 3u unix 0xffff8a5747ce3300 0t0 238 /run/rpcbind.sock type=STREAM (LISTEN) 2023-12-06T18:42:56.677 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc 4u IPv4 17661 0t0 TCP *:sunrpc (LISTEN) 2023-12-06T18:42:56.677 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc 5u IPv4 13809 0t0 UDP *:sunrpc 2023-12-06T18:42:56.677 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc 6u IPv6 13815 0t0 TCP *:sunrpc (LISTEN) 2023-12-06T18:42:56.677 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc 7u IPv6 19646 0t0 UDP *:sunrpc 2023-12-06T18:42:56.677 INFO:teuthology.orchestra.run.smithi152.stdout:rpcbind 649 rpc 8rW REG 0,24 0 1037 /run/rpcbind/rpcbind.lock 2023-12-06T18:42:56.677 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.677 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.677 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root txt REG 8,1 144576 11776 /usr/sbin/auditd 2023-12-06T18:42:56.677 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.678 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.678 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.678 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.678 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.678 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.678 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.678 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.678 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.678 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.678 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.678 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.678 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.678 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root mem REG 8,1 143568 9730 /usr/lib64/libauparse.so.0.0.0 2023-12-06T18:42:56.678 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.678 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.679 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.679 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.679 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.679 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root 3u netlink 0t0 13896 AUDIT 2023-12-06T18:42:56.679 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root 4w REG 8,1 111618 3377 /var/log/audit/audit.log 2023-12-06T18:42:56.679 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root 6u unix 0xffff8a575813aa80 0t0 13899 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.679 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root 7u unix 0xffff8a575806f700 0t0 13898 type=STREAM (CONNECTED) 2023-12-06T18:42:56.679 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root 8u unix 0xffff8a5758068000 0t0 13900 type=STREAM (CONNECTED) 2023-12-06T18:42:56.679 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root 9u unix 0xffff8a575806bb80 0t0 13901 type=STREAM (CONNECTED) 2023-12-06T18:42:56.679 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 root 10u a_inode 0,14 0 13316 [eventfd:0] 2023-12-06T18:42:56.679 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.679 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.679 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root txt REG 8,1 144576 11776 /usr/sbin/auditd 2023-12-06T18:42:56.679 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.679 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.679 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.680 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.680 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.680 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.680 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.680 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.680 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.680 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.680 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.680 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.680 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.680 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root mem REG 8,1 143568 9730 /usr/lib64/libauparse.so.0.0.0 2023-12-06T18:42:56.680 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.680 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.680 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.680 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.681 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.681 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root 3u netlink 0t0 13896 AUDIT 2023-12-06T18:42:56.681 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root 4w REG 8,1 111618 3377 /var/log/audit/audit.log 2023-12-06T18:42:56.681 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root 6u unix 0xffff8a575813aa80 0t0 13899 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.681 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root 7u unix 0xffff8a575806f700 0t0 13898 type=STREAM (CONNECTED) 2023-12-06T18:42:56.681 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root 8u unix 0xffff8a5758068000 0t0 13900 type=STREAM (CONNECTED) 2023-12-06T18:42:56.681 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root 9u unix 0xffff8a575806bb80 0t0 13901 type=STREAM (CONNECTED) 2023-12-06T18:42:56.681 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 660 auditd root 10u a_inode 0,14 0 13316 [eventfd:0] 2023-12-06T18:42:56.681 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.681 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.681 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root txt REG 8,1 144576 11776 /usr/sbin/auditd 2023-12-06T18:42:56.681 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.681 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.681 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.682 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.682 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.682 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.682 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.682 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.682 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.682 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.682 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.682 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.682 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.682 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root mem REG 8,1 143568 9730 /usr/lib64/libauparse.so.0.0.0 2023-12-06T18:42:56.682 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.682 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.682 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.682 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.683 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.683 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root 3u netlink 0t0 13896 AUDIT 2023-12-06T18:42:56.683 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root 4w REG 8,1 111618 3377 /var/log/audit/audit.log 2023-12-06T18:42:56.683 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root 6u unix 0xffff8a575813aa80 0t0 13899 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.683 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root 7u unix 0xffff8a575806f700 0t0 13898 type=STREAM (CONNECTED) 2023-12-06T18:42:56.683 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root 8u unix 0xffff8a5758068000 0t0 13900 type=STREAM (CONNECTED) 2023-12-06T18:42:56.684 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root 9u unix 0xffff8a575806bb80 0t0 13901 type=STREAM (CONNECTED) 2023-12-06T18:42:56.684 INFO:teuthology.orchestra.run.smithi152.stdout:auditd 659 662 auditd root 10u a_inode 0,14 0 13316 [eventfd:0] 2023-12-06T18:42:56.684 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.684 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.684 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root txt REG 8,1 19624 11914 /usr/sbin/sedispatch 2023-12-06T18:42:56.684 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.684 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.684 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.684 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.684 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.684 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.684 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.684 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.684 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.685 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.685 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.685 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.685 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root mem REG 8,1 143568 9730 /usr/lib64/libauparse.so.0.0.0 2023-12-06T18:42:56.685 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root mem REG 8,1 343752 10128 /usr/lib64/libdbus-1.so.3.19.13 2023-12-06T18:42:56.685 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.685 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.685 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root 0u unix 0xffff8a575806b740 0t0 13897 type=STREAM (CONNECTED) 2023-12-06T18:42:56.685 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.685 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.685 INFO:teuthology.orchestra.run.smithi152.stdout:sedispatc 661 root 3u unix 0xffff8a5747d8b300 0t0 23350 type=STREAM (CONNECTED) 2023-12-06T18:42:56.685 INFO:teuthology.orchestra.run.smithi152.stdout:rpciod 665 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.685 INFO:teuthology.orchestra.run.smithi152.stdout:rpciod 665 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.685 INFO:teuthology.orchestra.run.smithi152.stdout:rpciod 665 root txt unknown /proc/665/exe 2023-12-06T18:42:56.686 INFO:teuthology.orchestra.run.smithi152.stdout:xprtiod 666 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.686 INFO:teuthology.orchestra.run.smithi152.stdout:xprtiod 666 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.686 INFO:teuthology.orchestra.run.smithi152.stdout:xprtiod 666 root txt unknown /proc/666/exe 2023-12-06T18:42:56.686 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.686 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.686 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus txt REG 8,1 131896 11213 /usr/bin/dbus-broker-launch 2023-12-06T18:42:56.686 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus DEL REG 0,1 2048 /memfd:dbus-broker-log 2023-12-06T18:42:56.686 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.686 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.686 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.686 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.686 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.687 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.687 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.687 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.687 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.687 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.687 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.687 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.687 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-12-06T18:42:56.687 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.687 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.687 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.687 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.687 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus 1u unix 0xffff8a5747ce1100 0t0 552 type=STREAM (CONNECTED) 2023-12-06T18:42:56.687 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus 2u unix 0xffff8a5747ce1100 0t0 552 type=STREAM (CONNECTED) 2023-12-06T18:42:56.688 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus 3u unix 0xffff8a5748412200 0t0 17769 /run/dbus/system_bus_socket type=STREAM (LISTEN) 2023-12-06T18:42:56.688 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus 4u unix 0xffff8a5747cebb80 0t0 562 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.688 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus 5u a_inode 0,14 0 13316 [eventpoll:6,7,8,9,10,11] 2023-12-06T18:42:56.688 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus 6u a_inode 0,14 0 13316 [signalfd] 2023-12-06T18:42:56.688 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus 7r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.688 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus 8u unix 0xffff8a5747ce5500 0t0 564 type=STREAM (CONNECTED) 2023-12-06T18:42:56.688 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus 9u unix 0xffff8a5747d8bfc0 0t0 22743 type=STREAM (CONNECTED) 2023-12-06T18:42:56.688 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus 10u a_inode 0,14 0 13316 [pidfd:703] 2023-12-06T18:42:56.688 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus 11u a_inode 0,14 0 13316 [timerfd] 2023-12-06T18:42:56.688 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 702 dbus 12u REG 0,1 2097152 2048 /memfd:dbus-broker-log (deleted) 2023-12-06T18:42:56.688 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.688 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.688 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus txt REG 8,1 236336 11212 /usr/bin/dbus-broker 2023-12-06T18:42:56.689 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.689 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.689 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.689 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.689 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.689 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.689 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.689 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.689 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus mem REG 0,19 0 19 /sys/fs/selinux/status 2023-12-06T18:42:56.689 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.689 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus 1u unix 0xffff8a5747ce1100 0t0 552 type=STREAM (CONNECTED) 2023-12-06T18:42:56.689 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus 2u unix 0xffff8a5747ce1100 0t0 552 type=STREAM (CONNECTED) 2023-12-06T18:42:56.689 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus 3u sock 0,8 0t0 21099 protocol: NETLINK 2023-12-06T18:42:56.689 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus 4u unix 0xffff8a5747cebb80 0t0 562 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.689 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus 5u a_inode 0,14 0 13316 [eventpoll:6,7,8,9,10,11,12,13,15,16,17,18] 2023-12-06T18:42:56.690 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus 6u a_inode 0,14 0 13316 [signalfd] 2023-12-06T18:42:56.690 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus 7u unix 0xffff8a5748412200 0t0 17769 /run/dbus/system_bus_socket type=STREAM (LISTEN) 2023-12-06T18:42:56.690 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus 8u unix 0xffff8a57484172c0 0t0 21100 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-12-06T18:42:56.690 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus 9u unix 0xffff8a5747ce61c0 0t0 565 type=STREAM (CONNECTED) 2023-12-06T18:42:56.690 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus 10u unix 0xffff8a5747d89100 0t0 21102 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-12-06T18:42:56.690 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus 11u unix 0xffff8a574739b740 0t0 21104 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-12-06T18:42:56.690 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus 12u unix 0xffff8a5747d8e600 0t0 21132 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-12-06T18:42:56.690 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus 13u unix 0xffff8a57582c4c80 0t0 21134 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-12-06T18:42:56.690 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus 15u unix 0xffff8a5758288440 0t0 13954 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-12-06T18:42:56.690 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus 16u unix 0xffff8a5747d8b740 0t0 20331 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-12-06T18:42:56.690 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus 17u unix 0xffff8a5758289dc0 0t0 24774 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-12-06T18:42:56.690 INFO:teuthology.orchestra.run.smithi152.stdout:dbus-brok 703 dbus 18u unix 0xffff8a57582c72c0 0t0 21659 /run/dbus/system_bus_socket type=STREAM (CONNECTED) 2023-12-06T18:42:56.690 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.690 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.690 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root txt REG 8,1 3644720 11633 /usr/sbin/NetworkManager 2023-12-06T18:42:56.691 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 317552 52828 /usr/lib64/NetworkManager/1.44.0-2.el9/libnm-settings-plugin-ifcfg-rh.so 2023-12-06T18:42:56.691 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 343752 10128 /usr/lib64/libdbus-1.so.3.19.13 2023-12-06T18:42:56.691 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-12-06T18:42:56.691 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 62456 9782 /usr/lib64/libjansson.so.4.14.0 2023-12-06T18:42:56.691 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 31792 10954 /usr/lib64/libteamdctl.so.0.1.5 2023-12-06T18:42:56.691 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 57016 52811 /usr/lib64/NetworkManager/1.44.0-2.el9/libnm-device-plugin-team.so 2023-12-06T18:42:56.691 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.691 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-12-06T18:42:56.691 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-12-06T18:42:56.691 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-12-06T18:42:56.691 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.691 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.691 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.691 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:56.691 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:56.692 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:56.692 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.692 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.692 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.692 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.692 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.692 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.692 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.692 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.692 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:56.692 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:56.692 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.692 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.692 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:56.692 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.692 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.693 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.693 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.693 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:56.693 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:56.693 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:56.693 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.693 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.693 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.693 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.693 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.693 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.694 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.694 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 626152 9989 /usr/lib64/libhogweed.so.6.5 2023-12-06T18:42:56.694 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:56.694 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.694 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-12-06T18:42:56.694 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 94112 9901 /usr/lib64/libtasn1.so.6.6.0 2023-12-06T18:42:56.694 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:56.694 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-12-06T18:42:56.694 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.694 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-12-06T18:42:56.694 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-12-06T18:42:56.694 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:56.694 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.695 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 2291576 9994 /usr/lib64/libgnutls.so.30.33.1 2023-12-06T18:42:56.695 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.695 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-12-06T18:42:56.695 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.695 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 32408 11334 /usr/lib64/libndp.so.0.2.0 2023-12-06T18:42:56.695 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.695 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.695 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-12-06T18:42:56.695 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-12-06T18:42:56.695 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-12-06T18:42:56.695 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-12-06T18:42:56.695 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-12-06T18:42:56.695 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-12-06T18:42:56.695 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.695 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-12-06T18:42:56.695 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.696 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 1u unix 0xffff8a574739dd80 0t0 18717 type=STREAM (CONNECTED) 2023-12-06T18:42:56.696 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 2u unix 0xffff8a574739dd80 0t0 18717 type=STREAM (CONNECTED) 2023-12-06T18:42:56.696 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 3u a_inode 0,14 0 13316 [eventfd:6] 2023-12-06T18:42:56.696 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 4u a_inode 0,14 0 13316 [eventfd:7] 2023-12-06T18:42:56.696 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 5u unix 0xffff8a57582ce1c0 0t0 19844 type=DGRAM (UNCONNECTED) 2023-12-06T18:42:56.696 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 6u unix 0xffff8a57582c7b40 0t0 19845 type=STREAM (CONNECTED) 2023-12-06T18:42:56.696 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 7u a_inode 0,14 0 13316 [eventfd:8] 2023-12-06T18:42:56.696 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 8r REG 0,4 0 4026531840 net 2023-12-06T18:42:56.696 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 9r REG 0,4 0 4026532495 mnt 2023-12-06T18:42:56.696 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 10u netlink 0t0 18784 KOBJECT_UEVENT 2023-12-06T18:42:56.696 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 11u netlink 0t0 18785 GENERIC 2023-12-06T18:42:56.696 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 12u netlink 0t0 18786 GENERIC 2023-12-06T18:42:56.696 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 13u netlink 0t0 18787 ROUTE 2023-12-06T18:42:56.696 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 14r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.696 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 15r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.696 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 16u netlink 0t0 603 KOBJECT_UEVENT 2023-12-06T18:42:56.697 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 17u a_inode 0,14 0 13316 [eventpoll] 2023-12-06T18:42:56.697 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 18u unix 0xffff8a575828d0c0 0t0 21136 type=STREAM (CONNECTED) 2023-12-06T18:42:56.697 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 19u a_inode 0,14 0 13316 [eventpoll:21,26] 2023-12-06T18:42:56.697 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 20w FIFO 0,24 0t0 1158 /run/systemd/inhibit/1.ref 2023-12-06T18:42:56.697 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 21u a_inode 0,14 0 13316 [timerfd] 2023-12-06T18:42:56.697 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 23u a_inode 0,14 0 13316 [eventpoll:24,25] 2023-12-06T18:42:56.697 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 24u a_inode 0,14 0 13316 [timerfd] 2023-12-06T18:42:56.697 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 25u pack 22864 0t0 ARP type=SOCK_DGRAM 2023-12-06T18:42:56.697 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 root 26u IPv4 22865 0t0 UDP smithi152.front.sepia.ceph.com:bootpc->store01.front.sepia.ceph.com:bootps 2023-12-06T18:42:56.697 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.697 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.697 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root txt REG 8,1 3644720 11633 /usr/sbin/NetworkManager 2023-12-06T18:42:56.697 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 317552 52828 /usr/lib64/NetworkManager/1.44.0-2.el9/libnm-settings-plugin-ifcfg-rh.so 2023-12-06T18:42:56.697 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 343752 10128 /usr/lib64/libdbus-1.so.3.19.13 2023-12-06T18:42:56.697 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-12-06T18:42:56.698 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 62456 9782 /usr/lib64/libjansson.so.4.14.0 2023-12-06T18:42:56.698 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 31792 10954 /usr/lib64/libteamdctl.so.0.1.5 2023-12-06T18:42:56.698 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 57016 52811 /usr/lib64/NetworkManager/1.44.0-2.el9/libnm-device-plugin-team.so 2023-12-06T18:42:56.698 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.698 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-12-06T18:42:56.698 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-12-06T18:42:56.698 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-12-06T18:42:56.698 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.698 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.698 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.698 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:56.698 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:56.698 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:56.698 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.698 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.699 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.699 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.699 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.699 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.699 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.699 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.699 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:56.699 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:56.699 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.699 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.699 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:56.699 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.699 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.699 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.699 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.700 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:56.700 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:56.700 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:56.700 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.700 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.700 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.700 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.700 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.700 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.700 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.700 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 626152 9989 /usr/lib64/libhogweed.so.6.5 2023-12-06T18:42:56.700 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:56.700 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.700 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-12-06T18:42:56.700 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 94112 9901 /usr/lib64/libtasn1.so.6.6.0 2023-12-06T18:42:56.700 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:56.701 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-12-06T18:42:56.701 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.701 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-12-06T18:42:56.701 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-12-06T18:42:56.701 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:56.701 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.701 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 2291576 9994 /usr/lib64/libgnutls.so.30.33.1 2023-12-06T18:42:56.701 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.701 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-12-06T18:42:56.701 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.701 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 32408 11334 /usr/lib64/libndp.so.0.2.0 2023-12-06T18:42:56.701 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.701 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.701 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-12-06T18:42:56.701 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-12-06T18:42:56.701 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-12-06T18:42:56.701 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-12-06T18:42:56.702 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-12-06T18:42:56.702 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-12-06T18:42:56.702 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.702 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-12-06T18:42:56.702 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.702 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 1u unix 0xffff8a574739dd80 0t0 18717 type=STREAM (CONNECTED) 2023-12-06T18:42:56.702 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 2u unix 0xffff8a574739dd80 0t0 18717 type=STREAM (CONNECTED) 2023-12-06T18:42:56.702 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 3u a_inode 0,14 0 13316 [eventfd:6] 2023-12-06T18:42:56.702 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 4u a_inode 0,14 0 13316 [eventfd:7] 2023-12-06T18:42:56.702 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 5u unix 0xffff8a57582ce1c0 0t0 19844 type=DGRAM (UNCONNECTED) 2023-12-06T18:42:56.702 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 6u unix 0xffff8a57582c7b40 0t0 19845 type=STREAM (CONNECTED) 2023-12-06T18:42:56.702 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 7u a_inode 0,14 0 13316 [eventfd:8] 2023-12-06T18:42:56.703 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 8r REG 0,4 0 4026531840 net 2023-12-06T18:42:56.703 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 9r REG 0,4 0 4026532495 mnt 2023-12-06T18:42:56.703 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 10u netlink 0t0 18784 KOBJECT_UEVENT 2023-12-06T18:42:56.703 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 11u netlink 0t0 18785 GENERIC 2023-12-06T18:42:56.703 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 12u netlink 0t0 18786 GENERIC 2023-12-06T18:42:56.703 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 13u netlink 0t0 18787 ROUTE 2023-12-06T18:42:56.703 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 14r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.703 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 15r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.703 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 16u netlink 0t0 603 KOBJECT_UEVENT 2023-12-06T18:42:56.703 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 17u a_inode 0,14 0 13316 [eventpoll] 2023-12-06T18:42:56.703 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 18u unix 0xffff8a575828d0c0 0t0 21136 type=STREAM (CONNECTED) 2023-12-06T18:42:56.703 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 19u a_inode 0,14 0 13316 [eventpoll:21,26] 2023-12-06T18:42:56.703 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 20w FIFO 0,24 0t0 1158 /run/systemd/inhibit/1.ref 2023-12-06T18:42:56.703 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 21u a_inode 0,14 0 13316 [timerfd] 2023-12-06T18:42:56.704 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 23u a_inode 0,14 0 13316 [eventpoll:24,25] 2023-12-06T18:42:56.704 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 24u a_inode 0,14 0 13316 [timerfd] 2023-12-06T18:42:56.704 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 25u pack 22864 0t0 ARP type=SOCK_DGRAM 2023-12-06T18:42:56.704 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 732 gmain root 26u IPv4 22865 0t0 UDP smithi152.front.sepia.ceph.com:bootpc->store01.front.sepia.ceph.com:bootps 2023-12-06T18:42:56.704 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.704 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.704 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root txt REG 8,1 3644720 11633 /usr/sbin/NetworkManager 2023-12-06T18:42:56.704 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 317552 52828 /usr/lib64/NetworkManager/1.44.0-2.el9/libnm-settings-plugin-ifcfg-rh.so 2023-12-06T18:42:56.704 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 343752 10128 /usr/lib64/libdbus-1.so.3.19.13 2023-12-06T18:42:56.704 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-12-06T18:42:56.704 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 62456 9782 /usr/lib64/libjansson.so.4.14.0 2023-12-06T18:42:56.704 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 31792 10954 /usr/lib64/libteamdctl.so.0.1.5 2023-12-06T18:42:56.704 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 57016 52811 /usr/lib64/NetworkManager/1.44.0-2.el9/libnm-device-plugin-team.so 2023-12-06T18:42:56.704 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.704 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-12-06T18:42:56.705 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-12-06T18:42:56.705 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-12-06T18:42:56.705 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.705 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.705 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.705 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:56.705 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:56.705 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:56.705 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.705 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.705 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.705 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.705 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.705 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.705 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.706 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.706 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:56.706 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:56.706 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.706 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.706 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:56.706 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.706 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.706 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.706 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.706 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:56.706 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:56.706 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:56.706 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.707 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.707 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.707 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.707 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.707 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.707 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.707 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 626152 9989 /usr/lib64/libhogweed.so.6.5 2023-12-06T18:42:56.707 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:56.707 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.707 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-12-06T18:42:56.707 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 94112 9901 /usr/lib64/libtasn1.so.6.6.0 2023-12-06T18:42:56.707 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:56.707 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-12-06T18:42:56.707 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.707 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-12-06T18:42:56.708 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-12-06T18:42:56.708 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:56.708 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.708 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 2291576 9994 /usr/lib64/libgnutls.so.30.33.1 2023-12-06T18:42:56.708 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.708 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-12-06T18:42:56.708 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.708 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 32408 11334 /usr/lib64/libndp.so.0.2.0 2023-12-06T18:42:56.708 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.708 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.708 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-12-06T18:42:56.708 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-12-06T18:42:56.708 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-12-06T18:42:56.708 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-12-06T18:42:56.708 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-12-06T18:42:56.709 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-12-06T18:42:56.709 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.709 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-12-06T18:42:56.709 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.709 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 1u unix 0xffff8a574739dd80 0t0 18717 type=STREAM (CONNECTED) 2023-12-06T18:42:56.709 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 2u unix 0xffff8a574739dd80 0t0 18717 type=STREAM (CONNECTED) 2023-12-06T18:42:56.709 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 3u a_inode 0,14 0 13316 [eventfd:6] 2023-12-06T18:42:56.709 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 4u a_inode 0,14 0 13316 [eventfd:7] 2023-12-06T18:42:56.709 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 5u unix 0xffff8a57582ce1c0 0t0 19844 type=DGRAM (UNCONNECTED) 2023-12-06T18:42:56.709 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 6u unix 0xffff8a57582c7b40 0t0 19845 type=STREAM (CONNECTED) 2023-12-06T18:42:56.709 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 7u a_inode 0,14 0 13316 [eventfd:8] 2023-12-06T18:42:56.709 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 8r REG 0,4 0 4026531840 net 2023-12-06T18:42:56.709 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 9r REG 0,4 0 4026532495 mnt 2023-12-06T18:42:56.709 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 10u netlink 0t0 18784 KOBJECT_UEVENT 2023-12-06T18:42:56.709 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 11u netlink 0t0 18785 GENERIC 2023-12-06T18:42:56.709 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 12u netlink 0t0 18786 GENERIC 2023-12-06T18:42:56.710 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 13u netlink 0t0 18787 ROUTE 2023-12-06T18:42:56.710 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 14r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.710 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 15r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.710 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 16u netlink 0t0 603 KOBJECT_UEVENT 2023-12-06T18:42:56.710 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 17u a_inode 0,14 0 13316 [eventpoll] 2023-12-06T18:42:56.710 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 18u unix 0xffff8a575828d0c0 0t0 21136 type=STREAM (CONNECTED) 2023-12-06T18:42:56.710 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 19u a_inode 0,14 0 13316 [eventpoll:21,26] 2023-12-06T18:42:56.710 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 20w FIFO 0,24 0t0 1158 /run/systemd/inhibit/1.ref 2023-12-06T18:42:56.710 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 21u a_inode 0,14 0 13316 [timerfd] 2023-12-06T18:42:56.710 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 23u a_inode 0,14 0 13316 [eventpoll:24,25] 2023-12-06T18:42:56.710 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 24u a_inode 0,14 0 13316 [timerfd] 2023-12-06T18:42:56.710 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 25u pack 22864 0t0 ARP type=SOCK_DGRAM 2023-12-06T18:42:56.710 INFO:teuthology.orchestra.run.smithi152.stdout:NetworkMa 704 733 gdbus root 26u IPv4 22865 0t0 UDP smithi152.front.sepia.ceph.com:bootpc->store01.front.sepia.ceph.com:bootps 2023-12-06T18:42:56.711 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.711 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.711 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 root txt REG 8,1 61904 10507 /usr/sbin/irqbalance 2023-12-06T18:42:56.711 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.711 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 root mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.711 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.711 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 root DEL REG 8,1 9998 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:56.711 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 root mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.711 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.711 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.711 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.711 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 root 1u unix 0xffff8a5747398cc0 0t0 18727 type=STREAM (CONNECTED) 2023-12-06T18:42:56.711 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 root 2u unix 0xffff8a5747398cc0 0t0 18727 type=STREAM (CONNECTED) 2023-12-06T18:42:56.711 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 root 3u a_inode 0,14 0 13316 [eventfd:1] 2023-12-06T18:42:56.711 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 root 4u a_inode 0,14 0 13316 [eventfd:2] 2023-12-06T18:42:56.711 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 root 5u unix 0xffff8a57582c6600 0t0 19818 /run/irqbalance/irqbalance707.sock type=STREAM (LISTEN) 2023-12-06T18:42:56.712 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 root 6u unix 0xffff8a57582cd500 0t0 19965 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.712 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 714 gmain root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.712 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 714 gmain root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.712 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 714 gmain root txt REG 8,1 61904 10507 /usr/sbin/irqbalance 2023-12-06T18:42:56.712 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 714 gmain root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.712 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 714 gmain root mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.712 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 714 gmain root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.712 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 714 gmain root DEL REG 8,1 9998 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:56.712 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 714 gmain root mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.712 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 714 gmain root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.712 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 714 gmain root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.712 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 714 gmain root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.713 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 714 gmain root 1u unix 0xffff8a5747398cc0 0t0 18727 type=STREAM (CONNECTED) 2023-12-06T18:42:56.713 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 714 gmain root 2u unix 0xffff8a5747398cc0 0t0 18727 type=STREAM (CONNECTED) 2023-12-06T18:42:56.713 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 714 gmain root 3u a_inode 0,14 0 13316 [eventfd:1] 2023-12-06T18:42:56.713 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 714 gmain root 4u a_inode 0,14 0 13316 [eventfd:2] 2023-12-06T18:42:56.713 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 714 gmain root 5u unix 0xffff8a57582c6600 0t0 19818 /run/irqbalance/irqbalance707.sock type=STREAM (LISTEN) 2023-12-06T18:42:56.713 INFO:teuthology.orchestra.run.smithi152.stdout:irqbalanc 707 714 gmain root 6u unix 0xffff8a57582cd500 0t0 19965 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.713 INFO:teuthology.orchestra.run.smithi152.stdout:lsmd 708 libstoragemgmt cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.713 INFO:teuthology.orchestra.run.smithi152.stdout:lsmd 708 libstoragemgmt rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.713 INFO:teuthology.orchestra.run.smithi152.stdout:lsmd 708 libstoragemgmt txt REG 8,1 32280 11292 /usr/bin/lsmd 2023-12-06T18:42:56.713 INFO:teuthology.orchestra.run.smithi152.stdout:lsmd 708 libstoragemgmt mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.713 INFO:teuthology.orchestra.run.smithi152.stdout:lsmd 708 libstoragemgmt mem REG 8,1 56912 10811 /usr/lib64/libconfig.so.11.0.2 2023-12-06T18:42:56.713 INFO:teuthology.orchestra.run.smithi152.stdout:lsmd 708 libstoragemgmt mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.713 INFO:teuthology.orchestra.run.smithi152.stdout:lsmd 708 libstoragemgmt 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.713 INFO:teuthology.orchestra.run.smithi152.stdout:lsmd 708 libstoragemgmt 1u unix 0xffff8a5747d89540 0t0 22745 type=STREAM (CONNECTED) 2023-12-06T18:42:56.714 INFO:teuthology.orchestra.run.smithi152.stdout:lsmd 708 libstoragemgmt 2u unix 0xffff8a5747d89540 0t0 22745 type=STREAM (CONNECTED) 2023-12-06T18:42:56.714 INFO:teuthology.orchestra.run.smithi152.stdout:lsmd 708 libstoragemgmt 3uW REG 0,24 0 1112 /run/lsm/ipc/.lsmd-ipc-lock 2023-12-06T18:42:56.714 INFO:teuthology.orchestra.run.smithi152.stdout:lsmd 708 libstoragemgmt 5u unix 0xffff8a5747d88440 0t0 22754 /var/run/lsm/ipc/simc type=STREAM (LISTEN) 2023-12-06T18:42:56.714 INFO:teuthology.orchestra.run.smithi152.stdout:lsmd 708 libstoragemgmt 6u unix 0xffff8a5747d8d0c0 0t0 22755 /var/run/lsm/ipc/sim type=STREAM (LISTEN) 2023-12-06T18:42:56.714 INFO:teuthology.orchestra.run.smithi152.stdout:mcelog 709 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.714 INFO:teuthology.orchestra.run.smithi152.stdout:mcelog 709 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.714 INFO:teuthology.orchestra.run.smithi152.stdout:mcelog 709 root txt REG 8,1 194568 12461 /usr/sbin/mcelog 2023-12-06T18:42:56.714 INFO:teuthology.orchestra.run.smithi152.stdout:mcelog 709 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.714 INFO:teuthology.orchestra.run.smithi152.stdout:mcelog 709 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.714 INFO:teuthology.orchestra.run.smithi152.stdout:mcelog 709 root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.714 INFO:teuthology.orchestra.run.smithi152.stdout:mcelog 709 root 1u unix 0xffff8a5748550000 0t0 21608 type=STREAM (CONNECTED) 2023-12-06T18:42:56.714 INFO:teuthology.orchestra.run.smithi152.stdout:mcelog 709 root 2u unix 0xffff8a5748550000 0t0 21608 type=STREAM (CONNECTED) 2023-12-06T18:42:56.714 INFO:teuthology.orchestra.run.smithi152.stdout:mcelog 709 root 3r CHR 10,227 0t0 142 /dev/mcelog 2023-12-06T18:42:56.714 INFO:teuthology.orchestra.run.smithi152.stdout:mcelog 709 root 4u unix 0xffff8a57582c5d80 0t0 19821 /var/run/mcelog-client type=STREAM (LISTEN) 2023-12-06T18:42:56.714 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.715 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.715 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-12-06T18:42:56.715 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.715 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.715 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.715 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.715 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.715 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.715 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.715 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-12-06T18:42:56.715 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.715 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.715 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.715 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.715 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.716 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.716 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.716 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.716 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.716 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.716 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-12-06T18:42:56.716 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.716 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-12-06T18:42:56.716 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.716 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.716 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-12-06T18:42:56.716 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-12-06T18:42:56.716 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-12-06T18:42:56.716 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.717 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.717 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.717 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.717 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd 3r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.717 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd 4u a_inode 0,14 0 13316 [eventfd:3] 2023-12-06T18:42:56.717 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd 5u a_inode 0,14 0 13316 [eventfd:4] 2023-12-06T18:42:56.717 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd 6u unix 0xffff8a5747d89980 0t0 22784 type=STREAM (CONNECTED) 2023-12-06T18:42:56.717 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd 7u a_inode 0,14 0 13316 [eventfd:5] 2023-12-06T18:42:56.717 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd 8r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.717 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd 9u a_inode 0,14 0 13316 [eventfd:9] 2023-12-06T18:42:56.717 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 polkitd 10u unix 0xffff8a5747d95d80 0t0 22786 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.717 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.717 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.717 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-12-06T18:42:56.717 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.718 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.718 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.718 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.718 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.718 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.718 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.718 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-12-06T18:42:56.718 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.718 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.718 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.718 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.718 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.718 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.718 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.718 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.719 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.719 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.719 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-12-06T18:42:56.719 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.719 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-12-06T18:42:56.719 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.719 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.719 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-12-06T18:42:56.719 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-12-06T18:42:56.719 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-12-06T18:42:56.719 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.719 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.719 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.719 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.719 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd 3r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.720 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd 4u a_inode 0,14 0 13316 [eventfd:3] 2023-12-06T18:42:56.720 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd 5u a_inode 0,14 0 13316 [eventfd:4] 2023-12-06T18:42:56.720 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd 6u unix 0xffff8a5747d89980 0t0 22784 type=STREAM (CONNECTED) 2023-12-06T18:42:56.720 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd 7u a_inode 0,14 0 13316 [eventfd:5] 2023-12-06T18:42:56.720 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd 8r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.720 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd 9u a_inode 0,14 0 13316 [eventfd:9] 2023-12-06T18:42:56.720 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 722 gmain polkitd 10u unix 0xffff8a5747d95d80 0t0 22786 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.720 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.720 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.720 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-12-06T18:42:56.720 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.720 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.720 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.720 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.721 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.721 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.721 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.721 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-12-06T18:42:56.721 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.721 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.721 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.721 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.721 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.721 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.721 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.721 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.721 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.721 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.721 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-12-06T18:42:56.722 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.722 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-12-06T18:42:56.722 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.722 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.722 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-12-06T18:42:56.722 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-12-06T18:42:56.722 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-12-06T18:42:56.722 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.722 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.722 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.722 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.722 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd 3r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.722 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd 4u a_inode 0,14 0 13316 [eventfd:3] 2023-12-06T18:42:56.722 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd 5u a_inode 0,14 0 13316 [eventfd:4] 2023-12-06T18:42:56.722 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd 6u unix 0xffff8a5747d89980 0t0 22784 type=STREAM (CONNECTED) 2023-12-06T18:42:56.723 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd 7u a_inode 0,14 0 13316 [eventfd:5] 2023-12-06T18:42:56.723 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd 8r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.723 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd 9u a_inode 0,14 0 13316 [eventfd:9] 2023-12-06T18:42:56.723 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 723 gdbus polkitd 10u unix 0xffff8a5747d95d80 0t0 22786 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.723 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.723 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.723 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-12-06T18:42:56.723 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.723 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.723 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.723 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.723 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.723 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.723 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.724 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-12-06T18:42:56.724 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.724 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.724 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.724 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.724 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.724 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.724 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.724 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.724 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.724 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.724 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-12-06T18:42:56.724 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.724 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-12-06T18:42:56.724 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.725 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.725 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-12-06T18:42:56.725 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-12-06T18:42:56.725 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-12-06T18:42:56.725 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.725 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.725 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.725 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.725 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd 3r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.725 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd 4u a_inode 0,14 0 13316 [eventfd:3] 2023-12-06T18:42:56.725 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd 5u a_inode 0,14 0 13316 [eventfd:4] 2023-12-06T18:42:56.725 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd 6u unix 0xffff8a5747d89980 0t0 22784 type=STREAM (CONNECTED) 2023-12-06T18:42:56.725 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd 7u a_inode 0,14 0 13316 [eventfd:5] 2023-12-06T18:42:56.725 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd 8r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.725 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd 9u a_inode 0,14 0 13316 [eventfd:9] 2023-12-06T18:42:56.726 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 724 JS\x20Hel polkitd 10u unix 0xffff8a5747d95d80 0t0 22786 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.726 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.726 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.726 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-12-06T18:42:56.726 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.726 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.726 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.726 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.727 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.727 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.727 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.727 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-12-06T18:42:56.727 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.727 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.727 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.727 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.727 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.727 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.727 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.727 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.727 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.728 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.728 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-12-06T18:42:56.728 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.728 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-12-06T18:42:56.728 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.728 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.728 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-12-06T18:42:56.728 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-12-06T18:42:56.728 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-12-06T18:42:56.728 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.728 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.728 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.728 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.729 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd 3r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.729 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd 4u a_inode 0,14 0 13316 [eventfd:3] 2023-12-06T18:42:56.729 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd 5u a_inode 0,14 0 13316 [eventfd:4] 2023-12-06T18:42:56.729 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd 6u unix 0xffff8a5747d89980 0t0 22784 type=STREAM (CONNECTED) 2023-12-06T18:42:56.729 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd 7u a_inode 0,14 0 13316 [eventfd:5] 2023-12-06T18:42:56.729 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd 8r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.729 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd 9u a_inode 0,14 0 13316 [eventfd:9] 2023-12-06T18:42:56.729 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 725 JS\x20Hel polkitd 10u unix 0xffff8a5747d95d80 0t0 22786 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.729 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.729 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.729 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-12-06T18:42:56.729 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.729 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.729 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.729 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.730 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.730 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.730 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.730 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-12-06T18:42:56.730 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.730 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.730 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.730 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.730 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.730 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.730 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.730 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.730 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.730 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.730 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-12-06T18:42:56.731 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.731 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-12-06T18:42:56.731 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.731 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.731 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-12-06T18:42:56.731 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-12-06T18:42:56.731 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-12-06T18:42:56.731 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.731 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.731 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.731 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.731 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd 3r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.731 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd 4u a_inode 0,14 0 13316 [eventfd:3] 2023-12-06T18:42:56.731 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd 5u a_inode 0,14 0 13316 [eventfd:4] 2023-12-06T18:42:56.731 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd 6u unix 0xffff8a5747d89980 0t0 22784 type=STREAM (CONNECTED) 2023-12-06T18:42:56.732 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd 7u a_inode 0,14 0 13316 [eventfd:5] 2023-12-06T18:42:56.732 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd 8r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.732 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd 9u a_inode 0,14 0 13316 [eventfd:9] 2023-12-06T18:42:56.732 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 726 JS\x20Hel polkitd 10u unix 0xffff8a5747d95d80 0t0 22786 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.732 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.732 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.732 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-12-06T18:42:56.732 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.732 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.732 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.732 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.732 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.732 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.732 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.732 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-12-06T18:42:56.732 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.733 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.733 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.733 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.733 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.733 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.733 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.733 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.733 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.733 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.733 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-12-06T18:42:56.733 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.733 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-12-06T18:42:56.733 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.733 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.734 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-12-06T18:42:56.734 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-12-06T18:42:56.734 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-12-06T18:42:56.734 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.734 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.734 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.734 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.734 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd 3r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.734 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd 4u a_inode 0,14 0 13316 [eventfd:3] 2023-12-06T18:42:56.734 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd 5u a_inode 0,14 0 13316 [eventfd:4] 2023-12-06T18:42:56.734 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd 6u unix 0xffff8a5747d89980 0t0 22784 type=STREAM (CONNECTED) 2023-12-06T18:42:56.734 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd 7u a_inode 0,14 0 13316 [eventfd:5] 2023-12-06T18:42:56.734 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd 8r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.734 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd 9u a_inode 0,14 0 13316 [eventfd:9] 2023-12-06T18:42:56.734 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 727 JS\x20Hel polkitd 10u unix 0xffff8a5747d95d80 0t0 22786 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.735 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.735 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.735 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-12-06T18:42:56.735 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.735 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.735 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.735 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.735 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.735 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.735 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.735 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-12-06T18:42:56.735 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.735 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.735 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.735 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.736 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.736 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.736 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.736 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.736 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.736 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.736 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-12-06T18:42:56.736 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.736 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-12-06T18:42:56.736 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.736 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.736 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-12-06T18:42:56.736 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-12-06T18:42:56.736 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-12-06T18:42:56.736 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.736 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.737 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.737 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.737 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd 3r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.737 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd 4u a_inode 0,14 0 13316 [eventfd:3] 2023-12-06T18:42:56.737 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd 5u a_inode 0,14 0 13316 [eventfd:4] 2023-12-06T18:42:56.737 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd 6u unix 0xffff8a5747d89980 0t0 22784 type=STREAM (CONNECTED) 2023-12-06T18:42:56.737 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd 7u a_inode 0,14 0 13316 [eventfd:5] 2023-12-06T18:42:56.737 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd 8r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.737 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd 9u a_inode 0,14 0 13316 [eventfd:9] 2023-12-06T18:42:56.737 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 728 JS\x20Hel polkitd 10u unix 0xffff8a5747d95d80 0t0 22786 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.737 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.737 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.737 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-12-06T18:42:56.737 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.737 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.738 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.738 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.738 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.738 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.738 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.738 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-12-06T18:42:56.738 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.738 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.738 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.738 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.738 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.738 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.738 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.738 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.739 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.739 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.739 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-12-06T18:42:56.739 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.739 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-12-06T18:42:56.739 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.739 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.739 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-12-06T18:42:56.739 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-12-06T18:42:56.739 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-12-06T18:42:56.739 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.739 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.739 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.739 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.740 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd 3r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.740 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd 4u a_inode 0,14 0 13316 [eventfd:3] 2023-12-06T18:42:56.740 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd 5u a_inode 0,14 0 13316 [eventfd:4] 2023-12-06T18:42:56.740 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd 6u unix 0xffff8a5747d89980 0t0 22784 type=STREAM (CONNECTED) 2023-12-06T18:42:56.740 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd 7u a_inode 0,14 0 13316 [eventfd:5] 2023-12-06T18:42:56.740 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd 8r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.740 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd 9u a_inode 0,14 0 13316 [eventfd:9] 2023-12-06T18:42:56.740 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 729 JS\x20Hel polkitd 10u unix 0xffff8a5747d95d80 0t0 22786 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.740 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.740 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.740 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-12-06T18:42:56.740 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.740 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.740 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.740 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.740 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.741 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.741 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.741 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-12-06T18:42:56.741 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.741 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.741 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.741 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.741 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.741 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.741 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.741 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.741 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.741 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.741 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-12-06T18:42:56.742 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.742 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-12-06T18:42:56.742 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.742 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.742 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-12-06T18:42:56.742 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-12-06T18:42:56.742 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-12-06T18:42:56.742 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.742 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.742 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.742 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.742 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd 3r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.742 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd 4u a_inode 0,14 0 13316 [eventfd:3] 2023-12-06T18:42:56.742 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd 5u a_inode 0,14 0 13316 [eventfd:4] 2023-12-06T18:42:56.742 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd 6u unix 0xffff8a5747d89980 0t0 22784 type=STREAM (CONNECTED) 2023-12-06T18:42:56.743 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd 7u a_inode 0,14 0 13316 [eventfd:5] 2023-12-06T18:42:56.743 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd 8r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.743 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd 9u a_inode 0,14 0 13316 [eventfd:9] 2023-12-06T18:42:56.743 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 730 JS\x20Hel polkitd 10u unix 0xffff8a5747d95d80 0t0 22786 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.743 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.743 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.743 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-12-06T18:42:56.743 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.743 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.743 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.743 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.743 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.743 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.743 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.743 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-12-06T18:42:56.743 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.744 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.744 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.744 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.744 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.744 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.744 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.744 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.744 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.744 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.744 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-12-06T18:42:56.744 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.744 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-12-06T18:42:56.744 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.744 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.744 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-12-06T18:42:56.744 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-12-06T18:42:56.745 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-12-06T18:42:56.745 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.745 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.745 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.745 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.745 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd 3r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.745 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd 4u a_inode 0,14 0 13316 [eventfd:3] 2023-12-06T18:42:56.745 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd 5u a_inode 0,14 0 13316 [eventfd:4] 2023-12-06T18:42:56.745 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd 6u unix 0xffff8a5747d89980 0t0 22784 type=STREAM (CONNECTED) 2023-12-06T18:42:56.745 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd 7u a_inode 0,14 0 13316 [eventfd:5] 2023-12-06T18:42:56.745 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd 8r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.745 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd 9u a_inode 0,14 0 13316 [eventfd:9] 2023-12-06T18:42:56.745 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 731 JS\x20Hel polkitd 10u unix 0xffff8a5747d95d80 0t0 22786 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.746 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.746 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.746 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd txt REG 8,1 145440 49355 /usr/lib/polkit-1/polkitd 2023-12-06T18:42:56.746 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.746 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.746 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.746 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.746 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.746 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.746 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.746 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 29097776 36983 /usr/lib64/polkit-1/libmozjs-78.so 2023-12-06T18:42:56.746 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.746 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.747 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.747 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.747 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.747 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.747 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.747 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.747 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.747 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.747 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-12-06T18:42:56.747 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.747 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-12-06T18:42:56.747 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.747 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.747 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-12-06T18:42:56.747 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-12-06T18:42:56.748 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 131296 10178 /usr/lib64/libpolkit-gobject-1.so.0.0.0 2023-12-06T18:42:56.748 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.748 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.748 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.748 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.748 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd 3r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.748 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd 4u a_inode 0,14 0 13316 [eventfd:3] 2023-12-06T18:42:56.748 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd 5u a_inode 0,14 0 13316 [eventfd:4] 2023-12-06T18:42:56.748 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd 6u unix 0xffff8a5747d89980 0t0 22784 type=STREAM (CONNECTED) 2023-12-06T18:42:56.748 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd 7u a_inode 0,14 0 13316 [eventfd:5] 2023-12-06T18:42:56.748 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd 8r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.748 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd 9u a_inode 0,14 0 13316 [eventfd:9] 2023-12-06T18:42:56.748 INFO:teuthology.orchestra.run.smithi152.stdout:polkitd 710 737 polkitd polkitd 10u unix 0xffff8a5747d95d80 0t0 22786 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.748 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.748 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.748 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root txt REG 8,1 622048 12498 /usr/sbin/smartd 2023-12-06T18:42:56.749 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.749 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.749 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.749 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.749 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.749 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.749 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.749 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.749 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.749 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.749 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.749 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.749 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.749 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.749 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.749 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.750 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.750 INFO:teuthology.orchestra.run.smithi152.stdout:smartd 711 root 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.750 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.750 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.750 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root txt REG 8,1 306896 20362 /usr/lib/systemd/systemd-logind 2023-12-06T18:42:56.750 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-12-06T18:42:56.750 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 597073 1143 /etc/selinux/targeted/contexts/files/file_contexts.bin 2023-12-06T18:42:56.750 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.750 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.750 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-12-06T18:42:56.750 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.750 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.750 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.750 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.751 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.751 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.751 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.751 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.751 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.751 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 28552 9804 /usr/lib64/libattr.so.1.1.2501 2023-12-06T18:42:56.751 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 127680 9972 /usr/lib64/libseccomp.so.2.5.2 2023-12-06T18:42:56.751 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-12-06T18:42:56.751 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-12-06T18:42:56.751 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.751 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.751 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.751 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 107424 10180 /usr/lib64/libkmod.so.2.3.6 2023-12-06T18:42:56.751 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.751 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.751 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.752 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 3645472 49253 /usr/lib64/systemd/libsystemd-shared-252.so 2023-12-06T18:42:56.752 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.752 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 41120 9806 /usr/lib64/libacl.so.1.1.2301 2023-12-06T18:42:56.752 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.752 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 21219 1167 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2023-12-06T18:42:56.752 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.752 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root mem REG 0,19 0 19 /sys/fs/selinux/status 2023-12-06T18:42:56.752 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.752 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 1u unix 0xffff8a574739ea40 0t0 18739 type=STREAM (CONNECTED) 2023-12-06T18:42:56.752 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 2u unix 0xffff8a574739ea40 0t0 18739 type=STREAM (CONNECTED) 2023-12-06T18:42:56.752 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 3u unix 0xffff8a57473b0000 0t0 18764 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.752 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 4u a_inode 0,14 0 13316 [eventpoll:5,6,7,8,9,10,11,12,13,14,15,16,17,18,20] 2023-12-06T18:42:56.752 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 5u a_inode 0,14 0 13316 [signalfd] 2023-12-06T18:42:56.752 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 6u a_inode 0,14 0 13316 [timerfd] 2023-12-06T18:42:56.752 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 7r REG 0,21 4096 22932 /sys/devices/virtual/tty/tty0/active 2023-12-06T18:42:56.753 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 8u netlink 0t0 18775 KOBJECT_UEVENT 2023-12-06T18:42:56.753 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 9r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.753 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 10r FIFO 0,24 0t0 1158 /run/systemd/inhibit/1.ref 2023-12-06T18:42:56.753 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 11u netlink 0t0 18776 KOBJECT_UEVENT 2023-12-06T18:42:56.753 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 12u netlink 0t0 18777 KOBJECT_UEVENT 2023-12-06T18:42:56.753 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 13u netlink 0t0 18778 KOBJECT_UEVENT 2023-12-06T18:42:56.753 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 14u unix 0xffff8a574739f700 0t0 18779 type=STREAM (CONNECTED) 2023-12-06T18:42:56.753 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 15u a_inode 0,14 0 13316 [timerfd] 2023-12-06T18:42:56.753 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 16u CHR 13,65 0t0 138 /dev/input/event1 2023-12-06T18:42:56.753 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 17u CHR 13,64 0t0 137 /dev/input/event0 2023-12-06T18:42:56.753 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 18u CHR 13,66 0t0 148 /dev/input/event2 2023-12-06T18:42:56.753 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 19u CHR 4,6 0t0 25 /dev/tty6 2023-12-06T18:42:56.753 INFO:teuthology.orchestra.run.smithi152.stdout:systemd-l 713 root 20r FIFO 0,24 0t0 1635 /run/systemd/sessions/4.ref 2023-12-06T18:42:56.753 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.754 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.754 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root txt REG 8,1 968800 13744 /usr/sbin/sshd 2023-12-06T18:42:56.754 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.754 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.754 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.754 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.754 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.754 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.754 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.754 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.754 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.754 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.754 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.754 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.754 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.755 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.755 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.755 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-12-06T18:42:56.755 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.755 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.755 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.755 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.755 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.755 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.755 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.755 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.755 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-12-06T18:42:56.755 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.755 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.755 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.756 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root 1u unix 0xffff8a5747ce4840 0t0 650 type=STREAM (CONNECTED) 2023-12-06T18:42:56.756 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root 2u unix 0xffff8a5747ce4840 0t0 650 type=STREAM (CONNECTED) 2023-12-06T18:42:56.756 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root 3u IPv4 19869 0t0 TCP *:ssh (LISTEN) 2023-12-06T18:42:56.756 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 766 root 4u IPv6 19871 0t0 TCP *:ssh (LISTEN) 2023-12-06T18:42:56.756 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.756 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.756 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-12-06T18:42:56.756 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.756 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.756 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.756 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 204696 37013 /usr/lib64/girepository-1.0/GLib-2.0.typelib 2023-12-06T18:42:56.756 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.756 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.756 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.756 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.756 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-12-06T18:42:56.757 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-12-06T18:42:56.757 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.757 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-12-06T18:42:56.757 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 143200 10195 /usr/lib64/libgirepository-1.0.so.1.0.0 2023-12-06T18:42:56.757 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 357112 39324 /usr/lib64/python3.9/site-packages/gi/_gi.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.757 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.757 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.757 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 24736 39204 /usr/lib64/python3.9/site-packages/_dbus_glib_bindings.so 2023-12-06T18:42:56.757 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-12-06T18:42:56.757 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.757 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.757 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.757 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.757 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.757 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.757 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.758 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.758 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 343752 10128 /usr/lib64/libdbus-1.so.3.19.13 2023-12-06T18:42:56.758 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.758 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 171376 39203 /usr/lib64/python3.9/site-packages/_dbus_bindings.so 2023-12-06T18:42:56.758 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.758 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.758 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.758 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-12-06T18:42:56.758 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.758 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.758 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.758 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.758 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.758 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-12-06T18:42:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.759 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.760 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.760 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.760 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.760 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.760 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root 1u unix 0xffff8a5748411dc0 0t0 17856 type=STREAM (CONNECTED) 2023-12-06T18:42:56.760 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root 2u unix 0xffff8a5748411dc0 0t0 17856 type=STREAM (CONNECTED) 2023-12-06T18:42:56.760 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root 3w REG 8,1 10371 1216 /var/log/tuned/tuned.log 2023-12-06T18:42:56.760 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root 4u sock 0,8 0t0 19879 protocol: NETLINK 2023-12-06T18:42:56.760 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root 5u unix 0xffff8a57582c3b80 0t0 19883 type=STREAM (CONNECTED) 2023-12-06T18:42:56.760 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root 6u a_inode 0,14 0 13316 [eventfd:17] 2023-12-06T18:42:56.760 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root 7u unix 0xffff8a575828d940 0t0 21183 type=STREAM (CONNECTED) 2023-12-06T18:42:56.760 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 root 8u unix 0xffff8a5758289540 0t0 21184 type=STREAM (CONNECTED) 2023-12-06T18:42:56.760 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.760 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.760 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-12-06T18:42:56.761 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.761 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 130464 38212 /usr/lib64/python3.9/lib-dynload/_ctypes.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.761 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 107736 38215 /usr/lib64/python3.9/lib-dynload/_datetime.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.761 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 204696 37013 /usr/lib64/girepository-1.0/GLib-2.0.typelib 2023-12-06T18:42:56.761 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.761 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.761 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.761 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.761 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 1932656 10165 /usr/lib64/libgio-2.0.so.0.6800.4 2023-12-06T18:42:56.761 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 19480 10169 /usr/lib64/libgmodule-2.0.so.0.6800.4 2023-12-06T18:42:56.761 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.761 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 375136 10171 /usr/lib64/libgobject-2.0.so.0.6800.4 2023-12-06T18:42:56.761 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 143200 10195 /usr/lib64/libgirepository-1.0.so.1.0.0 2023-12-06T18:42:56.761 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 357112 39324 /usr/lib64/python3.9/site-packages/gi/_gi.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.761 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 489696 10000 /usr/lib64/libpcre.so.1.2.12 2023-12-06T18:42:56.762 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 1303016 10167 /usr/lib64/libglib-2.0.so.0.6800.4 2023-12-06T18:42:56.762 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 24736 39204 /usr/lib64/python3.9/site-packages/_dbus_glib_bindings.so 2023-12-06T18:42:56.762 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 197952 9776 /usr/lib64/libexpat.so.1.8.10 2023-12-06T18:42:56.762 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 71408 38249 /usr/lib64/python3.9/lib-dynload/pyexpat.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.762 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.762 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.762 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.762 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.762 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.762 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.762 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.762 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 343752 10128 /usr/lib64/libdbus-1.so.3.19.13 2023-12-06T18:42:56.762 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.762 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 171376 39203 /usr/lib64/python3.9/site-packages/_dbus_bindings.so 2023-12-06T18:42:56.763 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.763 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.763 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.763 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-12-06T18:42:56.763 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.763 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.763 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 50232 41979 /usr/lib64/python3.9/lib-dynload/_blake2.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.763 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.763 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.763 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 69048 38220 /usr/lib64/python3.9/lib-dynload/_hashlib.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.763 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 24160 41992 /usr/lib64/python3.9/lib-dynload/_random.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.763 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 124184 41989 /usr/lib64/python3.9/lib-dynload/_pickle.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.764 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.764 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.764 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.764 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 15984 38200 /usr/lib64/python3.9/lib-dynload/_bisect.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.764 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 15560 38227 /usr/lib64/python3.9/lib-dynload/_opcode.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.764 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.764 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.764 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.764 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-12-06T18:42:56.764 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.764 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.764 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.764 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.764 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.764 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 20528 41991 /usr/lib64/python3.9/lib-dynload/_queue.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.764 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.765 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.765 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root 1u unix 0xffff8a5748411dc0 0t0 17856 type=STREAM (CONNECTED) 2023-12-06T18:42:56.765 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root 2u unix 0xffff8a5748411dc0 0t0 17856 type=STREAM (CONNECTED) 2023-12-06T18:42:56.765 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root 3w REG 8,1 10371 1216 /var/log/tuned/tuned.log 2023-12-06T18:42:56.765 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root 4u sock 0,8 0t0 19879 protocol: NETLINK 2023-12-06T18:42:56.765 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root 5u unix 0xffff8a57582c3b80 0t0 19883 type=STREAM (CONNECTED) 2023-12-06T18:42:56.765 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root 6u a_inode 0,14 0 13316 [eventfd:17] 2023-12-06T18:42:56.765 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root 7u unix 0xffff8a575828d940 0t0 21183 type=STREAM (CONNECTED) 2023-12-06T18:42:56.765 INFO:teuthology.orchestra.run.smithi152.stdout:tuned 767 852 tuned root 8u unix 0xffff8a5758289540 0t0 21184 type=STREAM (CONNECTED) 2023-12-06T18:42:56.765 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.765 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.765 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root txt REG 8,1 127680 15021 /usr/sbin/gssproxy 2023-12-06T18:42:56.765 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-12-06T18:42:56.765 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.765 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.765 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 70960 15011 /usr/lib64/libev.so.4.0.0 2023-12-06T18:42:56.766 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 16168 15018 /usr/lib64/libverto-libev.so.1.0.0 2023-12-06T18:42:56.766 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.766 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-12-06T18:42:56.766 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-12-06T18:42:56.766 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-12-06T18:42:56.766 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.766 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.766 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.766 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 15912 9764 /usr/lib64/libbasicobjects.so.0.1.0 2023-12-06T18:42:56.766 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.766 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 20304 9968 /usr/lib64/libpath_utils.so.1.0.1 2023-12-06T18:42:56.766 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 58392 9766 /usr/lib64/libcollection.so.4.1.1 2023-12-06T18:42:56.766 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.766 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.767 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.767 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.767 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.767 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 142168 11395 /usr/lib64/libgssrpc.so.4.2 2023-12-06T18:42:56.767 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.767 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 16056 9768 /usr/lib64/libref_array.so.1.2.1 2023-12-06T18:42:56.767 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 120640 9970 /usr/lib64/libini_config.so.5.2.1 2023-12-06T18:42:56.767 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 28472 11392 /usr/lib64/libverto.so.1.0.0 2023-12-06T18:42:56.767 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.767 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.767 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 58368 9689 /usr/lib64/libpopt.so.0.0.1 2023-12-06T18:42:56.767 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-12-06T18:42:56.767 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-12-06T18:42:56.767 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-12-06T18:42:56.767 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-12-06T18:42:56.767 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-12-06T18:42:56.768 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-12-06T18:42:56.768 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.768 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-12-06T18:42:56.768 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.768 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root 1u unix 0xffff8a5747ce2200 0t0 641 type=STREAM (CONNECTED) 2023-12-06T18:42:56.768 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root 2u unix 0xffff8a5747ce2200 0t0 641 type=STREAM (CONNECTED) 2023-12-06T18:42:56.768 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root 3u unix 0xffff8a5747cef700 0t0 654 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.768 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root 4r FIFO 0,13 0t0 659 pipe 2023-12-06T18:42:56.768 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root 5u a_inode 0,14 0 13316 [eventpoll:4,6,7,8] 2023-12-06T18:42:56.768 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root 6u a_inode 0,14 0 13316 [eventfd:16] 2023-12-06T18:42:56.768 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root 7u unix 0xffff8a5747ce6a40 0t0 655 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2023-12-06T18:42:56.768 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root 8u unix 0xffff8a5747ce1dc0 0t0 656 /run/gssproxy.sock type=STREAM (LISTEN) 2023-12-06T18:42:56.768 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 root 9w FIFO 0,13 0t0 659 pipe 2023-12-06T18:42:56.768 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.768 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.769 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root txt REG 8,1 127680 15021 /usr/sbin/gssproxy 2023-12-06T18:42:56.769 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-12-06T18:42:56.769 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.769 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.769 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 70960 15011 /usr/lib64/libev.so.4.0.0 2023-12-06T18:42:56.769 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 16168 15018 /usr/lib64/libverto-libev.so.1.0.0 2023-12-06T18:42:56.769 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.769 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-12-06T18:42:56.769 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-12-06T18:42:56.769 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-12-06T18:42:56.769 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.769 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.769 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.769 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 15912 9764 /usr/lib64/libbasicobjects.so.0.1.0 2023-12-06T18:42:56.769 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.769 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 20304 9968 /usr/lib64/libpath_utils.so.1.0.1 2023-12-06T18:42:56.770 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 58392 9766 /usr/lib64/libcollection.so.4.1.1 2023-12-06T18:42:56.770 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.770 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.770 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.770 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.770 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.770 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 142168 11395 /usr/lib64/libgssrpc.so.4.2 2023-12-06T18:42:56.770 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.770 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 16056 9768 /usr/lib64/libref_array.so.1.2.1 2023-12-06T18:42:56.770 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 120640 9970 /usr/lib64/libini_config.so.5.2.1 2023-12-06T18:42:56.770 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 28472 11392 /usr/lib64/libverto.so.1.0.0 2023-12-06T18:42:56.770 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.770 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.770 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 58368 9689 /usr/lib64/libpopt.so.0.0.1 2023-12-06T18:42:56.771 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-12-06T18:42:56.771 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-12-06T18:42:56.771 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-12-06T18:42:56.771 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-12-06T18:42:56.771 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-12-06T18:42:56.771 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-12-06T18:42:56.771 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.771 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-12-06T18:42:56.771 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.771 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root 1u unix 0xffff8a5747ce2200 0t0 641 type=STREAM (CONNECTED) 2023-12-06T18:42:56.771 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root 2u unix 0xffff8a5747ce2200 0t0 641 type=STREAM (CONNECTED) 2023-12-06T18:42:56.771 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root 3u unix 0xffff8a5747cef700 0t0 654 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.771 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root 4r FIFO 0,13 0t0 659 pipe 2023-12-06T18:42:56.771 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root 5u a_inode 0,14 0 13316 [eventpoll:4,6,7,8] 2023-12-06T18:42:56.771 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root 6u a_inode 0,14 0 13316 [eventfd:16] 2023-12-06T18:42:56.771 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root 7u unix 0xffff8a5747ce6a40 0t0 655 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2023-12-06T18:42:56.772 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root 8u unix 0xffff8a5747ce1dc0 0t0 656 /run/gssproxy.sock type=STREAM (LISTEN) 2023-12-06T18:42:56.772 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 772 gssproxy root 9w FIFO 0,13 0t0 659 pipe 2023-12-06T18:42:56.772 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.772 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.772 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root txt REG 8,1 127680 15021 /usr/sbin/gssproxy 2023-12-06T18:42:56.772 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-12-06T18:42:56.772 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.772 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.772 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 70960 15011 /usr/lib64/libev.so.4.0.0 2023-12-06T18:42:56.772 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 16168 15018 /usr/lib64/libverto-libev.so.1.0.0 2023-12-06T18:42:56.772 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.772 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-12-06T18:42:56.772 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-12-06T18:42:56.772 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-12-06T18:42:56.772 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.773 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.773 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.773 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 15912 9764 /usr/lib64/libbasicobjects.so.0.1.0 2023-12-06T18:42:56.773 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.773 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 20304 9968 /usr/lib64/libpath_utils.so.1.0.1 2023-12-06T18:42:56.773 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 58392 9766 /usr/lib64/libcollection.so.4.1.1 2023-12-06T18:42:56.773 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.773 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.773 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.773 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.773 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.773 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 142168 11395 /usr/lib64/libgssrpc.so.4.2 2023-12-06T18:42:56.773 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.773 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 16056 9768 /usr/lib64/libref_array.so.1.2.1 2023-12-06T18:42:56.773 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 120640 9970 /usr/lib64/libini_config.so.5.2.1 2023-12-06T18:42:56.774 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 28472 11392 /usr/lib64/libverto.so.1.0.0 2023-12-06T18:42:56.774 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.774 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.774 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 58368 9689 /usr/lib64/libpopt.so.0.0.1 2023-12-06T18:42:56.774 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-12-06T18:42:56.774 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-12-06T18:42:56.774 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-12-06T18:42:56.774 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-12-06T18:42:56.774 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-12-06T18:42:56.774 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-12-06T18:42:56.774 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.774 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-12-06T18:42:56.774 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.774 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root 1u unix 0xffff8a5747ce2200 0t0 641 type=STREAM (CONNECTED) 2023-12-06T18:42:56.774 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root 2u unix 0xffff8a5747ce2200 0t0 641 type=STREAM (CONNECTED) 2023-12-06T18:42:56.775 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root 3u unix 0xffff8a5747cef700 0t0 654 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.775 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root 4r FIFO 0,13 0t0 659 pipe 2023-12-06T18:42:56.775 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root 5u a_inode 0,14 0 13316 [eventpoll:4,6,7,8] 2023-12-06T18:42:56.775 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root 6u a_inode 0,14 0 13316 [eventfd:16] 2023-12-06T18:42:56.775 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root 7u unix 0xffff8a5747ce6a40 0t0 655 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2023-12-06T18:42:56.775 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root 8u unix 0xffff8a5747ce1dc0 0t0 656 /run/gssproxy.sock type=STREAM (LISTEN) 2023-12-06T18:42:56.775 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 773 gssproxy root 9w FIFO 0,13 0t0 659 pipe 2023-12-06T18:42:56.775 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.775 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.775 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root txt REG 8,1 127680 15021 /usr/sbin/gssproxy 2023-12-06T18:42:56.775 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-12-06T18:42:56.775 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.775 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.775 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 70960 15011 /usr/lib64/libev.so.4.0.0 2023-12-06T18:42:56.775 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 16168 15018 /usr/lib64/libverto-libev.so.1.0.0 2023-12-06T18:42:56.776 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.776 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-12-06T18:42:56.776 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-12-06T18:42:56.776 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-12-06T18:42:56.776 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.776 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.776 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.776 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 15912 9764 /usr/lib64/libbasicobjects.so.0.1.0 2023-12-06T18:42:56.776 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.776 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 20304 9968 /usr/lib64/libpath_utils.so.1.0.1 2023-12-06T18:42:56.776 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 58392 9766 /usr/lib64/libcollection.so.4.1.1 2023-12-06T18:42:56.776 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.776 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.776 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.776 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.776 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.777 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 142168 11395 /usr/lib64/libgssrpc.so.4.2 2023-12-06T18:42:56.777 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.777 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 16056 9768 /usr/lib64/libref_array.so.1.2.1 2023-12-06T18:42:56.777 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 120640 9970 /usr/lib64/libini_config.so.5.2.1 2023-12-06T18:42:56.777 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 28472 11392 /usr/lib64/libverto.so.1.0.0 2023-12-06T18:42:56.777 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.777 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.777 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 58368 9689 /usr/lib64/libpopt.so.0.0.1 2023-12-06T18:42:56.777 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-12-06T18:42:56.777 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-12-06T18:42:56.778 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-12-06T18:42:56.778 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-12-06T18:42:56.778 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-12-06T18:42:56.778 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-12-06T18:42:56.778 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.778 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-12-06T18:42:56.778 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.778 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root 1u unix 0xffff8a5747ce2200 0t0 641 type=STREAM (CONNECTED) 2023-12-06T18:42:56.778 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root 2u unix 0xffff8a5747ce2200 0t0 641 type=STREAM (CONNECTED) 2023-12-06T18:42:56.778 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root 3u unix 0xffff8a5747cef700 0t0 654 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.778 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root 4r FIFO 0,13 0t0 659 pipe 2023-12-06T18:42:56.778 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root 5u a_inode 0,14 0 13316 [eventpoll:4,6,7,8] 2023-12-06T18:42:56.778 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root 6u a_inode 0,14 0 13316 [eventfd:16] 2023-12-06T18:42:56.778 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root 7u unix 0xffff8a5747ce6a40 0t0 655 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2023-12-06T18:42:56.779 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root 8u unix 0xffff8a5747ce1dc0 0t0 656 /run/gssproxy.sock type=STREAM (LISTEN) 2023-12-06T18:42:56.779 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 774 gssproxy root 9w FIFO 0,13 0t0 659 pipe 2023-12-06T18:42:56.779 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.779 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.779 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root txt REG 8,1 127680 15021 /usr/sbin/gssproxy 2023-12-06T18:42:56.779 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-12-06T18:42:56.779 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.779 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.779 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 70960 15011 /usr/lib64/libev.so.4.0.0 2023-12-06T18:42:56.779 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 16168 15018 /usr/lib64/libverto-libev.so.1.0.0 2023-12-06T18:42:56.779 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.779 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-12-06T18:42:56.779 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-12-06T18:42:56.779 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-12-06T18:42:56.779 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.779 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.780 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.780 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 15912 9764 /usr/lib64/libbasicobjects.so.0.1.0 2023-12-06T18:42:56.780 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.780 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 20304 9968 /usr/lib64/libpath_utils.so.1.0.1 2023-12-06T18:42:56.780 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 58392 9766 /usr/lib64/libcollection.so.4.1.1 2023-12-06T18:42:56.780 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.780 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.780 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.780 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.780 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.780 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 142168 11395 /usr/lib64/libgssrpc.so.4.2 2023-12-06T18:42:56.780 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.780 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 16056 9768 /usr/lib64/libref_array.so.1.2.1 2023-12-06T18:42:56.780 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 120640 9970 /usr/lib64/libini_config.so.5.2.1 2023-12-06T18:42:56.780 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 28472 11392 /usr/lib64/libverto.so.1.0.0 2023-12-06T18:42:56.781 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.781 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.781 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 58368 9689 /usr/lib64/libpopt.so.0.0.1 2023-12-06T18:42:56.781 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-12-06T18:42:56.781 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-12-06T18:42:56.781 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-12-06T18:42:56.781 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-12-06T18:42:56.781 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-12-06T18:42:56.781 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-12-06T18:42:56.781 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.781 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-12-06T18:42:56.781 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.781 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root 1u unix 0xffff8a5747ce2200 0t0 641 type=STREAM (CONNECTED) 2023-12-06T18:42:56.781 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root 2u unix 0xffff8a5747ce2200 0t0 641 type=STREAM (CONNECTED) 2023-12-06T18:42:56.781 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root 3u unix 0xffff8a5747cef700 0t0 654 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.782 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root 4r FIFO 0,13 0t0 659 pipe 2023-12-06T18:42:56.782 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root 5u a_inode 0,14 0 13316 [eventpoll:4,6,7,8] 2023-12-06T18:42:56.782 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root 6u a_inode 0,14 0 13316 [eventfd:16] 2023-12-06T18:42:56.782 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root 7u unix 0xffff8a5747ce6a40 0t0 655 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2023-12-06T18:42:56.782 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root 8u unix 0xffff8a5747ce1dc0 0t0 656 /run/gssproxy.sock type=STREAM (LISTEN) 2023-12-06T18:42:56.782 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 775 gssproxy root 9w FIFO 0,13 0t0 659 pipe 2023-12-06T18:42:56.782 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.782 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.782 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root txt REG 8,1 127680 15021 /usr/sbin/gssproxy 2023-12-06T18:42:56.783 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-12-06T18:42:56.783 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.783 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.783 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 70960 15011 /usr/lib64/libev.so.4.0.0 2023-12-06T18:42:56.783 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 16168 15018 /usr/lib64/libverto-libev.so.1.0.0 2023-12-06T18:42:56.783 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.783 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-12-06T18:42:56.783 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-12-06T18:42:56.783 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-12-06T18:42:56.783 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.783 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.783 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.783 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 15912 9764 /usr/lib64/libbasicobjects.so.0.1.0 2023-12-06T18:42:56.783 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.783 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 20304 9968 /usr/lib64/libpath_utils.so.1.0.1 2023-12-06T18:42:56.784 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 58392 9766 /usr/lib64/libcollection.so.4.1.1 2023-12-06T18:42:56.784 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.784 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.784 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.784 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.784 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.784 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 142168 11395 /usr/lib64/libgssrpc.so.4.2 2023-12-06T18:42:56.784 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.784 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 16056 9768 /usr/lib64/libref_array.so.1.2.1 2023-12-06T18:42:56.784 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 120640 9970 /usr/lib64/libini_config.so.5.2.1 2023-12-06T18:42:56.784 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 28472 11392 /usr/lib64/libverto.so.1.0.0 2023-12-06T18:42:56.784 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.784 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.784 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 58368 9689 /usr/lib64/libpopt.so.0.0.1 2023-12-06T18:42:56.784 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-12-06T18:42:56.784 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-12-06T18:42:56.785 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-12-06T18:42:56.785 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-12-06T18:42:56.785 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-12-06T18:42:56.785 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-12-06T18:42:56.785 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.785 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-12-06T18:42:56.785 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.785 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root 1u unix 0xffff8a5747ce2200 0t0 641 type=STREAM (CONNECTED) 2023-12-06T18:42:56.785 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root 2u unix 0xffff8a5747ce2200 0t0 641 type=STREAM (CONNECTED) 2023-12-06T18:42:56.785 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root 3u unix 0xffff8a5747cef700 0t0 654 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.785 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root 4r FIFO 0,13 0t0 659 pipe 2023-12-06T18:42:56.785 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root 5u a_inode 0,14 0 13316 [eventpoll:4,6,7,8] 2023-12-06T18:42:56.785 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root 6u a_inode 0,14 0 13316 [eventfd:16] 2023-12-06T18:42:56.785 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root 7u unix 0xffff8a5747ce6a40 0t0 655 /var/lib/gssproxy/default.sock type=STREAM (LISTEN) 2023-12-06T18:42:56.785 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root 8u unix 0xffff8a5747ce1dc0 0t0 656 /run/gssproxy.sock type=STREAM (LISTEN) 2023-12-06T18:42:56.786 INFO:teuthology.orchestra.run.smithi152.stdout:gssproxy 771 776 gssproxy root 9w FIFO 0,13 0t0 659 pipe 2023-12-06T18:42:56.786 INFO:teuthology.orchestra.run.smithi152.stdout:atd 778 root cwd DIR 8,1 4096 1657 /var/spool/at 2023-12-06T18:42:56.786 INFO:teuthology.orchestra.run.smithi152.stdout:atd 778 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.786 INFO:teuthology.orchestra.run.smithi152.stdout:atd 778 root txt REG 8,1 32680 12473 /usr/sbin/atd 2023-12-06T18:42:56.786 INFO:teuthology.orchestra.run.smithi152.stdout:atd 778 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.786 INFO:teuthology.orchestra.run.smithi152.stdout:atd 778 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.786 INFO:teuthology.orchestra.run.smithi152.stdout:atd 778 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.786 INFO:teuthology.orchestra.run.smithi152.stdout:atd 778 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.786 INFO:teuthology.orchestra.run.smithi152.stdout:atd 778 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-12-06T18:42:56.786 INFO:teuthology.orchestra.run.smithi152.stdout:atd 778 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.786 INFO:teuthology.orchestra.run.smithi152.stdout:atd 778 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.786 INFO:teuthology.orchestra.run.smithi152.stdout:atd 778 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-12-06T18:42:56.786 INFO:teuthology.orchestra.run.smithi152.stdout:atd 778 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.786 INFO:teuthology.orchestra.run.smithi152.stdout:atd 778 root 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.787 INFO:teuthology.orchestra.run.smithi152.stdout:atd 778 root 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.787 INFO:teuthology.orchestra.run.smithi152.stdout:atd 778 root 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.787 INFO:teuthology.orchestra.run.smithi152.stdout:atd 778 root 3uW REG 0,24 4 1180 /run/atd.pid 2023-12-06T18:42:56.787 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.787 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.787 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root txt REG 8,1 74552 11234 /usr/sbin/crond 2023-12-06T18:42:56.787 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-12-06T18:42:56.787 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.787 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.787 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.787 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-12-06T18:42:56.787 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-12-06T18:42:56.787 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-12-06T18:42:56.787 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.787 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.788 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-12-06T18:42:56.788 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.788 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.788 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-12-06T18:42:56.788 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.788 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-12-06T18:42:56.788 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-12-06T18:42:56.788 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-12-06T18:42:56.788 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-12-06T18:42:56.788 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-12-06T18:42:56.788 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-12-06T18:42:56.788 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.788 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-12-06T18:42:56.788 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.788 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root 1u unix 0xffff8a574739ee80 0t0 18898 type=STREAM (CONNECTED) 2023-12-06T18:42:56.788 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root 2u unix 0xffff8a574739ee80 0t0 18898 type=STREAM (CONNECTED) 2023-12-06T18:42:56.789 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root 3uW REG 0,24 4 1182 /run/crond.pid 2023-12-06T18:42:56.789 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root 4u unix 0xffff8a57473b2ec0 0t0 18901 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.789 INFO:teuthology.orchestra.run.smithi152.stdout:crond 779 root 5r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.789 INFO:teuthology.orchestra.run.smithi152.stdout:nrpe 863 nrpe cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.789 INFO:teuthology.orchestra.run.smithi152.stdout:nrpe 863 nrpe rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.789 INFO:teuthology.orchestra.run.smithi152.stdout:nrpe 863 nrpe txt REG 8,1 80984 12959 /usr/sbin/nrpe 2023-12-06T18:42:56.789 INFO:teuthology.orchestra.run.smithi152.stdout:nrpe 863 nrpe mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.789 INFO:teuthology.orchestra.run.smithi152.stdout:nrpe 863 nrpe mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.789 INFO:teuthology.orchestra.run.smithi152.stdout:nrpe 863 nrpe mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.789 INFO:teuthology.orchestra.run.smithi152.stdout:nrpe 863 nrpe mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.789 INFO:teuthology.orchestra.run.smithi152.stdout:nrpe 863 nrpe mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.789 INFO:teuthology.orchestra.run.smithi152.stdout:nrpe 863 nrpe 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.789 INFO:teuthology.orchestra.run.smithi152.stdout:nrpe 863 nrpe 1w CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.789 INFO:teuthology.orchestra.run.smithi152.stdout:nrpe 863 nrpe 2w CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.790 INFO:teuthology.orchestra.run.smithi152.stdout:nrpe 863 nrpe 3u IPv4 19913 0t0 TCP *:nrpe (LISTEN) 2023-12-06T18:42:56.790 INFO:teuthology.orchestra.run.smithi152.stdout:nrpe 863 nrpe 4u unix 0xffff8a57582cdd80 0t0 19905 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.790 INFO:teuthology.orchestra.run.smithi152.stdout:nrpe 863 nrpe 5u IPv6 19914 0t0 TCP *:nrpe (LISTEN) 2023-12-06T18:42:56.790 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1303 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.790 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1303 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.790 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1303 root txt REG 8,1 58072 13690 /usr/sbin/agetty 2023-12-06T18:42:56.790 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1303 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.790 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1303 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.790 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1303 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.790 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1303 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.790 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1303 root 0u CHR 4,1 0t0 20 /dev/tty1 2023-12-06T18:42:56.790 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1303 root 1u CHR 4,1 0t0 20 /dev/tty1 2023-12-06T18:42:56.790 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1303 root 2u CHR 4,1 0t0 20 /dev/tty1 2023-12-06T18:42:56.790 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1303 root 4r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.790 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.791 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.791 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root txt REG 8,1 58072 13690 /usr/sbin/agetty 2023-12-06T18:42:56.791 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-12-06T18:42:56.791 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.791 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.791 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-12-06T18:42:56.791 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-12-06T18:42:56.791 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-12-06T18:42:56.791 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.791 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-12-06T18:42:56.791 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-12-06T18:42:56.791 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-12-06T18:42:56.791 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-12-06T18:42:56.791 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-12-06T18:42:56.791 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-12-06T18:42:56.792 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.792 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-12-06T18:42:56.792 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root 0u CHR 4,65 0t0 114 /dev/ttyS1 2023-12-06T18:42:56.792 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root 1u CHR 4,65 0t0 114 /dev/ttyS1 2023-12-06T18:42:56.792 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root 2u CHR 4,65 0t0 114 /dev/ttyS1 2023-12-06T18:42:56.792 INFO:teuthology.orchestra.run.smithi152.stdout:agetty 1305 root 4r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.792 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/5 1309 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.792 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/5 1309 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.792 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/5 1309 root txt unknown /proc/1309/exe 2023-12-06T18:42:56.792 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.792 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.792 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu txt REG 8,1 102120 20353 /usr/lib/systemd/systemd 2023-12-06T18:42:56.792 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu DEL REG 8,1 1027 /etc/selinux/targeted/contexts/files/file_contexts.bin 2023-12-06T18:42:56.792 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.792 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.792 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.793 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.793 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-12-06T18:42:56.793 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.793 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu DEL REG 8,1 1168 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2023-12-06T18:42:56.793 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.793 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 28552 9804 /usr/lib64/libattr.so.1.1.2501 2023-12-06T18:42:56.793 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.793 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.793 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 3645472 49253 /usr/lib64/systemd/libsystemd-shared-252.so 2023-12-06T18:42:56.793 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.793 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.793 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-12-06T18:42:56.793 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.793 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.793 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 2174088 49251 /usr/lib64/systemd/libsystemd-core-252.so 2023-12-06T18:42:56.793 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.794 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.794 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.794 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 41120 9806 /usr/lib64/libacl.so.1.1.2301 2023-12-06T18:42:56.794 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.794 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.794 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 107424 10180 /usr/lib64/libkmod.so.2.3.6 2023-12-06T18:42:56.794 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.794 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-12-06T18:42:56.794 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 127680 9972 /usr/lib64/libseccomp.so.2.5.2 2023-12-06T18:42:56.794 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.794 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.794 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2023-12-06T18:42:56.794 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.794 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 1u unix 0xffff8a5758288000 0t0 26809 type=STREAM (CONNECTED) 2023-12-06T18:42:56.795 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 2u unix 0xffff8a5758288000 0t0 26809 type=STREAM (CONNECTED) 2023-12-06T18:42:56.795 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 3u unix 0xffff8a575829c400 0t0 26837 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.795 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 4u a_inode 0,14 0 13316 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2023-12-06T18:42:56.795 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 5u a_inode 0,14 0 13316 [signalfd] 2023-12-06T18:42:56.795 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 6r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.795 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 7r DIR 0,25 0 3391 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2023-12-06T18:42:56.795 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 8u a_inode 0,14 0 13316 [timerfd] 2023-12-06T18:42:56.795 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 9u a_inode 0,14 0 13316 [eventpoll:10,13] 2023-12-06T18:42:56.795 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 10r REG 0,20 0 26826 /proc/4852/mountinfo 2023-12-06T18:42:56.795 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 11r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.795 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 12u unix 0xffff8a575828ea40 0t0 26859 /run/user/1000/bus type=STREAM (LISTEN) 2023-12-06T18:42:56.795 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 13r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.795 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 14r REG 0,20 0 4026532076 /proc/swaps 2023-12-06T18:42:56.795 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 15u netlink 0t0 26841 KOBJECT_UEVENT 2023-12-06T18:42:56.795 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 16u unix 0xffff8a575829d0c0 0t0 26846 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2023-12-06T18:42:56.795 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 17u unix 0xffff8a5758298440 0t0 26847 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.796 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 18u unix 0xffff8a575829fb40 0t0 26848 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.796 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 19u unix 0xffff8a575828aec0 0t0 26849 /run/user/1000/systemd/private type=STREAM (LISTEN) 2023-12-06T18:42:56.796 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 20u unix 0xffff8a575828ee80 0t0 26851 type=STREAM (CONNECTED) 2023-12-06T18:42:56.796 INFO:teuthology.orchestra.run.smithi152.stdout:systemd 4852 ubuntu 21u a_inode 0,14 0 13316 [timerfd] 2023-12-06T18:42:56.796 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.796 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.796 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu txt REG 8,1 102120 20353 /usr/lib/systemd/systemd 2023-12-06T18:42:56.796 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-12-06T18:42:56.796 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-12-06T18:42:56.796 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-12-06T18:42:56.796 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-12-06T18:42:56.796 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-12-06T18:42:56.796 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-12-06T18:42:56.796 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 44152 32291 /usr/lib64/security/pam_namespace.so 2023-12-06T18:42:56.796 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 15184 32309 /usr/lib64/security/pam_umask.so 2023-12-06T18:42:56.796 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-12-06T18:42:56.797 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 15240 32287 /usr/lib64/security/pam_loginuid.so 2023-12-06T18:42:56.797 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 27616 32299 /usr/lib64/security/pam_selinux.so 2023-12-06T18:42:56.797 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-12-06T18:42:56.797 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu DEL REG 8,1 63926 /etc/selinux/targeted/contexts/files/file_contexts.bin 2023-12-06T18:42:56.797 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.797 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.797 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.797 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.797 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.797 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.797 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 28552 9804 /usr/lib64/libattr.so.1.1.2501 2023-12-06T18:42:56.797 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.797 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.797 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-12-06T18:42:56.797 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.797 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.798 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.798 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.798 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.798 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 283152 10156 /usr/lib64/libmount.so.1.1.0 2023-12-06T18:42:56.798 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 3645472 49253 /usr/lib64/systemd/libsystemd-shared-252.so 2023-12-06T18:42:56.798 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.798 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-12-06T18:42:56.798 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.798 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.798 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 107424 10180 /usr/lib64/libkmod.so.2.3.6 2023-12-06T18:42:56.798 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.798 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 2174088 49251 /usr/lib64/systemd/libsystemd-core-252.so 2023-12-06T18:42:56.798 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 41120 9806 /usr/lib64/libacl.so.1.1.2301 2023-12-06T18:42:56.798 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-12-06T18:42:56.798 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 127680 9972 /usr/lib64/libseccomp.so.2.5.2 2023-12-06T18:42:56.799 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.799 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu DEL REG 8,1 63928 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2023-12-06T18:42:56.799 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.799 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2023-12-06T18:42:56.799 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.799 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu 1u unix 0xffff8a5758288000 0t0 26809 type=STREAM (CONNECTED) 2023-12-06T18:42:56.799 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu 2u unix 0xffff8a5758288000 0t0 26809 type=STREAM (CONNECTED) 2023-12-06T18:42:56.799 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu 3u a_inode 0,14 0 13316 [eventfd:10] 2023-12-06T18:42:56.799 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu 4u a_inode 0,14 0 13316 [eventfd:11] 2023-12-06T18:42:56.799 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu 6w FIFO 0,13 0t0 26813 pipe 2023-12-06T18:42:56.799 INFO:teuthology.orchestra.run.smithi152.stdout:(sd-pam) 4854 ubuntu 7u unix 0xffff8a575829b300 0t0 26829 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.799 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.799 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.799 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root txt REG 8,1 769928 12237 /usr/sbin/rsyslogd 2023-12-06T18:42:56.799 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 0,24 8388608 58 /run/log/journal/15949e3c88704300882482eddfd7baae/system.journal 2023-12-06T18:42:56.799 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 8,1 33592 12207 /usr/lib64/rsyslog/imjournal.so 2023-12-06T18:42:56.800 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.800 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.800 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.800 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.800 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 8,1 42248 12214 /usr/lib64/rsyslog/imuxsock.so 2023-12-06T18:42:56.800 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 8,1 32808 12215 /usr/lib64/rsyslog/lmnet.so 2023-12-06T18:42:56.800 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.800 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.800 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.800 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.800 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.800 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-12-06T18:42:56.800 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.800 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 8,1 52856 12201 /usr/lib64/libfastjson.so.4.3.0 2023-12-06T18:42:56.800 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 8,1 16032 12200 /usr/lib64/libestr.so.0.0.0 2023-12-06T18:42:56.800 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.801 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.801 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.801 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root 1w CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.801 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root 2w CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.801 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root 3r CHR 1,9 0t0 9 /dev/urandom 2023-12-06T18:42:56.801 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root 4w REG 8,1 1200987 1642 /var/log/secure 2023-12-06T18:42:56.801 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root 5u unix 0xffff8a5747cec400 0t0 25989 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.801 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root 6r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.801 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root 8w REG 8,1 1448868 1641 /var/log/messages 2023-12-06T18:42:56.801 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 root 9r REG 0,24 8388608 58 /run/log/journal/15949e3c88704300882482eddfd7baae/system.journal 2023-12-06T18:42:56.801 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.801 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.801 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root txt REG 8,1 769928 12237 /usr/sbin/rsyslogd 2023-12-06T18:42:56.801 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 0,24 8388608 58 /run/log/journal/15949e3c88704300882482eddfd7baae/system.journal 2023-12-06T18:42:56.801 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 8,1 33592 12207 /usr/lib64/rsyslog/imjournal.so 2023-12-06T18:42:56.801 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.802 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.802 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.802 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.802 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 8,1 42248 12214 /usr/lib64/rsyslog/imuxsock.so 2023-12-06T18:42:56.802 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 8,1 32808 12215 /usr/lib64/rsyslog/lmnet.so 2023-12-06T18:42:56.802 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.802 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.802 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.802 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.802 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.802 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-12-06T18:42:56.802 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.802 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 8,1 52856 12201 /usr/lib64/libfastjson.so.4.3.0 2023-12-06T18:42:56.802 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 8,1 16032 12200 /usr/lib64/libestr.so.0.0.0 2023-12-06T18:42:56.802 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.803 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.803 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.803 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root 1w CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.803 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root 2w CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.803 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root 3r CHR 1,9 0t0 9 /dev/urandom 2023-12-06T18:42:56.803 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root 4w REG 8,1 1200987 1642 /var/log/secure 2023-12-06T18:42:56.803 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root 5u unix 0xffff8a5747cec400 0t0 25989 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.803 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root 6r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.803 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root 8w REG 8,1 1448868 1641 /var/log/messages 2023-12-06T18:42:56.803 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5119 in:imjour root 9r REG 0,24 8388608 58 /run/log/journal/15949e3c88704300882482eddfd7baae/system.journal 2023-12-06T18:42:56.803 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.803 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.803 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root txt REG 8,1 769928 12237 /usr/sbin/rsyslogd 2023-12-06T18:42:56.803 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 0,24 8388608 58 /run/log/journal/15949e3c88704300882482eddfd7baae/system.journal 2023-12-06T18:42:56.803 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 8,1 33592 12207 /usr/lib64/rsyslog/imjournal.so 2023-12-06T18:42:56.804 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.804 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.804 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.804 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.804 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 8,1 42248 12214 /usr/lib64/rsyslog/imuxsock.so 2023-12-06T18:42:56.804 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 8,1 32808 12215 /usr/lib64/rsyslog/lmnet.so 2023-12-06T18:42:56.804 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.804 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.804 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.804 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.804 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.804 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 8,1 36328 9712 /usr/lib64/libuuid.so.1.3.0 2023-12-06T18:42:56.804 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.804 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 8,1 52856 12201 /usr/lib64/libfastjson.so.4.3.0 2023-12-06T18:42:56.804 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 8,1 16032 12200 /usr/lib64/libestr.so.0.0.0 2023-12-06T18:42:56.805 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.805 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.805 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.805 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root 1w CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.805 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root 2w CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.805 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root 3r CHR 1,9 0t0 9 /dev/urandom 2023-12-06T18:42:56.805 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root 4w REG 8,1 1200987 1642 /var/log/secure 2023-12-06T18:42:56.805 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root 5u unix 0xffff8a5747cec400 0t0 25989 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.805 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root 6r a_inode 0,14 0 13316 inotify 2023-12-06T18:42:56.805 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root 8w REG 8,1 1448868 1641 /var/log/messages 2023-12-06T18:42:56.805 INFO:teuthology.orchestra.run.smithi152.stdout:rsyslogd 5117 5120 rs:main root 9r REG 0,24 8388608 58 /run/log/journal/15949e3c88704300882482eddfd7baae/system.journal 2023-12-06T18:42:56.805 INFO:teuthology.orchestra.run.smithi152.stdout:tls-strp 60485 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.805 INFO:teuthology.orchestra.run.smithi152.stdout:tls-strp 60485 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.805 INFO:teuthology.orchestra.run.smithi152.stdout:tls-strp 60485 root txt unknown /proc/60485/exe 2023-12-06T18:42:56.805 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/6 63002 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.806 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/6 63002 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.806 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/6 63002 root txt unknown /proc/63002/exe 2023-12-06T18:42:56.806 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/0 66159 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.806 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/0 66159 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.806 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/0 66159 root txt unknown /proc/66159/exe 2023-12-06T18:42:56.806 INFO:teuthology.orchestra.run.smithi152.stdout:kdmflush/ 69757 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.806 INFO:teuthology.orchestra.run.smithi152.stdout:kdmflush/ 69757 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.806 INFO:teuthology.orchestra.run.smithi152.stdout:kdmflush/ 69757 root txt unknown /proc/69757/exe 2023-12-06T18:42:56.806 INFO:teuthology.orchestra.run.smithi152.stdout:kdmflush/ 69876 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.806 INFO:teuthology.orchestra.run.smithi152.stdout:kdmflush/ 69876 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.806 INFO:teuthology.orchestra.run.smithi152.stdout:kdmflush/ 69876 root txt unknown /proc/69876/exe 2023-12-06T18:42:56.806 INFO:teuthology.orchestra.run.smithi152.stdout:kdmflush/ 69995 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.806 INFO:teuthology.orchestra.run.smithi152.stdout:kdmflush/ 69995 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.806 INFO:teuthology.orchestra.run.smithi152.stdout:kdmflush/ 69995 root txt unknown /proc/69995/exe 2023-12-06T18:42:56.806 INFO:teuthology.orchestra.run.smithi152.stdout:kdmflush/ 70114 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.807 INFO:teuthology.orchestra.run.smithi152.stdout:kdmflush/ 70114 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.807 INFO:teuthology.orchestra.run.smithi152.stdout:kdmflush/ 70114 root txt unknown /proc/70114/exe 2023-12-06T18:42:56.807 INFO:teuthology.orchestra.run.smithi152.stdout:kdmflush/ 70233 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.807 INFO:teuthology.orchestra.run.smithi152.stdout:kdmflush/ 70233 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.807 INFO:teuthology.orchestra.run.smithi152.stdout:kdmflush/ 70233 root txt unknown /proc/70233/exe 2023-12-06T18:42:56.807 INFO:teuthology.orchestra.run.smithi152.stdout:xfsalloc 71118 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.807 INFO:teuthology.orchestra.run.smithi152.stdout:xfsalloc 71118 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.807 INFO:teuthology.orchestra.run.smithi152.stdout:xfsalloc 71118 root txt unknown /proc/71118/exe 2023-12-06T18:42:56.807 INFO:teuthology.orchestra.run.smithi152.stdout:xfs_mru_c 71119 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.807 INFO:teuthology.orchestra.run.smithi152.stdout:xfs_mru_c 71119 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.807 INFO:teuthology.orchestra.run.smithi152.stdout:xfs_mru_c 71119 root txt unknown /proc/71119/exe 2023-12-06T18:42:56.807 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-buf/d 71120 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.807 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-buf/d 71120 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.807 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-buf/d 71120 root txt unknown /proc/71120/exe 2023-12-06T18:42:56.807 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-conv/ 71121 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.807 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-conv/ 71121 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.808 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-conv/ 71121 root txt unknown /proc/71121/exe 2023-12-06T18:42:56.808 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-recla 71122 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.808 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-recla 71122 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.808 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-recla 71122 root txt unknown /proc/71122/exe 2023-12-06T18:42:56.808 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-block 71123 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.808 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-block 71123 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.808 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-block 71123 root txt unknown /proc/71123/exe 2023-12-06T18:42:56.808 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-inode 71124 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.808 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-inode 71124 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.808 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-inode 71124 root txt unknown /proc/71124/exe 2023-12-06T18:42:56.808 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-log/d 71125 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.808 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-log/d 71125 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.808 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-log/d 71125 root txt unknown /proc/71125/exe 2023-12-06T18:42:56.808 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-cil/d 71126 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.808 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-cil/d 71126 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.809 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-cil/d 71126 root txt unknown /proc/71126/exe 2023-12-06T18:42:56.809 INFO:teuthology.orchestra.run.smithi152.stdout:xfsaild/d 71127 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.809 INFO:teuthology.orchestra.run.smithi152.stdout:xfsaild/d 71127 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.809 INFO:teuthology.orchestra.run.smithi152.stdout:xfsaild/d 71127 root txt unknown /proc/71127/exe 2023-12-06T18:42:56.809 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.809 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.809 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root txt REG 8,1 968800 13744 /usr/sbin/sshd 2023-12-06T18:42:56.809 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-12-06T18:42:56.809 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 19520 32283 /usr/lib64/security/pam_lastlog.so 2023-12-06T18:42:56.809 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 15184 32309 /usr/lib64/security/pam_umask.so 2023-12-06T18:42:56.809 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.809 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-12-06T18:42:56.809 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-12-06T18:42:56.809 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-12-06T18:42:56.809 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-12-06T18:42:56.809 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-12-06T18:42:56.810 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 15264 32290 /usr/lib64/security/pam_motd.so 2023-12-06T18:42:56.810 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-12-06T18:42:56.810 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 44152 32291 /usr/lib64/security/pam_namespace.so 2023-12-06T18:42:56.810 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 27616 32299 /usr/lib64/security/pam_selinux.so 2023-12-06T18:42:56.810 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-12-06T18:42:56.810 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-12-06T18:42:56.810 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 15240 32287 /usr/lib64/security/pam_loginuid.so 2023-12-06T18:42:56.810 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-12-06T18:42:56.810 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 15232 32292 /usr/lib64/security/pam_nologin.so 2023-12-06T18:42:56.810 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 19368 32301 /usr/lib64/security/pam_sepermit.so 2023-12-06T18:42:56.810 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-12-06T18:42:56.810 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-12-06T18:42:56.810 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-12-06T18:42:56.810 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.810 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.811 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.811 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.811 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.811 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.811 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.811 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.811 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.811 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.811 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.811 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.811 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.811 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-12-06T18:42:56.811 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.811 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.811 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.812 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.812 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.812 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.812 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.812 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.812 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.812 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.812 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-12-06T18:42:56.812 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.812 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-12-06T18:42:56.812 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.812 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root mem REG 0,19 0 19 /sys/fs/selinux/status 2023-12-06T18:42:56.812 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.812 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.812 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.813 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root 3u unix 0xffff8a574855d940 0t0 164074 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.813 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root 4u IPv4 160509 0t0 TCP smithi152.front.sepia.ceph.com:ssh->teuthology.front.sepia.ceph.com:56074 (ESTABLISHED) 2023-12-06T18:42:56.813 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root 6u unix 0xffff8a5748553300 0t0 164049 type=STREAM (CONNECTED) 2023-12-06T18:42:56.813 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root 7w FIFO 0,24 0t0 1635 /run/systemd/sessions/4.ref 2023-12-06T18:42:56.813 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75221 root 8u unix 0xffff8a5748550880 0t0 164078 type=STREAM (CONNECTED) 2023-12-06T18:42:56.813 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.813 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.813 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu txt REG 8,1 968800 13744 /usr/sbin/sshd 2023-12-06T18:42:56.813 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-12-06T18:42:56.813 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 19520 32283 /usr/lib64/security/pam_lastlog.so 2023-12-06T18:42:56.813 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 15184 32309 /usr/lib64/security/pam_umask.so 2023-12-06T18:42:56.814 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.814 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-12-06T18:42:56.814 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-12-06T18:42:56.814 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-12-06T18:42:56.814 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-12-06T18:42:56.814 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-12-06T18:42:56.814 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 15264 32290 /usr/lib64/security/pam_motd.so 2023-12-06T18:42:56.814 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-12-06T18:42:56.814 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 44152 32291 /usr/lib64/security/pam_namespace.so 2023-12-06T18:42:56.814 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 27616 32299 /usr/lib64/security/pam_selinux.so 2023-12-06T18:42:56.814 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-12-06T18:42:56.814 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-12-06T18:42:56.814 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 15240 32287 /usr/lib64/security/pam_loginuid.so 2023-12-06T18:42:56.814 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-12-06T18:42:56.814 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 15232 32292 /usr/lib64/security/pam_nologin.so 2023-12-06T18:42:56.815 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 19368 32301 /usr/lib64/security/pam_sepermit.so 2023-12-06T18:42:56.815 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-12-06T18:42:56.815 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-12-06T18:42:56.815 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-12-06T18:42:56.815 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 153600 9809 /usr/lib64/libgpg-error.so.0.32.0 2023-12-06T18:42:56.815 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.815 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.815 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 882376 9699 /usr/lib64/libzstd.so.1.5.1 2023-12-06T18:42:56.815 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 1288504 9883 /usr/lib64/libgcrypt.so.20.4.0 2023-12-06T18:42:56.815 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.815 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.815 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.815 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.815 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.815 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.816 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.816 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.816 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-12-06T18:42:56.816 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.816 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.816 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.816 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.816 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.816 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.816 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.816 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.816 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.816 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 915808 10122 /usr/lib64/libsystemd.so.0.35.0 2023-12-06T18:42:56.816 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-12-06T18:42:56.816 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.817 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-12-06T18:42:56.817 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.817 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2023-12-06T18:42:56.817 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu 0u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.817 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu 1u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.817 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.817 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu 3u unix 0xffff8a574855d940 0t0 164074 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.817 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu 4u IPv4 160509 0t0 TCP smithi152.front.sepia.ceph.com:ssh->teuthology.front.sepia.ceph.com:56074 (ESTABLISHED) 2023-12-06T18:42:56.817 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu 5u unix 0xffff8a5748552a80 0t0 164077 type=STREAM (CONNECTED) 2023-12-06T18:42:56.817 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu 6u unix 0xffff8a5748553300 0t0 164049 type=STREAM (CONNECTED) 2023-12-06T18:42:56.817 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu 7w FIFO 0,24 0t0 1635 /run/systemd/sessions/4.ref 2023-12-06T18:42:56.817 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu 11w FIFO 0,13 0t0 204573 pipe 2023-12-06T18:42:56.817 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu 13r FIFO 0,13 0t0 253923 pipe 2023-12-06T18:42:56.817 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu 15r FIFO 0,13 0t0 253924 pipe 2023-12-06T18:42:56.817 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu 21r FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:56.817 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu 22w FIFO 0,13 0t0 183522 pipe 2023-12-06T18:42:56.818 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu 23r FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:56.818 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu 24r FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:56.818 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu 25r FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:56.818 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu 26r FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.818 INFO:teuthology.orchestra.run.smithi152.stdout:sshd 75224 ubuntu 31r FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:56.818 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.818 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.818 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony txt REG 8,1 353408 12490 /usr/sbin/chronyd 2023-12-06T18:42:56.818 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony mem REG 8,1 626152 9989 /usr/lib64/libhogweed.so.6.5 2023-12-06T18:42:56.818 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.818 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony mem REG 8,1 340456 9991 /usr/lib64/libnettle.so.8.5 2023-12-06T18:42:56.818 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:56.818 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony mem REG 8,1 2291576 9994 /usr/lib64/libgnutls.so.30.33.1 2023-12-06T18:42:56.818 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.819 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony mem REG 8,1 94112 9901 /usr/lib64/libtasn1.so.6.6.0 2023-12-06T18:42:56.819 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:56.819 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-12-06T18:42:56.819 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony mem REG 8,1 127680 9972 /usr/lib64/libseccomp.so.2.5.2 2023-12-06T18:42:56.819 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.819 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.819 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.819 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.819 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony 1w CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.819 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony 2u CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.819 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony 3u unix 0xffff8a57473b61c0 0t0 159658 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.819 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony 5u IPv4 159663 0t0 UDP localhost:323 2023-12-06T18:42:56.819 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony 6u IPv6 159664 0t0 UDP localhost:323 2023-12-06T18:42:56.819 INFO:teuthology.orchestra.run.smithi152.stdout:chronyd 75284 chrony 7u unix 0xffff8a57473b1dc0 0t0 159665 /run/chrony/chronyd.sock type=DGRAM (UNCONNECTED) 2023-12-06T18:42:56.819 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.820 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.820 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-12-06T18:42:56.820 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.820 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.820 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-12-06T18:42:56.820 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.820 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.820 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.820 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.820 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 68000 38241 /usr/lib64/python3.9/lib-dynload/array.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.820 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 101208 41993 /usr/lib64/python3.9/lib-dynload/_socket.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.820 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.820 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.821 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.821 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.821 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.821 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-12-06T18:42:56.821 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.821 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.821 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.821 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.821 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.821 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph 1u unix 0xffff8a57484150c0 0t0 165229 type=STREAM (CONNECTED) 2023-12-06T18:42:56.821 INFO:teuthology.orchestra.run.smithi152.stdout:ceph-cras 75500 ceph 2u unix 0xffff8a57484150c0 0t0 165229 type=STREAM (CONNECTED) 2023-12-06T18:42:56.821 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/4 75519 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.821 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/4 75519 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.821 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/4 75519 root txt unknown /proc/75519/exe 2023-12-06T18:42:56.821 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/7 76912 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.821 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/7 76912 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.822 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/7 76912 root txt unknown /proc/76912/exe 2023-12-06T18:42:56.822 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/3 77847 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.822 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/3 77847 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.822 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/3 77847 root txt unknown /proc/77847/exe 2023-12-06T18:42:56.822 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/3 77848 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.822 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/3 77848 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.822 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/3 77848 root txt unknown /proc/77848/exe 2023-12-06T18:42:56.822 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/1 78519 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.822 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/1 78519 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.822 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/1 78519 root txt unknown /proc/78519/exe 2023-12-06T18:42:56.822 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/2 78768 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.822 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/2 78768 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.822 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/2 78768 root txt unknown /proc/78768/exe 2023-12-06T18:42:56.822 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/0 80308 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.822 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/0 80308 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.823 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/0 80308 root txt unknown /proc/80308/exe 2023-12-06T18:42:56.823 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 80431 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.823 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 80431 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.823 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 80431 root txt unknown /proc/80431/exe 2023-12-06T18:42:56.823 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-buf/d 81272 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.823 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-buf/d 81272 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.823 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-buf/d 81272 root txt unknown /proc/81272/exe 2023-12-06T18:42:56.823 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-conv/ 81273 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.823 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-conv/ 81273 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.823 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-conv/ 81273 root txt unknown /proc/81273/exe 2023-12-06T18:42:56.823 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-recla 81274 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.823 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-recla 81274 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.823 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-recla 81274 root txt unknown /proc/81274/exe 2023-12-06T18:42:56.823 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-block 81275 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.823 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-block 81275 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.823 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-block 81275 root txt unknown /proc/81275/exe 2023-12-06T18:42:56.824 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-inode 81276 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.824 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-inode 81276 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.824 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-inode 81276 root txt unknown /proc/81276/exe 2023-12-06T18:42:56.824 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-log/d 81277 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.824 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-log/d 81277 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.824 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-log/d 81277 root txt unknown /proc/81277/exe 2023-12-06T18:42:56.824 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-cil/d 81278 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.824 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-cil/d 81278 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.824 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-cil/d 81278 root txt unknown /proc/81278/exe 2023-12-06T18:42:56.824 INFO:teuthology.orchestra.run.smithi152.stdout:xfsaild/d 81279 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.824 INFO:teuthology.orchestra.run.smithi152.stdout:xfsaild/d 81279 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.824 INFO:teuthology.orchestra.run.smithi152.stdout:xfsaild/d 81279 root txt unknown /proc/81279/exe 2023-12-06T18:42:56.824 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-buf/d 81356 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.824 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-buf/d 81356 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.824 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-buf/d 81356 root txt unknown /proc/81356/exe 2023-12-06T18:42:56.825 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-conv/ 81357 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.825 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-conv/ 81357 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.825 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-conv/ 81357 root txt unknown /proc/81357/exe 2023-12-06T18:42:56.825 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-recla 81358 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.825 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-recla 81358 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.825 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-recla 81358 root txt unknown /proc/81358/exe 2023-12-06T18:42:56.825 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-block 81359 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.825 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-block 81359 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.825 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-block 81359 root txt unknown /proc/81359/exe 2023-12-06T18:42:56.825 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-inode 81360 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.825 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-inode 81360 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.825 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-inode 81360 root txt unknown /proc/81360/exe 2023-12-06T18:42:56.825 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-log/d 81361 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.825 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-log/d 81361 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.825 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-log/d 81361 root txt unknown /proc/81361/exe 2023-12-06T18:42:56.825 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-cil/d 81362 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.826 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-cil/d 81362 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.826 INFO:teuthology.orchestra.run.smithi152.stdout:xfs-cil/d 81362 root txt unknown /proc/81362/exe 2023-12-06T18:42:56.826 INFO:teuthology.orchestra.run.smithi152.stdout:xfsaild/d 81363 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.826 INFO:teuthology.orchestra.run.smithi152.stdout:xfsaild/d 81363 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.826 INFO:teuthology.orchestra.run.smithi152.stdout:xfsaild/d 81363 root txt unknown /proc/81363/exe 2023-12-06T18:42:56.826 INFO:teuthology.orchestra.run.smithi152.stdout:dio/dm-2 83757 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.826 INFO:teuthology.orchestra.run.smithi152.stdout:dio/dm-2 83757 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.826 INFO:teuthology.orchestra.run.smithi152.stdout:dio/dm-2 83757 root txt unknown /proc/83757/exe 2023-12-06T18:42:56.826 INFO:teuthology.orchestra.run.smithi152.stdout:dio/dm-3 84930 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:56.826 INFO:teuthology.orchestra.run.smithi152.stdout:dio/dm-3 84930 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.826 INFO:teuthology.orchestra.run.smithi152.stdout:dio/dm-3 84930 root txt unknown /proc/84930/exe 2023-12-06T18:42:56.826 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:56.826 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.826 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root txt REG 8,1 185296 11759 /usr/bin/sudo 2023-12-06T18:42:56.826 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-12-06T18:42:56.827 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.829 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.830 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-12-06T18:42:56.830 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.830 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-12-06T18:42:56.830 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-12-06T18:42:56.830 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-12-06T18:42:56.830 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-12-06T18:42:56.830 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-12-06T18:42:56.830 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-12-06T18:42:56.830 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-12-06T18:42:56.830 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-12-06T18:42:56.830 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-12-06T18:42:56.830 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-12-06T18:42:56.830 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.830 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.831 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.831 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.831 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.831 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.831 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.831 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-12-06T18:42:56.831 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-12-06T18:42:56.831 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.831 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.831 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:56.831 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:56.831 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:56.831 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 472568 54513 /usr/libexec/sudo/sudoers.so 2023-12-06T18:42:56.831 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.831 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.832 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.832 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-12-06T18:42:56.832 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.832 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:56.832 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-12-06T18:42:56.832 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-12-06T18:42:56.832 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.832 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-12-06T18:42:56.832 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-12-06T18:42:56.832 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-12-06T18:42:56.832 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-12-06T18:42:56.832 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-12-06T18:42:56.832 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-12-06T18:42:56.832 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-12-06T18:42:56.832 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-12-06T18:42:56.832 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-12-06T18:42:56.833 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.833 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.833 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 119832 54509 /usr/libexec/sudo/libsudo_util.so.0.0.0 2023-12-06T18:42:56.833 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.833 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.833 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.833 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.833 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-12-06T18:42:56.833 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root 0r FIFO 0,13 0t0 188461 pipe 2023-12-06T18:42:56.833 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:56.833 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:56.833 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root 3r FIFO 0,13 0t0 182697 pipe 2023-12-06T18:42:56.833 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root 4w FIFO 0,13 0t0 182697 pipe 2023-12-06T18:42:56.833 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root 5u netlink 0t0 182708 AUDIT 2023-12-06T18:42:56.833 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87040 root 6u unix 0xffff8a574855bfc0 0t0 182709 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.834 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:56.834 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.834 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root txt REG 8,1 185296 11759 /usr/bin/sudo 2023-12-06T18:42:56.834 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-12-06T18:42:56.834 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:56.834 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.834 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-12-06T18:42:56.834 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.834 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-12-06T18:42:56.834 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-12-06T18:42:56.834 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.834 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.834 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-12-06T18:42:56.834 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-12-06T18:42:56.834 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-12-06T18:42:56.835 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-12-06T18:42:56.835 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-12-06T18:42:56.835 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-12-06T18:42:56.835 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-12-06T18:42:56.835 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-12-06T18:42:56.835 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-12-06T18:42:56.835 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.835 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.835 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.835 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.835 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.835 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.835 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.835 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.835 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:56.836 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:56.836 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.836 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-12-06T18:42:56.836 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.836 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:56.836 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:56.836 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-12-06T18:42:56.836 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-12-06T18:42:56.836 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-12-06T18:42:56.836 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 472568 54513 /usr/libexec/sudo/sudoers.so 2023-12-06T18:42:56.836 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.836 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-12-06T18:42:56.836 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-12-06T18:42:56.836 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-12-06T18:42:56.836 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-12-06T18:42:56.836 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-12-06T18:42:56.837 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-12-06T18:42:56.837 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-12-06T18:42:56.837 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.837 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.837 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-12-06T18:42:56.837 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-12-06T18:42:56.837 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:56.837 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 119832 54509 /usr/libexec/sudo/libsudo_util.so.0.0.0 2023-12-06T18:42:56.837 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.837 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:56.837 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.837 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.837 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-12-06T18:42:56.837 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root 0r FIFO 0,13 0t0 183522 pipe 2023-12-06T18:42:56.837 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:56.838 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.838 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root 3r FIFO 0,13 0t0 186457 pipe 2023-12-06T18:42:56.838 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root 4w FIFO 0,13 0t0 186457 pipe 2023-12-06T18:42:56.838 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root 5u netlink 0t0 186464 AUDIT 2023-12-06T18:42:56.838 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 87047 root 6u unix 0xffff8a57473b2a80 0t0 186465 type=DGRAM (CONNECTED) 2023-12-06T18:42:56.838 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:56.838 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.838 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-12-06T18:42:56.838 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.838 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-12-06T18:42:56.838 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.838 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.838 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.838 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.838 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.838 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.839 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.839 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.839 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.839 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.839 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.839 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.839 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-12-06T18:42:56.839 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.839 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.839 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.839 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.839 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root 0r FIFO 0,13 0t0 188461 pipe 2023-12-06T18:42:56.839 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:56.839 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87106 root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:56.839 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:56.840 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.840 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-12-06T18:42:56.840 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:56.840 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-12-06T18:42:56.840 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.840 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.840 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.840 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.840 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.840 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.840 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.840 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.840 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:56.840 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-12-06T18:42:56.840 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.841 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.841 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.841 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:56.841 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.841 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:56.841 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.841 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root 0r FIFO 0,13 0t0 183522 pipe 2023-12-06T18:42:56.841 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:56.841 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 87109 root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:56.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:56.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:56.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:56.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:56.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:56.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:56.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:56.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:56.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:56.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:56.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:56.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:56.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:56.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:56.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:56.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:56.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:56.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:56.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:56.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:56.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:56.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:56.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:56.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:56.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:56.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:56.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:56.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:56.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:56.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:56.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:56.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:56.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:56.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:56.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:56.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:56.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:56.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:56.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:56.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:56.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:56.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:56.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:56.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:56.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:56.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:56.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:56.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:56.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:56.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:56.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:56.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:56.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:56.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:56.866 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:56.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:56.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:56.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:56.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:56.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:56.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:56.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:56.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:56.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:56.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:56.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:56.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:56.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:56.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:56.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:56.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:56.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:56.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:56.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:56.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:56.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:56.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:56.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:56.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:56.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:56.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:56.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:56.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:56.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:56.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:56.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:56.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:56.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:56.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:56.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:56.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:56.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:56.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:56.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:56.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:56.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:56.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:56.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:56.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:56.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:56.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:56.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:56.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:56.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:56.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:56.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:56.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:56.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:56.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:56.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:56.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:56.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:56.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:56.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:56.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:56.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:56.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:56.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:56.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:56.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:56.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:56.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:56.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:56.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:56.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:56.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:56.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:56.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:56.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:56.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:56.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:56.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:56.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:56.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:56.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:56.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:56.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:56.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:56.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:56.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:56.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:56.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:56.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:56.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:56.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:56.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:56.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:56.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:56.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:56.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:56.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:56.882 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:56.882 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.882 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.882 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.882 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.882 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87153 log root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:56.882 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:56.882 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.882 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:56.882 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.882 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.882 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:56.882 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:56.882 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:56.882 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.883 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:56.883 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:56.883 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:56.883 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.883 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.883 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:56.883 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:56.883 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.883 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.883 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.883 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.883 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:56.883 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:56.884 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.884 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.884 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.884 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.884 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:56.884 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:56.884 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:56.884 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:56.884 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:56.884 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.884 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.884 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.884 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.884 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.884 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:56.885 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:56.885 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:56.885 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:56.885 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:56.885 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:56.885 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.885 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.885 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:56.885 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:56.885 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:56.885 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.885 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.885 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.885 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:56.885 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:56.885 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:56.886 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:56.886 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:56.886 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:56.886 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:56.886 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:56.886 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:56.886 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:56.886 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:56.886 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:56.886 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:56.886 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:56.886 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:56.886 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:56.886 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:56.886 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:56.887 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:56.887 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:56.887 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:56.887 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:56.887 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:56.887 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:56.887 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:56.887 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:56.887 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:56.887 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:56.887 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:56.887 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:56.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:56.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:56.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:56.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:56.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:56.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:56.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:56.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:56.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:56.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:56.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:56.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:56.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:56.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:56.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:56.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:56.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:56.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:56.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:56.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:56.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:56.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:56.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:56.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87160 msgr-work root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:56.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:56.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:56.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:56.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:56.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:56.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:56.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:56.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:56.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:56.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:56.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:56.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:56.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:56.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:56.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:56.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:56.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:56.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:56.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:56.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:56.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:56.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:56.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:56.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:56.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:56.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:56.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:56.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:56.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:56.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:56.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:56.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:56.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:56.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:56.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:56.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:56.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:56.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:56.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:56.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:56.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:56.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:56.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:56.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:56.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:56.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:56.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:56.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:56.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:56.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:56.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:56.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:56.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:56.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:56.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:56.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:56.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:56.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:56.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:56.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:56.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:56.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:56.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:56.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:56.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:56.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:56.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:56.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:56.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:56.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:56.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:56.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:56.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:56.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:56.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:56.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:56.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:56.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:56.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:56.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87162 msgr-work root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:56.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:56.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:56.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:56.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:56.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:56.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:56.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:56.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:56.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:56.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:56.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:56.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:56.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:56.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:56.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:56.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:56.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:56.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:56.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:56.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:56.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:56.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:56.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:56.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:56.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:56.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:56.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:56.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:56.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:56.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:56.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:56.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:56.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:56.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:56.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:56.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:56.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:56.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:56.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:56.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:56.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:56.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:56.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:56.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:56.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:56.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:56.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:56.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:56.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:56.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:56.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:56.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:56.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:56.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:56.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:56.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:56.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:56.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:56.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:56.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:56.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:56.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:56.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:56.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:56.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:56.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:56.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:56.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:56.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:56.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:56.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:56.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:56.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:56.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:56.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:56.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:56.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:56.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:56.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:56.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87164 msgr-work root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:56.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:56.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:56.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:56.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:56.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:56.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:56.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:56.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:56.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:56.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:56.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:56.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:56.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:56.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:56.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:56.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:56.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:56.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:56.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:56.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:56.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:56.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:56.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:56.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:56.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:56.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:56.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:56.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:56.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:56.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:56.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:56.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:56.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:56.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:56.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:56.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:56.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:56.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:56.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:56.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:56.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:56.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:56.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:56.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:56.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:56.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:56.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:56.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:56.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:56.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:56.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:56.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:56.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:56.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:56.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:56.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:56.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:56.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:56.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:56.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:56.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:56.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:56.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:56.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:56.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87182 service root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:56.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:56.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:56.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:56.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:56.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:56.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:56.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:56.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:56.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:56.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:56.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:56.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:56.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:56.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:56.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:56.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:56.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:56.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:56.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:56.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:56.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:56.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:56.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:56.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:56.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:56.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:56.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:56.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:56.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:56.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:56.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:56.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:56.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:56.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:56.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:56.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:56.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:56.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:56.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:56.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:56.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:56.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:56.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:56.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:56.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:56.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:56.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:56.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:56.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:56.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:56.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:56.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:56.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:56.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:56.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:56.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:56.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:56.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:56.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:56.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:56.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:56.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:56.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:56.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:56.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:56.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:56.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:56.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:56.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:56.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:56.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:56.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:56.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:56.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:56.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:56.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:56.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:56.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:56.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:56.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87185 admin_soc root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:56.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:56.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:56.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:56.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:56.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:56.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:56.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:56.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:56.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:56.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:56.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:56.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:56.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:56.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:56.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:56.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:56.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:56.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:56.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:56.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:56.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:56.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:56.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:56.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:56.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:56.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:56.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:56.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:56.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:56.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:56.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:56.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:56.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:56.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:56.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:56.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:56.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:56.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:56.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:56.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:56.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:56.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:56.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:56.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:56.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:56.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:56.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:56.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:56.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:56.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:56.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:56.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:56.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:56.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:56.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:56.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:56.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:56.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:56.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:56.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:56.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:56.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:56.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:56.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:56.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:56.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:56.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:56.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:56.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:56.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:56.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:56.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87205 signal_ha root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:56.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:56.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:56.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:56.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:56.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:56.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:56.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:56.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:56.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:56.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:56.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:56.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:56.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:56.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:56.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:56.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:56.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:56.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:56.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:56.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:56.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:56.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:56.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:56.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:56.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:56.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:56.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:56.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:56.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:56.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:56.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:56.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:56.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:56.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:56.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:56.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:56.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:56.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:56.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:56.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:56.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:56.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:56.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:56.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:56.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:56.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:56.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:56.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:56.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:56.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:56.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:56.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:56.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:56.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:56.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:56.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:56.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:56.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:56.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:56.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:56.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:56.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:56.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:56.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:56.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:56.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:56.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:56.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:56.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:56.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:56.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:56.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:56.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:56.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:56.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:56.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:56.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:56.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:56.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:56.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87208 io_contex root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:56.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:56.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:56.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:56.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:56.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:56.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:56.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:56.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:56.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:56.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:56.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:56.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:56.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:56.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:56.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:56.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:56.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:56.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:56.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:56.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:56.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:56.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:56.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:56.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:56.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:56.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:56.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:56.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:56.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:56.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:56.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:56.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:56.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:56.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:56.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:56.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:56.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:56.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:56.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:56.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:56.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:56.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:56.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:56.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:56.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:56.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:56.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:56.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:56.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:56.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:56.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:56.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:56.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:56.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:56.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:56.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:56.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:56.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:56.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:56.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:56.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:56.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:56.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:56.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:56.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:56.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:56.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:56.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:56.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:56.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:56.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:56.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:56.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:56.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:56.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:56.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:56.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:56.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:56.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:56.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87209 io_contex root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:56.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:56.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:56.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:56.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:56.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:56.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:56.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:56.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:56.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:56.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:56.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:56.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:56.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:56.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:56.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:56.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:56.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:56.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:56.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:56.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:56.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:56.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:56.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:56.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:56.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:56.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:56.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:56.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:56.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:56.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:56.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:56.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:56.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:56.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:56.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:56.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:56.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:56.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:56.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:56.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:56.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:56.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:56.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:56.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:56.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:56.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:56.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:56.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:56.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:56.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:56.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:56.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:56.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:56.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:56.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:56.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:56.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:56.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:56.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:56.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:56.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:56.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:56.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:56.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:56.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:56.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:56.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:56.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:56.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:56.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:56.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:56.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:56.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:56.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:56.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:56.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:56.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:56.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:56.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:56.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87227 OpHistory root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:56.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:56.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:56.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:56.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:56.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:56.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:56.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:56.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:56.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:56.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:56.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:56.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:56.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:56.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:56.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:56.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:56.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:56.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:56.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:56.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:56.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:56.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:56.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:56.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:56.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:56.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:56.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:56.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:56.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:56.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:56.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:56.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:56.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:56.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:56.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:56.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:56.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:56.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:56.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:56.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:56.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:56.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:56.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:56.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:56.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:56.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:56.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:56.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:56.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:56.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:56.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:56.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:56.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:56.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:56.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:56.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:56.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:56.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:56.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:56.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:56.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:56.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:56.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:56.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:56.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:56.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:56.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:56.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:56.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:56.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:56.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:56.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:56.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:56.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:56.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:56.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:56.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:56.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:56.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:56.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87229 ceph_time root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:56.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:56.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:56.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:56.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:56.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:56.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:56.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:56.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:56.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:56.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:56.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:56.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:56.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:56.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:56.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:56.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:56.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:56.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:56.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:56.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:56.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:56.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:56.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:56.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:56.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:56.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:56.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:56.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:56.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:56.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:56.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:56.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:56.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:56.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:56.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:56.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:56.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:56.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:56.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:56.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:56.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:56.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:56.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:56.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:56.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:56.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:56.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:56.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:56.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:56.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:56.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:56.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:56.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:56.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:56.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:56.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:56.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:56.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:56.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:56.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:56.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:56.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:56.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:56.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:56.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:56.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:56.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:56.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:56.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:56.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:56.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:56.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:56.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:56.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:56.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:56.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:56.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:56.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:56.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:56.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:56.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:56.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:56.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:56.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:56.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:56.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:56.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:56.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:56.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:56.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:56.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:56.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:56.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:56.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:56.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:56.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:56.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:56.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:56.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:56.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:56.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:56.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:56.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:56.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87267 memcheck- root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:56.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:56.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:56.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:56.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:56.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:56.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:56.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:56.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:56.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:56.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:56.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:56.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:56.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:56.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:56.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:56.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:56.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:56.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:56.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:56.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:56.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:56.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:56.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:56.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:56.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:56.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:56.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:56.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:56.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:56.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:56.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:56.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87279 memcheck- root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87292 memcheck- root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87307 memcheck- root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87330 memcheck- root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87338 memcheck- root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87339 safe_time root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87340 safe_time root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87341 safe_time root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87342 safe_time root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87343 fn_anonym root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87362 rocksdb:l root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87363 rocksdb:l root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87364 rocksdb:l root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 87366 rocksdb:h root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88836 bstore_ai root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 88837 bstore_di root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89530 bstore_ai root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 89532 bstore_di root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90280 memcheck- root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90282 cfin root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90283 bstore_kv root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90284 bstore_kv root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90285 bstore_me root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90300 ms_dispat root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90301 ms_local root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.259 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.259 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.259 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.259 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.259 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.259 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.259 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.259 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.259 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.259 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.259 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.259 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.259 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.259 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.260 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.260 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.260 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.260 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.260 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.260 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.260 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.260 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.260 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.260 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.260 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.260 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.260 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.260 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.260 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.260 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90303 safe_time root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90306 safe_time root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90308 ms_dispat root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90310 ms_local root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90312 ms_dispat root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90314 ms_local root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90316 ms_dispat root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90318 ms_local root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90320 ms_dispat root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90322 ms_local root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90324 ms_dispat root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90326 ms_local root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90328 ms_dispat root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90331 ms_local root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90334 fn_anonym root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90338 finisher root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90341 safe_time root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.440 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90344 safe_time root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90347 memcheck- root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.463 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.463 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.463 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.463 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.463 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.463 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.463 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.463 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.463 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.463 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.463 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.463 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.463 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.463 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.463 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.463 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.464 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.464 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.464 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.464 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.464 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90350 osd_srv_a root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.464 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.464 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.464 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.464 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.464 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.464 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.464 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.464 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.464 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.464 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.464 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90353 tp_osd_tp root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90356 tp_osd_tp root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90360 tp_osd_tp root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90362 tp_osd_tp root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90365 tp_osd_tp root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90370 tp_osd_tp root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90373 tp_osd_tp root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90376 tp_osd_tp root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90379 tp_osd_tp root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90383 tp_osd_tp root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90387 tp_osd_tp root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90393 tp_osd_tp root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90397 tp_osd_tp root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90401 tp_osd_tp root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90405 tp_osd_tp root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90407 tp_osd_tp root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root mem REG 8,1 48 3359 /tmp/vgdb-pipe-shared-mem-vgdb-87114-by-root-on-??? 2023-12-06T18:42:57.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root DEL REG 0,17 183637 /[aio] 2023-12-06T18:42:57.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root DEL REG 0,17 184555 /[aio] 2023-12-06T18:42:57.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 1w FIFO 0,13 0t0 183523 pipe 2023-12-06T18:42:57.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 2w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 3w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 4r FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 5w FIFO 0,13 0t0 185591 pipe 2023-12-06T18:42:57.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,63] 2023-12-06T18:42:57.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 7r FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 8w FIFO 0,13 0t0 185598 pipe 2023-12-06T18:42:57.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 9u a_inode 0,14 0 13316 [eventpoll:10,61,79] 2023-12-06T18:42:57.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 10r FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 11w FIFO 0,13 0t0 185599 pipe 2023-12-06T18:42:57.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 12u a_inode 0,14 0 13316 [eventpoll:13,60,80] 2023-12-06T18:42:57.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 13r FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 14w FIFO 0,13 0t0 185600 pipe 2023-12-06T18:42:57.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 15w REG 8,1 13456671 267043 /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:57.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 16uW REG 0,24 6 1995 /run/ceph/ceph-osd.6.pid 2023-12-06T18:42:57.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 17r FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 18w FIFO 0,13 0t0 185608 pipe 2023-12-06T18:42:57.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 19u unix 0xffff8a5747ce7b40 0t0 185609 /var/run/ceph/ceph-osd.6.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 20u IPv4 178173 0t0 TCP *:6808 (LISTEN) 2023-12-06T18:42:57.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 21u IPv4 189449 0t0 TCP *:6811 (LISTEN) 2023-12-06T18:42:57.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 22u IPv4 189463 0t0 TCP *:6813 (LISTEN) 2023-12-06T18:42:57.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 23u IPv4 189479 0t0 TCP *:6815 (LISTEN) 2023-12-06T18:42:57.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 24r FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 25w FIFO 0,13 0t0 183616 pipe 2023-12-06T18:42:57.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 26r FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 27w FIFO 0,13 0t0 183617 pipe 2023-12-06T18:42:57.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 28u sock 0,8 0t0 187553 protocol: UDPv6 2023-12-06T18:42:57.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 29r DIR 253,2 357 64 /var/lib/ceph/osd/ceph-6 2023-12-06T18:42:57.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 30uW REG 253,2 37 69 /var/lib/ceph/osd/ceph-6/fsid 2023-12-06T18:42:57.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 31u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 32u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 33u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 34u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 35u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 36u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 37u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 38u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 39u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 40u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 41u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 42u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 43u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 44u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 45u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 46u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 47u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 48u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 49u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 50u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 51u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 52u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 53u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 54u REG 253,2 96636764160 70 /var/lib/ceph/osd/ceph-6/block 2023-12-06T18:42:57.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 55r FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 56w FIFO 0,13 0t0 184563 pipe 2023-12-06T18:42:57.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 58r FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 59w FIFO 0,13 0t0 184564 pipe 2023-12-06T18:42:57.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 60u IPv4 259654 0t0 TCP smithi152.front.sepia.ceph.com:6813->smithi152.front.sepia.ceph.com:60502 (ESTABLISHED) 2023-12-06T18:42:57.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 61u IPv4 259652 0t0 TCP smithi152.front.sepia.ceph.com:6815->smithi152.front.sepia.ceph.com:34772 (ESTABLISHED) 2023-12-06T18:42:57.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 63u IPv4 226023 0t0 TCP smithi152.front.sepia.ceph.com:39970->smithi152.front.sepia.ceph.com:6807 (ESTABLISHED) 2023-12-06T18:42:57.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 79u IPv4 231640 0t0 TCP smithi152.front.sepia.ceph.com:44768->smithi152.front.sepia.ceph.com:6806 (ESTABLISHED) 2023-12-06T18:42:57.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root 80u IPv4 211517 0t0 TCP smithi152.front.sepia.ceph.com:6811->smithi152.front.sepia.ceph.com:42056 (ESTABLISHED) 2023-12-06T18:42:57.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root *525u REG 8,1 32 3347 /tmp/valgrind_proc_87114_cmdline_153a139a (deleted) 2023-12-06T18:42:57.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root *526u REG 8,1 336 3348 /tmp/valgrind_proc_87114_auxv_153a139a (deleted) 2023-12-06T18:42:57.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root *527w FIFO 0,13 0t0 183524 pipe 2023-12-06T18:42:57.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root *528w REG 8,1 1234 655633 /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:57.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root *529r FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root *530w FIFO 0,13 0t0 185590 pipe 2023-12-06T18:42:57.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87114 90411 osd_srv_h root *531r FIFO 8,1 0t0 3361 /tmp/vgdb-pipe-from-vgdb-to-87114-by-root-on-??? 2023-12-06T18:42:57.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87154 log root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87161 msgr-work root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.663 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.663 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.663 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.663 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.663 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.663 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.663 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.663 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.663 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.663 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.663 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.663 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.663 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.663 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.663 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.663 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.664 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.664 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.664 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.664 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.664 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.664 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87163 msgr-work root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.664 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.664 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.664 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.664 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.664 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.664 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.664 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.664 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.664 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.664 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.665 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.665 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.665 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.665 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.665 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.665 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.665 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.665 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.665 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.665 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.665 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.665 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.665 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.665 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.666 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.666 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.666 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.666 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.666 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.666 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.666 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.666 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.666 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.666 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.666 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.666 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.666 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.666 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.667 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.667 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.667 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.667 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.667 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.667 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.667 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.667 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.667 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.667 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.667 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.667 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.667 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.667 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.667 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.668 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.668 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.668 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.668 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.668 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.668 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.668 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.668 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.668 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.668 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.668 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.668 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.668 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.668 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.668 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.668 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.669 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.669 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.669 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.669 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.669 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.669 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.669 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.669 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.669 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.669 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.669 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.669 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.669 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.670 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.670 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.670 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.670 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.670 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.670 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.670 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.670 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.670 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.670 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.670 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.670 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.670 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.670 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.671 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.671 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.671 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.671 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.671 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.671 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87165 msgr-work root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.671 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.671 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.671 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.671 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.671 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.671 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.671 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.671 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.671 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.672 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.672 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.672 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.672 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.672 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.672 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.672 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.672 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.672 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.672 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.672 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.672 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.672 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.672 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.672 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.673 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.673 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.673 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.673 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.673 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.673 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.673 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.673 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.673 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.673 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.673 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.673 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.673 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.673 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.673 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.674 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.674 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.674 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.674 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.674 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.674 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.674 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.674 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.674 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.674 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.674 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.674 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.674 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.674 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.674 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.674 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.675 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.675 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.675 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.675 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.675 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.675 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.675 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.675 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.675 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.675 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.675 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.675 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.675 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.675 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.675 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.675 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.676 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.676 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.676 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.676 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.676 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.676 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.676 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.676 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.676 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.676 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.676 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.676 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.676 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.676 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.676 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.676 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.677 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.677 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.677 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.677 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.677 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.677 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.677 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.677 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.677 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.677 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.677 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.677 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.677 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.677 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.677 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87184 service root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.678 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.678 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.678 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.678 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.678 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.678 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.678 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.678 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.678 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.678 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.678 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.678 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.678 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.678 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.678 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.678 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.679 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.679 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.679 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.679 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.679 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.679 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.679 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.679 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.679 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.679 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.679 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.679 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.679 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.679 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.679 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.679 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.680 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.680 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.680 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.680 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.680 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.680 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.680 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.680 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.680 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.680 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.680 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.680 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.680 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.680 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.680 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.680 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.681 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.681 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.681 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.681 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.681 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.681 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.681 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.681 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.681 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.681 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.681 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.681 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.681 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.681 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.681 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.682 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.682 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.682 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.682 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.682 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.682 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.682 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.682 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.682 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.682 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.682 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.682 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.682 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.682 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.682 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.683 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.683 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.683 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.683 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.683 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.683 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.683 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.683 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.683 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.683 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.683 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.683 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.683 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.683 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.683 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.683 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.684 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.684 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.684 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.684 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.684 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.684 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.684 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.684 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87186 admin_soc root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.684 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.684 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.684 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.684 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.684 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.684 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.684 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.685 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.685 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.685 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.685 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.685 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.685 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.685 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.685 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.685 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.685 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.685 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.685 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.685 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.685 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.685 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.686 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.686 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.686 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.686 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.686 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.686 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.686 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.686 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.686 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.686 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.686 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.686 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.686 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.686 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.686 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.686 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.687 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.687 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.687 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.687 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.687 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.687 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.687 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.687 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.687 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.687 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.687 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.687 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.687 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.687 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.687 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.687 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.688 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.688 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.688 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.688 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.688 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.688 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.688 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.688 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.688 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.688 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.688 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.688 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.688 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.688 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.688 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.688 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.689 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.689 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.689 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.689 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.689 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.689 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.689 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.689 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.689 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.689 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.689 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.689 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.689 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.689 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.690 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.690 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.690 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.690 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.690 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.690 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.690 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.690 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.690 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.690 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.690 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.690 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.690 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.690 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.691 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.691 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.691 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.691 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87204 signal_ha root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.691 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.691 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.691 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.691 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.691 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.691 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.691 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.691 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.691 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.691 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.691 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.692 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.692 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.692 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.692 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.692 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.692 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.692 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.692 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.692 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.692 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.692 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.692 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.692 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.692 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.692 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.692 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.693 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.693 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.693 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.693 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.693 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.693 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.693 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.693 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.693 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.693 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.693 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.693 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.693 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.693 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.693 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.694 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.694 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.694 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.694 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.694 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.694 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.694 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.694 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.694 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.694 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.694 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.694 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.694 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.694 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.694 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.695 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.695 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.695 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.695 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.695 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.695 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.695 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.695 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.695 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.695 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.695 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.695 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.695 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.695 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.695 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.695 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.696 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.696 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.696 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.696 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.696 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.696 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.696 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.696 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.696 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.696 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.696 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.696 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.696 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.696 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.696 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.696 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.697 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.697 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.697 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.697 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.697 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.697 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.697 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.697 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.697 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.697 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.697 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.697 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.697 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87206 io_contex root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.697 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.697 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.698 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.698 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.698 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.698 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.698 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.698 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.698 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.698 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.698 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.698 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.698 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.698 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.698 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.698 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.698 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.699 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.700 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.700 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.700 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.700 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.700 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.700 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.700 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.700 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.700 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.700 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.700 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.700 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.700 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.700 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.700 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.701 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.701 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.701 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.701 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.701 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.701 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.701 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.701 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.701 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.701 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.701 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.701 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.701 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.701 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.701 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.702 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.702 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.702 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.702 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.702 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.702 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.702 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.702 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.702 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.702 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.702 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.702 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.702 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.702 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.702 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.703 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.703 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.703 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.703 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.703 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.703 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.703 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.703 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.703 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.703 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.703 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.703 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.703 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.703 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.703 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.703 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.704 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.704 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.704 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.704 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.704 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.704 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.704 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87207 io_contex root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.704 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.704 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.704 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.704 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.704 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.704 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.704 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.704 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.705 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.705 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.705 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.705 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.705 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.705 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.705 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.705 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.705 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.705 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.705 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.705 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.705 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.705 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.706 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.706 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.706 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.706 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.706 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.706 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.706 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.706 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.706 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.706 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.706 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.706 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.706 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.706 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.706 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.707 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.707 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.707 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.707 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.707 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.707 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.707 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.707 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.707 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.707 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.707 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.707 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.707 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.707 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.707 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.708 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.708 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.708 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.708 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.708 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.708 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.708 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.708 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.708 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.708 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.708 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.708 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.708 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.708 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.708 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.708 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.709 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.709 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.709 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.709 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.709 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.709 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.709 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.709 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.709 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.709 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.709 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.709 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.709 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.710 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.710 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.710 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.710 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.710 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.710 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.710 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.710 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.710 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.710 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.710 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.710 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.710 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.710 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.710 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.711 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.711 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.711 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.711 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.711 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.711 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87228 ceph_time root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.711 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.711 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.711 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.711 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.711 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.711 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.711 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.711 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.711 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.711 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.712 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.712 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.712 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.712 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.712 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.712 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.712 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.712 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.712 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.712 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.712 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.712 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.712 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.712 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.712 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.712 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.713 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.713 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.713 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.713 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.713 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.713 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.713 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.713 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.713 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.713 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.713 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.713 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.714 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.714 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.714 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.714 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.714 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.714 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.714 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.714 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.714 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.714 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.714 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.714 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.714 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.714 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.715 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.715 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.715 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.715 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.715 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.715 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.715 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.715 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.715 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.715 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.715 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.715 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.715 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.715 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.715 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.715 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.716 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.716 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.716 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.716 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.716 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.716 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.716 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.716 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.716 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.716 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.716 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.716 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.716 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.716 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.716 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.717 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.717 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.717 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.717 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.717 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.717 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.717 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.717 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.717 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.717 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.717 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.717 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.717 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.717 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.717 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.718 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.718 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.718 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.718 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87272 memcheck- root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.718 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.718 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.718 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.718 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.718 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.718 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.718 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.718 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.718 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.718 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.718 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.719 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.719 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.719 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.719 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.719 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.719 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.719 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.719 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.719 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.719 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.719 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.719 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.719 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.719 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.719 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.719 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.720 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.720 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.720 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.720 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.720 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.720 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.720 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.720 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.720 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.720 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.720 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.720 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.720 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.720 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.720 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.720 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.721 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.721 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.721 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.721 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.721 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.721 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.721 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.721 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.721 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.721 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.721 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.721 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.721 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.721 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.721 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.722 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.722 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.722 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.722 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.722 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.722 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.722 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.722 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.722 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.722 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.722 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.722 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.722 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.722 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.722 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.723 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.723 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.723 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.723 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.723 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.723 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.723 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.723 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.723 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.723 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.723 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.723 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.723 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.723 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.723 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.724 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.724 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.724 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.724 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.724 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.724 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.724 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.724 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.724 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.724 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.724 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.724 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.724 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.724 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87284 memcheck- root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.724 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.724 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.725 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.725 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.725 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.725 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.725 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.725 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.725 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.725 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.725 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.725 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.725 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.725 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.725 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.725 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.725 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.726 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.726 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.726 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.726 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.726 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.726 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.726 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.726 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.726 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.726 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.726 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.726 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.726 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.726 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.727 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.727 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.727 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.727 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.727 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.727 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.727 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.727 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.727 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.727 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.727 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.727 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.727 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.727 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.727 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.727 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.728 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.728 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.728 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.728 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.728 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.728 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.728 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.728 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.728 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.728 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.728 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.728 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.728 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.728 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.728 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.729 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.729 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.729 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.729 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.729 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.729 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.729 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.729 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.729 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.729 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.729 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.729 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.729 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.729 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.729 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.729 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.730 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.730 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.730 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.730 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.730 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.730 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.730 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.730 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.730 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.730 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.730 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.730 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.730 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.731 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.731 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.731 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.731 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.731 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.731 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.731 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.731 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.731 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.731 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.731 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87298 memcheck- root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.731 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.731 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.731 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.731 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.731 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.732 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.732 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.732 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.732 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.732 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.732 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.732 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.732 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.732 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.732 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.732 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.732 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.732 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.732 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.732 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.733 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.733 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.733 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.733 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.733 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.733 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.733 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.733 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.733 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.733 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.733 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.733 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.733 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.733 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.733 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.733 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.734 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.734 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.734 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.734 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.734 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.734 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.734 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.734 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.734 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.734 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.734 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.734 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.734 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.734 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.735 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.735 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.735 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.735 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.735 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.735 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.735 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.735 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.735 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.735 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.735 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.735 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.735 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.735 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.735 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.736 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.736 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.736 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.736 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.736 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.736 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.736 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.736 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.736 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.736 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.736 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.736 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.736 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.736 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.736 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.737 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.737 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.737 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.737 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.737 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.737 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.737 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.737 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.737 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.737 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.737 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.737 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.737 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.737 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.737 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.737 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.738 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.738 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.738 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.738 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.738 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.738 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87313 memcheck- root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.738 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.738 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.738 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.738 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.738 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.738 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.738 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.738 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.738 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.739 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.739 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.739 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.739 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.739 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.739 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.739 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.739 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.739 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.739 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.739 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.739 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.739 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.739 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.739 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.739 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.740 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.740 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.740 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.740 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.740 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.740 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.740 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.740 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.740 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.740 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.740 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.740 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.740 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.740 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.740 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.741 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.741 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.741 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.741 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.741 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.741 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.741 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.741 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.741 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.741 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.741 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.741 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.741 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.741 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.741 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.741 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.742 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.742 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.742 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.742 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.742 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.742 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.742 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.742 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.742 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.742 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.742 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.742 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.742 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.742 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.743 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.743 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.743 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.743 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.743 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.743 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.743 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.743 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.743 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.743 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.743 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.743 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.743 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.743 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.743 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.743 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.743 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.744 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.744 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.744 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.744 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.744 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.744 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.744 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.744 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.744 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.744 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.744 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.744 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.744 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.744 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.744 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87318 memcheck- root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.744 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.745 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.745 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.745 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.745 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.745 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.745 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.745 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.745 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.745 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.745 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.745 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.745 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.745 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.745 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.745 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.746 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.746 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.746 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.746 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.746 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.746 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.746 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.746 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.746 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.746 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.746 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.746 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.746 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.747 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.747 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.747 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.747 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.747 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.747 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.747 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.747 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.747 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.747 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.747 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.747 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.747 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.747 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.747 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.747 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.748 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.748 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.748 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.748 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.748 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.748 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.748 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.748 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.748 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.748 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.748 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.748 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.748 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.748 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.748 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.749 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.749 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.749 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.749 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.749 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.749 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.749 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.749 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.749 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.749 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.749 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.749 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.749 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.749 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.749 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.750 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.750 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.750 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.750 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.750 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.750 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.750 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.750 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.750 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.750 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.750 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.750 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.750 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.750 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.750 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.751 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.751 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.751 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.751 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.751 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.751 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.751 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.751 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.751 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.751 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.751 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.751 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87353 rocksdb:l root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.751 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.752 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.752 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.752 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.752 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.752 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.752 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.752 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.752 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.752 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.752 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.752 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.752 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.752 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.752 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.752 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.753 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.753 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.753 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.753 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.753 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.753 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.753 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.753 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.753 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.753 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.753 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.753 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.753 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.753 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.753 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.753 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.754 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.754 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.754 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.754 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.754 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.754 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.754 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.754 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.754 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.754 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.754 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.754 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.754 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.754 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.755 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.755 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.755 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.755 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.755 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.755 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.755 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.755 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.755 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.755 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.755 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.755 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.755 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.755 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.755 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.756 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.756 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.756 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.756 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.756 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.756 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.756 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.756 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.756 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.756 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.756 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.756 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.756 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.756 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.756 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.756 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.756 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.757 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.757 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.757 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.757 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.757 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.757 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.757 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.757 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.757 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.757 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.757 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.757 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.757 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.757 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.757 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.758 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.758 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.758 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.758 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.758 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.758 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.758 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.758 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.758 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87355 rocksdb:l root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.758 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.758 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.758 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.758 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.758 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.758 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.759 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.759 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.759 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.759 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.759 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.759 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.759 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.759 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.759 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.759 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.759 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.759 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.759 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.759 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.759 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.759 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.759 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.760 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.760 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.760 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.760 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.760 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.760 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.760 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.760 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.760 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.760 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.760 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.760 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.760 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.760 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.761 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.761 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.761 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.761 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.761 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.761 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.761 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.761 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.761 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.761 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.761 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.761 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.761 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.761 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.761 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.761 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.762 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.762 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.762 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.762 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.762 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.762 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.762 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.762 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.762 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.762 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.762 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.762 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.762 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.762 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.762 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.763 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.763 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.763 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.763 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.763 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.763 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.763 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.763 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.763 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.763 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.763 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.763 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.763 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.763 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.763 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.763 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.763 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.764 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.764 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.764 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.764 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.764 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.764 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.764 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.764 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.764 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.764 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.764 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.764 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.764 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.764 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.764 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.764 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.765 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87357 rocksdb:l root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.765 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.765 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.765 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.765 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.765 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.765 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.765 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.765 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.765 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.765 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.765 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.765 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.765 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.765 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.766 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.766 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.766 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.766 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.766 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.766 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.766 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.766 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.766 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.766 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.766 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.766 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.766 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.766 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.766 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.767 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.767 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.767 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.767 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.767 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.767 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.767 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.767 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.767 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.767 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.767 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.767 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.767 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.767 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.767 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.767 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.768 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.768 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.768 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.768 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.768 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.768 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.768 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.768 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.768 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.768 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.768 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.768 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.768 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.768 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.768 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.768 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.769 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.769 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.769 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.769 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.769 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.769 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.769 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.769 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.769 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.769 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.769 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.769 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.769 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.769 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.769 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.769 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.770 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.770 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.770 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.770 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.770 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.770 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.770 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.770 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.770 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.770 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.770 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.770 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.770 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.770 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.770 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.771 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.771 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.771 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.771 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.771 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.771 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.771 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.771 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.771 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.771 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 87359 rocksdb:h root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.771 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.771 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.771 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.771 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.771 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.772 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.772 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.772 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.772 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.772 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.772 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.772 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.772 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.772 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.772 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.772 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.772 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.772 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.772 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.772 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.772 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.773 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.773 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.773 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.773 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.773 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.773 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.773 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.773 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.773 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.773 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.773 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.773 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.773 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.773 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.773 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.774 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.774 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.774 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.774 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.774 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.774 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.774 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.774 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.774 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.774 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.774 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.774 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.774 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.774 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.774 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.775 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.775 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.775 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.775 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.775 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.775 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.775 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.775 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.775 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.775 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.775 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.775 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.775 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.775 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.775 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.776 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.776 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.776 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.776 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.776 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.776 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.776 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.776 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.776 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.776 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.776 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.776 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.776 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.776 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.776 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.776 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.777 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.777 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.777 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.777 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.777 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.777 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.777 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.777 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.777 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.777 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.777 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.777 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.777 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.777 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.777 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.778 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.778 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.778 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.778 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.778 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90380 ms_dispat root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.778 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.778 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.778 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.778 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.778 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.778 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.778 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.778 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.778 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.778 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.779 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.779 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.779 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.779 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.779 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.779 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.779 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.779 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.779 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.779 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.779 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.779 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.779 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.779 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.779 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.780 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.780 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.780 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.780 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.780 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.780 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.780 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.780 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.780 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.780 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.780 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.780 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.780 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.780 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.780 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.781 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.781 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.781 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.781 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.781 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.781 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.781 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.781 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.781 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.781 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.781 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.781 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.781 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.781 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.781 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.781 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.782 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.782 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.782 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.782 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.782 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.782 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.782 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.782 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.782 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.782 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.782 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.782 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.782 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.782 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.782 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.783 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.783 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.783 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.783 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.783 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.783 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.783 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.783 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.783 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.783 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.783 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.783 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.783 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.783 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.783 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.783 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.784 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.784 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.784 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.784 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.784 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.784 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.784 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.784 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.784 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.784 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.784 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.784 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.784 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.784 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.784 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90384 ms_local root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.784 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.785 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.785 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.785 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.785 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.785 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.785 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.785 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.785 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.785 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.785 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.785 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.785 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.785 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.785 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.785 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.785 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.786 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.786 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.786 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.786 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.786 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.786 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.786 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.786 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.786 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.786 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.786 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.786 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.786 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.786 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.786 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.787 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.787 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.787 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.787 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.787 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.787 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.787 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.787 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.787 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.787 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.787 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.787 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.787 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.787 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.787 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.788 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.788 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.788 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.788 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.788 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.788 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.788 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.788 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.788 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.788 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.788 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.788 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.788 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.788 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.788 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.788 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.789 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.789 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.789 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.789 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.789 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.789 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.789 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.789 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.789 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.789 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.789 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.789 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.789 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.789 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.789 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.790 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.790 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.790 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.790 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.790 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.790 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.790 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.790 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.790 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.790 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.790 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.790 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.790 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.790 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.791 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.791 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.791 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.791 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.791 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.791 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.791 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.791 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.791 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.791 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90390 ms_dispat root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.791 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.791 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.791 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.791 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.791 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.791 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.792 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.792 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.792 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.792 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.792 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.792 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.792 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.792 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.792 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.792 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.792 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.792 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.792 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.792 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.792 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.792 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.793 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.793 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.793 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.793 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.793 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.793 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.793 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.793 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.793 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.793 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.793 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.793 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.793 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.793 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.793 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.794 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.794 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.794 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.794 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.794 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.794 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.794 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.794 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.794 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.794 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.794 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.794 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.794 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.794 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.795 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.795 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.795 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.795 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.795 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.795 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.795 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.795 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.795 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.795 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.795 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.795 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.795 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.795 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.795 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.795 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.796 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.796 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.796 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.796 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.796 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.796 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.796 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.796 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.796 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.796 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.796 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.796 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.796 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.796 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.796 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.796 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.797 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.797 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.797 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.797 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.797 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.797 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.797 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.797 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.797 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.797 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.797 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.797 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.797 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.797 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.797 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.798 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.798 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.798 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.798 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90394 ms_local root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.798 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.798 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.798 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.798 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.798 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.798 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.798 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.798 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.798 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.798 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.798 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.798 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.799 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.799 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.799 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.799 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.799 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.799 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.799 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.799 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.799 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.799 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.799 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.799 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.799 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.799 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.799 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.799 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.799 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.800 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.800 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.800 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.800 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.800 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.800 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.800 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.800 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.800 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.800 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.800 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.800 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.800 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.800 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.800 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.800 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.801 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.801 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.801 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.801 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.801 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.801 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.801 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.801 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.801 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.801 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.801 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.801 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.801 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.801 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.801 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.802 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.802 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.802 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.802 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.802 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.802 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.802 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.802 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.802 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.802 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.802 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.802 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.802 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.802 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.802 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.803 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.803 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.803 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.803 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.803 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.803 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.803 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.803 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.803 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.803 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.803 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.803 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.803 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.803 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.803 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.804 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.804 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.804 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.804 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.804 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.804 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.804 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.804 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.804 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.804 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.804 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.804 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90399 ms_dispat root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.804 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.804 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.804 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.804 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.805 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.805 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.805 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.805 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.805 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.805 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.805 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.805 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.805 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.805 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.805 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.805 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.805 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.805 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.805 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.805 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.806 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.806 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.806 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.806 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.806 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.831 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.831 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.832 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.832 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.832 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.832 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.832 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.832 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.832 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.832 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.832 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.832 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.832 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.832 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.832 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.832 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.832 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.833 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.833 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.833 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.833 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.833 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.833 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.833 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.833 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.833 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.833 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.833 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.833 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.833 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.833 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.833 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.833 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.834 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.834 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.834 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.834 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.834 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.834 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.834 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.834 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.834 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.834 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.834 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.834 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.834 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.834 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.834 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.835 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.835 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.835 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.835 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.835 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.835 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.835 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.835 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.835 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.835 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.835 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.835 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.835 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.835 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.835 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.836 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.836 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.836 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.836 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.836 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.836 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.836 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.836 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.836 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.836 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.836 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.836 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.836 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.837 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90403 ms_local root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.837 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.837 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.837 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.837 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.837 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.837 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.837 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.837 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.837 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.837 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.837 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.837 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.837 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.838 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.838 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.838 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.838 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.838 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.838 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.838 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.838 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.838 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.838 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.838 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.838 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.838 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.838 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.838 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.839 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.839 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.839 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.839 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.839 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.839 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.839 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.839 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.839 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.839 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.839 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.839 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.839 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.839 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.839 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.840 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.840 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.840 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.840 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.840 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.840 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.840 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.840 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.840 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.840 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.840 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.840 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.840 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.841 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.842 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.843 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90408 ms_dispat root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.844 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.845 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.846 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.847 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.847 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.847 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.847 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.847 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.847 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.847 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.847 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.847 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.847 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.847 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.847 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.847 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.847 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.847 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.848 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.848 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.848 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.848 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.848 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.848 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.848 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.848 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.848 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.848 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.848 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.848 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.848 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.848 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.849 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.849 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.849 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.849 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.849 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.849 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.849 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.849 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.849 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.849 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.849 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.849 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.849 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.849 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.849 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.849 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.850 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.850 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.850 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.850 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.850 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.850 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.850 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.850 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.853 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.853 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.853 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.853 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.853 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.853 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.853 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.854 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.854 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90413 ms_local root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.854 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.854 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.854 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.854 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.854 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.854 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.854 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.854 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.854 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.854 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.854 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.854 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.854 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.855 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.855 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.855 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.855 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.855 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.855 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.855 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.855 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.855 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.855 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.855 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.855 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.855 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.855 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.856 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.856 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.856 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.856 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.856 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.856 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.856 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.856 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.856 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.856 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.856 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.856 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.856 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.856 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.857 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.857 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.857 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.857 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.857 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.857 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.857 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.857 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.857 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.857 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.857 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.857 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.857 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.857 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.857 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.857 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.858 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.858 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.858 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.858 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.858 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.858 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.858 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.858 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.858 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.858 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.858 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.858 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.858 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.858 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.858 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.859 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.859 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.859 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.859 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.859 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.859 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.859 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.859 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.859 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.859 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.859 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.859 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.859 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.859 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.859 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.859 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.860 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.860 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.860 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.860 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.860 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.860 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.860 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.860 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.860 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.860 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.860 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.860 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.860 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.861 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90415 ms_dispat root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.861 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.861 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.861 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.861 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.861 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.861 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.861 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.861 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.861 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.861 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.861 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.861 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.861 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.861 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.861 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.862 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.862 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.862 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.862 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.862 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.862 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.862 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.862 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.862 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.862 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.862 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.862 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.862 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.862 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.863 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.863 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.863 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.863 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.863 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.863 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.863 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.863 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.863 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.863 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.863 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.863 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.863 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.863 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.863 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.863 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.864 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.864 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.864 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.864 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.864 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.864 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.864 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.864 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.864 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.864 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.864 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.864 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.864 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.865 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.865 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.865 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.865 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.865 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.865 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.865 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.865 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.865 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.865 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.865 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.865 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.865 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.865 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.865 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.865 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.866 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.866 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.866 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.866 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.866 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.866 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.866 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.866 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.866 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.866 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.866 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.866 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.866 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.866 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.866 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90417 ms_local root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.867 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.868 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.869 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.870 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.871 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.872 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.873 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90419 ms_dispat root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.874 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.875 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.876 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.877 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root mem REG 8,1 48 3362 /tmp/vgdb-pipe-shared-mem-vgdb-87116-by-root-on-??? 2023-12-06T18:42:57.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 1w FIFO 0,13 0t0 188462 pipe 2023-12-06T18:42:57.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 2w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 3w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 4r FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 5w FIFO 0,13 0t0 187499 pipe 2023-12-06T18:42:57.878 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,22,23,60] 2023-12-06T18:42:57.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 7r FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 8w FIFO 0,13 0t0 187502 pipe 2023-12-06T18:42:57.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,71] 2023-12-06T18:42:57.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 10r FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 11w FIFO 0,13 0t0 187503 pipe 2023-12-06T18:42:57.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,56] 2023-12-06T18:42:57.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 13r FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 14w FIFO 0,13 0t0 187504 pipe 2023-12-06T18:42:57.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 15w REG 8,1 15729540 267042 /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:57.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 16uW REG 0,24 6 1996 /run/ceph/ceph-osd.5.pid 2023-12-06T18:42:57.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 17r FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 18w FIFO 0,13 0t0 187510 pipe 2023-12-06T18:42:57.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 19u unix 0xffff8a57582c4840 0t0 187511 /var/run/ceph/ceph-osd.5.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 22u IPv4 186525 0t0 TCP *:6812 (LISTEN) 2023-12-06T18:42:57.879 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 23u IPv4 186540 0t0 TCP *:6814 (LISTEN) 2023-12-06T18:42:57.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 24r FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 25w FIFO 0,13 0t0 184521 pipe 2023-12-06T18:42:57.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 26r FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 27w FIFO 0,13 0t0 184522 pipe 2023-12-06T18:42:57.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 28u sock 0,8 0t0 184543 protocol: UDPv6 2023-12-06T18:42:57.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 55r FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 56u IPv4 255566 0t0 TCP smithi152.front.sepia.ceph.com:6812->smithi152.front.sepia.ceph.com:49462 (CLOSE_WAIT) 2023-12-06T18:42:57.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 57w FIFO 0,13 0t0 189481 pipe 2023-12-06T18:42:57.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 58r FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 59w FIFO 0,13 0t0 189482 pipe 2023-12-06T18:42:57.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 60u IPv4 255564 0t0 TCP smithi152.front.sepia.ceph.com:6814->smithi152.front.sepia.ceph.com:42234 (CLOSE_WAIT) 2023-12-06T18:42:57.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root 71u IPv4 214231 0t0 TCP smithi152.front.sepia.ceph.com:6810->smithi152.front.sepia.ceph.com:38416 (CLOSE_WAIT) 2023-12-06T18:42:57.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.880 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root *525u REG 8,1 32 3351 /tmp/valgrind_proc_87116_cmdline_9b944453 (deleted) 2023-12-06T18:42:57.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root *526u REG 8,1 336 3352 /tmp/valgrind_proc_87116_auxv_9b944453 (deleted) 2023-12-06T18:42:57.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root *527w FIFO 0,13 0t0 188463 pipe 2023-12-06T18:42:57.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root *528w REG 8,1 1234 655635 /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:57.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root *529r FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root *530w FIFO 0,13 0t0 187498 pipe 2023-12-06T18:42:57.881 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 87116 90421 ms_local root *531r FIFO 8,1 0t0 3364 /tmp/vgdb-pipe-from-vgdb-to-87116-by-root-on-??? 2023-12-06T18:42:57.881 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/1 87264 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:57.881 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/1 87264 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.881 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/1 87264 root txt unknown /proc/87264/exe 2023-12-06T18:42:57.881 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/2 87291 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:57.881 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/2 87291 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.881 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/2 87291 root txt unknown /proc/87291/exe 2023-12-06T18:42:57.881 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 90476 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:57.881 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 90476 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.881 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 90476 root txt unknown /proc/90476/exe 2023-12-06T18:42:57.881 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/7 98985 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:57.882 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/7 98985 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.882 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/7 98985 root txt unknown /proc/98985/exe 2023-12-06T18:42:57.882 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.882 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.882 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root txt REG 8,1 185296 11759 /usr/bin/sudo 2023-12-06T18:42:57.882 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-12-06T18:42:57.882 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:57.882 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.882 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-12-06T18:42:57.882 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.882 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-12-06T18:42:57.882 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.882 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.882 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-12-06T18:42:57.882 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-12-06T18:42:57.883 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-12-06T18:42:57.883 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-12-06T18:42:57.883 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-12-06T18:42:57.883 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-12-06T18:42:57.883 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-12-06T18:42:57.883 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-12-06T18:42:57.883 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-12-06T18:42:57.883 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-12-06T18:42:57.883 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.883 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.883 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.883 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.883 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.883 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.883 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.884 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.884 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.884 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.884 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-12-06T18:42:57.884 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.884 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.884 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.884 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-12-06T18:42:57.884 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.884 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-12-06T18:42:57.884 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-12-06T18:42:57.884 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 472568 54513 /usr/libexec/sudo/sudoers.so 2023-12-06T18:42:57.884 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:57.884 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-12-06T18:42:57.885 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-12-06T18:42:57.885 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-12-06T18:42:57.885 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.885 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.885 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-12-06T18:42:57.885 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-12-06T18:42:57.885 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-12-06T18:42:57.885 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-12-06T18:42:57.885 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-12-06T18:42:57.885 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-12-06T18:42:57.885 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:57.885 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 119832 54509 /usr/libexec/sudo/libsudo_util.so.0.0.0 2023-12-06T18:42:57.885 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.885 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:57.885 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:57.886 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.886 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-12-06T18:42:57.886 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root 0r FIFO 0,13 0t0 204573 pipe 2023-12-06T18:42:57.886 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:57.886 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.886 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root 3r FIFO 0,13 0t0 208948 pipe 2023-12-06T18:42:57.886 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root 4w FIFO 0,13 0t0 208948 pipe 2023-12-06T18:42:57.886 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root 5u netlink 0t0 208953 AUDIT 2023-12-06T18:42:57.886 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 99407 root 6u unix 0xffff8a575813c400 0t0 208954 type=DGRAM (CONNECTED) 2023-12-06T18:42:57.886 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.886 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.886 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root txt REG 8,1 15448 10498 /usr/bin/python3.9 2023-12-06T18:42:57.886 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 178728 9694 /usr/lib64/liblzma.so.5.2.5 2023-12-06T18:42:57.886 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 76792 9701 /usr/lib64/libbz2.so.1.0.8 2023-12-06T18:42:57.886 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.887 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 67528 42002 /usr/lib64/python3.9/lib-dynload/math.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:57.887 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 23800 38230 /usr/lib64/python3.9/lib-dynload/_posixsubprocess.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:57.887 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 20000 42001 /usr/lib64/python3.9/lib-dynload/grp.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:57.887 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 56424 38237 /usr/lib64/python3.9/lib-dynload/_struct.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:57.887 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.887 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 37592 38251 /usr/lib64/python3.9/lib-dynload/select.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:57.887 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 23832 38245 /usr/lib64/python3.9/lib-dynload/fcntl.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:57.887 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 28056 41985 /usr/lib64/python3.9/lib-dynload/_heapq.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:57.887 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.887 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 3486240 13688 /usr/lib64/libpython3.9.so.1.0 2023-12-06T18:42:57.887 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 46560 38224 /usr/lib64/python3.9/lib-dynload/_lzma.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:57.887 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 29224 41980 /usr/lib64/python3.9/lib-dynload/_bz2.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:57.887 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 41752 38253 /usr/lib64/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so 2023-12-06T18:42:57.887 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:57.887 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:57.888 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.888 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root 0r FIFO 0,13 0t0 204573 pipe 2023-12-06T18:42:57.888 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:57.888 INFO:teuthology.orchestra.run.smithi152.stdout:daemon-he 99441 root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.888 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.889 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.890 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.891 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.892 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.893 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,56,62,89] 2023-12-06T18:42:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.894 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 15w REG 8,1 12073159 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:57.895 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.896 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 56 unknown /proc/99450/fd/56 (readlink: No such file or directory) 2023-12-06T18:42:57.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.897 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:57.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:57.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:57.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:57.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:57.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:57.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:57.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:57.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.898 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.899 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.900 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.901 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.902 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.903 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:57.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:57.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:57.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:57.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:57.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.904 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:57.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:57.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:57.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:57.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:57.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:57.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:57.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:57.905 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:57.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:57.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:57.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.906 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.907 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:57.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:57.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:57.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:57.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:57.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:57.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:57.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.908 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99767 log root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:57.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.909 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.910 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.911 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.912 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.913 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:57.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:57.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:57.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:57.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.914 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:57.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:57.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:57.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:57.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:57.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.915 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:57.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:57.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:57.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:57.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:57.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:57.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:57.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.916 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.917 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:57.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:57.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:57.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:57.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:57.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.918 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:57.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:57.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99769 msgr-work root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:57.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.919 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.920 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.921 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.922 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.923 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:57.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:57.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:57.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:57.924 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:57.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:57.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:57.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:57.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:57.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.925 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:57.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:57.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:57.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:57.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:57.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:57.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:57.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.926 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.927 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:57.928 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:57.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:57.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:57.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99770 msgr-work root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:57.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.929 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.930 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.931 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.932 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.933 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:57.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:57.934 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:57.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:57.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:57.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:57.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:57.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.935 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:57.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:57.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:57.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:57.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:57.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:57.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:57.936 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:57.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:57.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.937 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.938 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:57.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:57.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:57.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:57.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:57.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:57.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:57.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99771 msgr-work root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:57.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.939 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.940 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.941 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.942 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.943 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.944 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:57.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:57.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:57.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:57.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:57.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:57.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.945 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:57.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:57.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:57.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:57.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:57.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:57.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:57.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.946 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:57.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:57.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:57.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.947 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.948 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:57.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:57.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:57.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:57.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:57.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:57.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:57.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99795 service root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:57.949 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.950 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.951 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.952 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.953 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.954 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:57.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:57.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:57.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:57.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:57.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.955 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:57.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:57.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:57.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:57.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:57.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:57.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:57.956 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:57.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:57.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:57.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:57.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.957 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.958 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:57.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:57.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:57.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:57.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:57.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:57.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:57.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.959 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99796 admin_soc root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:57.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.960 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.961 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.962 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.963 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.964 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:57.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:57.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:57.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:57.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.965 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:57.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:57.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:57.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:57.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:57.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.966 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:57.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:57.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:57.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:57.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:57.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:57.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:57.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.967 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.968 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:57.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:57.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:57.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:57.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:57.969 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:57.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:57.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99901 signal_ha root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:57.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.970 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.971 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.972 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.973 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.974 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:57.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:57.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:57.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.975 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:57.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:57.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:57.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:57.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.976 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:57.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:57.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:57.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:57.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:57.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:57.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:57.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:57.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:57.977 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.978 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:57.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:57.979 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:57.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:57.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:57.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:57.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:57.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99902 io_contex root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:57.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.980 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.981 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.982 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.983 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.984 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.985 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:57.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:57.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:57.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:57.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:57.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:57.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.986 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:57.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:57.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:57.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:57.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:57.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:57.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:57.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.987 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:57.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:57.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:57.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.988 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.989 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:57.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:57.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:57.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:57.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:57.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:57.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:57.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:57.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:57.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.990 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:57.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99903 io_contex root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:57.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:57.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:57.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:57.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:57.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:57.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:57.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:57.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:57.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:57.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:57.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:57.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:57.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:57.991 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:57.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:57.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:57.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:57.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:57.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:57.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:57.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:57.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:57.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:57.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:57.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:57.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:57.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:57.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:57.992 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:57.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:57.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:57.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:57.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:57.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:57.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:57.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:57.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:57.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:57.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:57.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:57.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:57.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:57.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:57.993 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:57.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:57.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:57.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:57.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:57.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:57.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:57.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:57.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:57.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:57.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:57.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:57.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:57.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:57.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:57.994 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:57.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:57.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:57.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:57.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:57.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:57.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:57.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:57.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:57.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:57.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:57.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:57.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:57.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:57.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:57.995 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:57.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:57.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:57.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:57.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:57.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:57.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:57.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:57.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:57.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:57.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:57.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:57.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:57.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.996 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:57.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:57.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:57.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:57.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:57.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:57.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:57.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:57.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:57.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:57.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:57.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:57.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:57.997 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:57.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:57.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:57.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:57.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:57.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:57.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:57.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.998 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:57.999 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.000 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99966 OpHistory root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.001 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.002 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.003 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.004 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.005 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.006 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.007 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.008 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.009 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.010 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99967 ceph_time root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.011 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.012 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.013 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.014 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.015 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.016 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.017 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.018 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.019 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.020 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.021 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99976 memcheck- root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.022 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.023 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.024 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.025 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.026 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.027 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.028 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.029 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.030 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.031 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 99999 memcheck- root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.032 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.033 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.034 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.035 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.035 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.035 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.035 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.035 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.035 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.035 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.035 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.035 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.035 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.035 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.035 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.035 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.035 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.035 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.035 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.036 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.036 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.036 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.036 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.036 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.036 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.036 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.036 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.036 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.036 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.036 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.036 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.036 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.036 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.036 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.037 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.038 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.039 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.040 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.041 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.042 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100002 memcheck- root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.043 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.044 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.045 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.046 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.047 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.048 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.049 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.050 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.051 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.052 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100011 memcheck- root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.053 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.054 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.055 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.056 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.057 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.058 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.059 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.060 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.061 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.062 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.063 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.064 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100048 memcheck- root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.065 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.066 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.067 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.068 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.069 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.070 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.071 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.072 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.073 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.074 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100069 memcheck- root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.075 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.076 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.077 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.078 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.079 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.080 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.081 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.082 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.083 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.084 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100070 safe_time root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.085 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.086 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.087 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.088 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.089 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.090 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.091 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.092 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.093 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.094 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100071 safe_time root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.095 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.096 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.097 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.098 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.099 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.100 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.101 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.102 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.103 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.104 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100072 safe_time root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.105 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.106 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.107 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.108 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.109 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.110 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.111 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.112 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.113 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.114 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.115 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100073 safe_time root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.116 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.117 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.118 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.119 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.120 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.121 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.122 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.123 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.124 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.125 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100074 fn_anonym root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.126 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.127 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.128 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.129 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.130 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.131 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.132 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.133 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.134 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.135 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100159 rocksdb:l root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.136 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.137 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.138 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.139 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.140 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.141 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.142 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.143 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.144 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.145 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.146 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100160 rocksdb:l root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.147 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.148 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.149 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.150 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.151 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.152 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.153 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.154 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.155 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.156 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100178 rocksdb:l root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.157 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.158 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.159 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.160 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.161 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.162 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.163 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.164 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.165 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.166 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100182 rocksdb:h root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.167 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.168 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.169 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.170 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.171 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.172 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.173 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.174 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.175 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.176 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100946 bstore_ai root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.177 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.178 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.179 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.180 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.181 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.182 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.183 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.184 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.185 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.186 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 100947 bstore_di root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.187 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.188 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.189 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.190 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.191 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.192 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.193 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.194 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.195 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.196 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101330 bstore_ai root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.197 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.198 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.199 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.200 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.201 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.202 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.203 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.204 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.205 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.206 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101331 bstore_di root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.207 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.208 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.209 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.210 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.211 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.212 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.213 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.214 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.215 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.216 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.217 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101712 memcheck- root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.218 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.220 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.221 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.222 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.223 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.224 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.225 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.226 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.227 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101713 cfin root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.228 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.229 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.230 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.231 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.232 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.233 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.234 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.235 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.236 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.237 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.238 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101714 bstore_kv root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.239 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.240 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.241 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.242 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.243 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.244 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.245 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.246 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.247 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.248 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101715 bstore_kv root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.249 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.250 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.251 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.252 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.253 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.254 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.255 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.256 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.257 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.258 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.261 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101716 bstore_me root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.262 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.263 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.264 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.265 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.266 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.267 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.268 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.269 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.270 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.271 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.272 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101897 ms_dispat root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.273 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.274 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.275 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.276 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.277 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.278 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.279 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.280 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.281 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.282 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101914 ms_local root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.283 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.284 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.285 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.286 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.287 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.288 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.289 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.290 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.291 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.292 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101919 safe_time root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.293 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.294 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.295 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.296 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.297 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.298 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.299 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.300 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.301 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.302 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101920 safe_time root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.303 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.304 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.305 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.306 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.307 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.308 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.309 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.310 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.311 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.312 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.313 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101921 ms_dispat root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.314 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.315 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.316 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.317 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.318 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.319 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.320 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.321 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.322 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.323 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101922 ms_local root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.324 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.325 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.326 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.327 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.328 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.329 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.330 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.331 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.332 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.333 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101923 ms_dispat root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.334 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.335 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.336 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.337 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.338 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.339 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.340 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.341 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.342 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.343 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101924 ms_local root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.344 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.345 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.346 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.347 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.348 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.349 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.350 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.351 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.352 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.353 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.354 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101925 ms_dispat root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.355 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.356 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.357 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.358 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.359 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.360 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.361 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.362 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.363 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.364 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101926 ms_local root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.365 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.366 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.367 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.368 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.369 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.370 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.371 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.372 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.373 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.374 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101927 ms_dispat root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.375 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.376 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.377 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.378 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.379 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.380 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.381 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.382 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.383 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.384 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101928 ms_local root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.385 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.386 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.387 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.388 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.389 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.390 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.391 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.392 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.393 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.394 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101929 ms_dispat root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.395 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.396 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.397 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.398 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.400 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.401 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.402 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.403 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.404 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101930 ms_local root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.405 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.406 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.407 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.408 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.409 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.410 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.411 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.412 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.413 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.414 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.415 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101931 ms_dispat root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.416 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.417 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.418 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.419 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.420 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.421 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.422 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.423 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.424 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.425 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101932 ms_local root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.426 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.427 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.428 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.429 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.430 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.431 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.432 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.433 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.434 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.435 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101933 fn_anonym root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.436 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.437 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.438 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.439 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.440 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.440 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.440 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.440 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.440 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.440 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.440 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.440 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.440 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.440 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.440 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.440 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.440 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.440 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.440 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.441 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.442 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.443 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.444 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.445 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101934 finisher root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.446 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.447 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.448 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.449 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.450 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.451 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.452 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.453 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.454 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.455 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101935 safe_time root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.456 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.457 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.458 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.459 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.460 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.461 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.462 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.463 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.463 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.465 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.466 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.467 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.468 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101936 safe_time root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.469 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.470 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.471 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.472 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.473 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.474 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.475 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.476 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.477 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.479 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101937 memcheck- root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.480 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.481 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.482 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.483 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.484 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.485 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.486 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 15w REG 8,1 12070640 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.487 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.488 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.489 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101938 osd_srv_a root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.490 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.491 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.492 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.493 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.494 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.495 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.496 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 15w REG 8,1 12070967 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.497 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.498 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.499 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.500 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101939 tp_osd_tp root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.501 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.502 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.503 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.504 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.505 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.506 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 15w REG 8,1 12072552 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.507 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.508 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.509 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.510 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101940 tp_osd_tp root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.511 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.512 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.513 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.514 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.515 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.516 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 15w REG 8,1 12072552 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.517 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.518 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.519 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.520 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101941 tp_osd_tp root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.521 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.522 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.523 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.524 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.525 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.526 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 15w REG 8,1 12072552 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.527 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.528 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.529 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.530 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101942 tp_osd_tp root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.531 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.532 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.533 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.534 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.535 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.536 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 15w REG 8,1 12072552 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.537 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.538 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.539 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.540 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101943 tp_osd_tp root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.541 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.542 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.543 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.544 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.545 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.546 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.547 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 15w REG 8,1 12072552 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.548 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.549 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.550 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101944 tp_osd_tp root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.551 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.552 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.553 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.554 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.555 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.556 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.557 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 15w REG 8,1 12072552 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.558 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.559 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.560 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101945 tp_osd_tp root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.561 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.562 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.563 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.564 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.565 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.567 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 15w REG 8,1 12072552 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.568 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.569 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.570 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.571 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101946 tp_osd_tp root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.572 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.573 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.574 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.575 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.576 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.577 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 15w REG 8,1 12072552 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.578 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.579 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.580 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.581 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101947 tp_osd_tp root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.582 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.583 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.584 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.585 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.586 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.587 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 15w REG 8,1 12072552 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.588 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.589 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.590 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.591 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101948 tp_osd_tp root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.592 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.593 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.594 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.595 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.596 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.597 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 15w REG 8,1 12072552 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.598 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.599 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.600 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.601 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101949 tp_osd_tp root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.602 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.603 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.604 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.605 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.606 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.607 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 15w REG 8,1 12072552 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.608 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.609 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.610 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.611 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101950 tp_osd_tp root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.612 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.613 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.614 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.615 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.616 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.617 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 15w REG 8,1 12072552 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.618 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.619 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.620 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.621 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101951 tp_osd_tp root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.622 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.623 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.624 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.625 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.626 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.627 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 15w REG 8,1 12072552 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.628 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.629 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.630 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.631 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101952 tp_osd_tp root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.632 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.633 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.634 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.635 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.636 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.637 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 15w REG 8,1 12072552 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.638 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.639 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.640 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.641 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101953 tp_osd_tp root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.642 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.643 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.644 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.644 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.644 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.644 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.644 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.644 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.644 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.644 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.644 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.644 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.644 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.644 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.644 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.644 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.644 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.644 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.645 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.646 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.647 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 15w REG 8,1 12072552 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.648 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.649 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.650 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.651 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101954 tp_osd_tp root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root cwd DIR 8,1 4096 265244 /home/ubuntu/cephtest 2023-12-06T18:42:58.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root txt REG 8,1 2658080 27585 /usr/libexec/valgrind/memcheck-amd64-linux 2023-12-06T18:42:58.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 22256 27655 /usr/libexec/valgrind/vgpreload_core-amd64-linux.so 2023-12-06T18:42:58.652 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 342328 27660 /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so 2023-12-06T18:42:58.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 326760 2334 /usr/lib64/libtcmalloc.so.4.5.9 2023-12-06T18:42:58.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 229360 10126 /usr/lib64/libblkid.so.1.1.0 2023-12-06T18:42:58.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 270800 9964 /usr/lib64/libfuse.so.2.9.9 2023-12-06T18:42:58.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 16448 9879 /usr/lib64/libaio.so.1.0.1 2023-12-06T18:42:58.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 49744 10813 /usr/lib64/libsnappy.so.1.1.8 2023-12-06T18:42:58.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 144120 9867 /usr/lib64/liblz4.so.1.9.3 2023-12-06T18:42:58.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 662304 11485 /usr/lib64/libcurl.so.4.7.0 2023-12-06T18:42:58.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 639224 2283 /usr/lib64/libthrift-0.15.0.so 2023-12-06T18:42:58.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 200144 10124 /usr/lib64/libudev.so.1.7.5 2023-12-06T18:42:58.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 137720 10598 /usr/lib64/libibverbs.so.1.14.46.0 2023-12-06T18:42:58.653 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 112536 10937 /usr/lib64/librdmacm.so.1.3.46.0 2023-12-06T18:42:58.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 2306456 9691 /usr/lib64/libstdc++.so.6.0.29 2023-12-06T18:42:58.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 61880 2324 /usr/lib64/libunwind.so.8.0.1 2023-12-06T18:42:58.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 169704 11377 /usr/lib64/libnghttp2.so.14.20.1 2023-12-06T18:42:58.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 130952 9772 /usr/lib64/libidn2.so.0.3.7 2023-12-06T18:42:58.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 483672 11481 /usr/lib64/libssh.so.4.9.4 2023-12-06T18:42:58.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 77488 9888 /usr/lib64/libpsl.so.5.3.3 2023-12-06T18:42:58.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.654 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 53048 9892 /usr/lib64/libbrotlidec.so.1.0.9 2023-12-06T18:42:58.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 600544 9820 /usr/lib64/libnl-route-3.so.200.26.0 2023-12-06T18:42:58.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 146872 9812 /usr/lib64/libnl-3.so.200.26.0 2023-12-06T18:42:58.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 1591920 9734 /usr/lib64/libunistring.so.2.1.0 2023-12-06T18:42:58.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 138712 9890 /usr/lib64/libbrotlicommon.so.1.0.9 2023-12-06T18:42:58.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 53272 2260 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-12-06T18:42:58.655 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 53816 1768 /usr/lib64/libnuma.so.1.0.0 2023-12-06T18:42:58.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 41360 11862 /usr/lib64/liburcu-bp.so.6.1.0 2023-12-06T18:42:58.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 45480 11864 /usr/lib64/liburcu-cds.so.6.1.0 2023-12-06T18:42:58.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 24248 11866 /usr/lib64/liburcu-common.so.6.1.0 2023-12-06T18:42:58.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 74680 265543 /usr/lib64/ceph/extblkdev/libceph_ebd_vdo.so 2023-12-06T18:42:58.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 353376 265533 /usr/lib64/ceph/erasure-code/libec_jerasure.so 2023-12-06T18:42:58.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 1473200 265537 /usr/lib64/ceph/erasure-code/libec_lrc.so 2023-12-06T18:42:58.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 32664 265518 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-12-06T18:42:58.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 32512 265577 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-12-06T18:42:58.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 37064 265562 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-12-06T18:42:58.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 409304 265586 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-12-06T18:42:58.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 70432 265580 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-12-06T18:42:58.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 69736 2298 /usr/lib64/liboath.so.0.1.3 2023-12-06T18:42:58.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 70536 265556 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-12-06T18:42:58.656 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 36976 265610 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-12-06T18:42:58.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 41168 265553 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-12-06T18:42:58.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 74936 265550 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-12-06T18:42:58.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 65968 265607 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-12-06T18:42:58.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 45176 265604 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-12-06T18:42:58.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 864216 265589 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-12-06T18:42:58.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 212272 265559 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-12-06T18:42:58.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 1338632 265592 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-12-06T18:42:58.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 53512 265571 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-12-06T18:42:58.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 83008 265565 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-12-06T18:42:58.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 91240 265583 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-12-06T18:42:58.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 128912 265547 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-12-06T18:42:58.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 116424 265595 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-12-06T18:42:58.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 78600 265568 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-12-06T18:42:58.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root mem REG 8,1 48 3357 /tmp/vgdb-pipe-shared-mem-vgdb-99450-by-root-on-??? 2023-12-06T18:42:58.657 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root DEL REG 0,17 209393 /[aio] 2023-12-06T18:42:58.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root DEL REG 0,17 209390 /[aio] 2023-12-06T18:42:58.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 0r CHR 1,3 0t0 4 /dev/null 2023-12-06T18:42:58.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 1w FIFO 0,13 0t0 204574 pipe 2023-12-06T18:42:58.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 2w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 3w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 4r FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 5w FIFO 0,13 0t0 207932 pipe 2023-12-06T18:42:58.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 6u a_inode 0,14 0 13316 [eventpoll:7,20,21,22,23,62,89] 2023-12-06T18:42:58.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 7r FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 8w FIFO 0,13 0t0 206294 pipe 2023-12-06T18:42:58.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 9u a_inode 0,14 0 13316 [eventpoll:10,82] 2023-12-06T18:42:58.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 10r FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 11w FIFO 0,13 0t0 206295 pipe 2023-12-06T18:42:58.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 12u a_inode 0,14 0 13316 [eventpoll:13,61,81] 2023-12-06T18:42:58.658 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 13r FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 14w FIFO 0,13 0t0 206296 pipe 2023-12-06T18:42:58.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 15w REG 8,1 12072552 267044 /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:58.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 16uW REG 0,24 6 2001 /run/ceph/ceph-osd.7.pid 2023-12-06T18:42:58.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 17r FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 18w FIFO 0,13 0t0 207371 pipe 2023-12-06T18:42:58.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 19u unix 0xffff8a5747d8ee80 0t0 207372 /var/run/ceph/ceph-osd.7.asok type=STREAM (LISTEN) 2023-12-06T18:42:58.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 20u IPv4 206401 0t0 TCP *:6804 (LISTEN) 2023-12-06T18:42:58.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 21u IPv4 206407 0t0 TCP *:6805 (LISTEN) 2023-12-06T18:42:58.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 22u IPv4 206414 0t0 TCP *:6806 (LISTEN) 2023-12-06T18:42:58.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 23u IPv4 206422 0t0 TCP *:6807 (LISTEN) 2023-12-06T18:42:58.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 24r FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 25w FIFO 0,13 0t0 210004 pipe 2023-12-06T18:42:58.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 26r FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 27w FIFO 0,13 0t0 210005 pipe 2023-12-06T18:42:58.659 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 28u sock 0,8 0t0 210081 protocol: UDPv6 2023-12-06T18:42:58.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 29r DIR 253,3 357 64 /var/lib/ceph/osd/ceph-7 2023-12-06T18:42:58.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 30uW REG 253,3 37 69 /var/lib/ceph/osd/ceph-7/fsid 2023-12-06T18:42:58.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 31u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 32u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 33u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 34u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 35u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 36u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 37u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 38u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 39u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 40u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.660 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 41u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 42u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 43u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 44u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 45u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 46u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 47u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 48u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 49u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 50u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 51u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 52u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 53u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 54u REG 253,3 96636764160 70 /var/lib/ceph/osd/ceph-7/block 2023-12-06T18:42:58.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 55r FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.661 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 57w FIFO 0,13 0t0 212714 pipe 2023-12-06T18:42:58.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 58r FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 59w FIFO 0,13 0t0 216497 pipe 2023-12-06T18:42:58.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 61u IPv4 256770 0t0 TCP smithi152.front.sepia.ceph.com:34772->smithi152.front.sepia.ceph.com:6815 (ESTABLISHED) 2023-12-06T18:42:58.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 62u IPv4 255574 0t0 TCP smithi152.front.sepia.ceph.com:60502->smithi152.front.sepia.ceph.com:6813 (ESTABLISHED) 2023-12-06T18:42:58.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 81u IPv4 230741 0t0 TCP smithi152.front.sepia.ceph.com:6807->smithi152.front.sepia.ceph.com:39970 (ESTABLISHED) 2023-12-06T18:42:58.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 82u IPv4 215472 0t0 TCP smithi152.front.sepia.ceph.com:42056->smithi152.front.sepia.ceph.com:6811 (ESTABLISHED) 2023-12-06T18:42:58.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root 89u IPv4 230743 0t0 TCP smithi152.front.sepia.ceph.com:6806->smithi152.front.sepia.ceph.com:44768 (ESTABLISHED) 2023-12-06T18:42:58.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root *524r REG 8,1 25562848 2808 /usr/bin/ceph-osd 2023-12-06T18:42:58.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root *525u REG 8,1 32 3349 /tmp/valgrind_proc_99450_cmdline_d3ad7e56 (deleted) 2023-12-06T18:42:58.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root *526u REG 8,1 336 3358 /tmp/valgrind_proc_99450_auxv_d3ad7e56 (deleted) 2023-12-06T18:42:58.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root *527w FIFO 0,13 0t0 204575 pipe 2023-12-06T18:42:58.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root *528w REG 8,1 1234 655634 /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:58.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root *529r FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root *530w FIFO 0,13 0t0 207931 pipe 2023-12-06T18:42:58.662 INFO:teuthology.orchestra.run.smithi152.stdout:memcheck- 99450 101955 osd_srv_h root *531r FIFO 8,1 0t0 3367 /tmp/vgdb-pipe-from-vgdb-to-99450-by-root-on-??? 2023-12-06T18:42:58.663 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 103036 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:58.663 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 103036 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.663 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 103036 root txt unknown /proc/103036/exe 2023-12-06T18:42:58.663 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/6 109816 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:58.663 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/6 109816 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.663 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/6 109816 root txt unknown /proc/109816/exe 2023-12-06T18:42:58.663 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/4 116136 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:58.663 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/4 116136 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.663 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/4 116136 root txt unknown /proc/116136/exe 2023-12-06T18:42:58.663 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/6 116137 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:58.663 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/6 116137 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.663 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/6 116137 root txt unknown /proc/116137/exe 2023-12-06T18:42:58.663 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/2 116139 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:58.663 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/2 116139 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.664 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/2 116139 root txt unknown /proc/116139/exe 2023-12-06T18:42:58.664 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/7 116140 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:58.664 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/7 116140 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.664 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/7 116140 root txt unknown /proc/116140/exe 2023-12-06T18:42:58.664 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 116177 root cwd DIR 8,1 4096 2 / 2023-12-06T18:42:58.664 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 116177 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.664 INFO:teuthology.orchestra.run.smithi152.stdout:kworker/u 116177 root txt unknown /proc/116177/exe 2023-12-06T18:42:58.664 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu cwd DIR 8,1 4096 1026 /home/ubuntu 2023-12-06T18:42:58.664 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.664 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu txt REG 8,1 1388920 9667 /usr/bin/bash 2023-12-06T18:42:58.664 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-12-06T18:42:58.664 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.664 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:58.664 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-12-06T18:42:58.664 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-12-06T18:42:58.664 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-12-06T18:42:58.664 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-12-06T18:42:58.665 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-12-06T18:42:58.665 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-12-06T18:42:58.665 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-12-06T18:42:58.665 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-12-06T18:42:58.665 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-12-06T18:42:58.665 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu mem REG 8,1 195080 9664 /usr/lib64/libtinfo.so.6.2 2023-12-06T18:42:58.665 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:58.665 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.665 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-12-06T18:42:58.665 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu 0r FIFO 0,13 0t0 253922 pipe 2023-12-06T18:42:58.665 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu 1w FIFO 0,13 0t0 253923 pipe 2023-12-06T18:42:58.665 INFO:teuthology.orchestra.run.smithi152.stdout:bash 116199 ubuntu 2w FIFO 0,13 0t0 253924 pipe 2023-12-06T18:42:58.665 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-12-06T18:42:58.665 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.665 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root txt REG 8,1 185296 11759 /usr/bin/sudo 2023-12-06T18:42:58.665 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 357408 10120 /usr/lib64/libnss_systemd.so.2 2023-12-06T18:42:58.666 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 44784 9785 /usr/lib64/libffi.so.8.1.0 2023-12-06T18:42:58.666 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 108136 13107 /usr/lib64/libgcc_s-11-20230605.so.1 2023-12-06T18:42:58.666 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 1293840 9789 /usr/lib64/libp11-kit.so.0.3.0 2023-12-06T18:42:58.666 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 36296 9702 /usr/lib64/libcap.so.2.48 2023-12-06T18:42:58.666 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 19600 10967 /usr/lib64/libpam_misc.so.0.82.1 2023-12-06T18:42:58.666 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 19400 32305 /usr/lib64/security/pam_succeed_if.so 2023-12-06T18:42:58.666 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 514280 32422 /usr/lib64/security/pam_systemd.so 2023-12-06T18:42:58.666 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 27536 32284 /usr/lib64/security/pam_limits.so 2023-12-06T18:42:58.666 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 48392 10228 /usr/lib64/libcrack.so.2.9.0 2023-12-06T18:42:58.666 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.666 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 32608 10961 /usr/lib64/libpwquality.so.1.0.2 2023-12-06T18:42:58.666 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 15352 32282 /usr/lib64/security/pam_keyinit.so 2023-12-06T18:42:58.667 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 15840 32264 /usr/lib64/security/pam_pwquality.so 2023-12-06T18:42:58.667 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 14928 32270 /usr/lib64/security/pam_deny.so 2023-12-06T18:42:58.667 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 56712 32310 /usr/lib64/security/pam_unix.so 2023-12-06T18:42:58.667 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 15184 32274 /usr/lib64/security/pam_faildelay.so 2023-12-06T18:42:58.667 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.667 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.667 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.667 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.667 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.667 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.667 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.667 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 201808 9696 /usr/lib64/libcrypt.so.2.0.0 2023-12-06T18:42:58.667 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 688208 10009 /usr/lib64/libssl.so.3.0.7 2023-12-06T18:42:58.667 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 366424 10411 /usr/lib64/libevent-2.1.so.7.0.1 2023-12-06T18:42:58.667 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 904672 9627 /usr/lib64/libm.so.6 2023-12-06T18:42:58.668 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 2586930 17848 /usr/lib/locale/en_US.utf8/LC_COLLATE 2023-12-06T18:42:58.668 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 128416 11415 /usr/lib64/libsasl2.so.3.0.0 2023-12-06T18:42:58.668 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 40424 10956 /usr/lib64/libeconf.so.0.4.1 2023-12-06T18:42:58.668 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.668 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 70400 11419 /usr/lib64/liblber.so.2.0.200 2023-12-06T18:42:58.668 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 417792 11423 /usr/lib64/libldap_r.so.2.0.200 2023-12-06T18:42:58.668 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 472568 54513 /usr/libexec/sudo/sudoers.so 2023-12-06T18:42:58.668 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:58.668 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.668 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 69472 10965 /usr/lib64/libpam.so.0.85.1 2023-12-06T18:42:58.668 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 19464 32272 /usr/lib64/security/pam_env.so 2023-12-06T18:42:58.668 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 54 17851 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2023-12-06T18:42:58.668 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 3284 30787 /usr/lib/locale/en_US.utf8/LC_TIME 2023-12-06T18:42:58.668 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 286 30784 /usr/lib/locale/en_US.utf8/LC_MONETARY 2023-12-06T18:42:58.669 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 57 28351 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2023-12-06T18:42:58.669 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 34 30837 /usr/lib/locale/en_US.utf8/LC_PAPER 2023-12-06T18:42:58.669 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 77 17850 /usr/lib/locale/en_US.utf8/LC_NAME 2023-12-06T18:42:58.669 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 167 30779 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2023-12-06T18:42:58.669 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 59 30786 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2023-12-06T18:42:58.669 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 23 30783 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2023-12-06T18:42:58.669 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.669 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 32528 9723 /usr/lib64/libcap-ng.so.0.0.0 2023-12-06T18:42:58.669 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 119832 54509 /usr/libexec/sudo/libsudo_util.so.0.0.0 2023-12-06T18:42:58.669 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.669 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 134624 9727 /usr/lib64/libaudit.so.1.0.0 2023-12-06T18:42:58.669 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:58.669 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.669 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root mem REG 8,1 369 30782 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2023-12-06T18:42:58.670 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root 0r FIFO 0,13 0t0 253922 pipe 2023-12-06T18:42:58.670 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root 1w FIFO 0,13 0t0 253923 pipe 2023-12-06T18:42:58.670 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root 2w FIFO 0,13 0t0 253924 pipe 2023-12-06T18:42:58.670 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root 3r FIFO 0,13 0t0 261422 pipe 2023-12-06T18:42:58.670 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root 4w FIFO 0,13 0t0 261422 pipe 2023-12-06T18:42:58.670 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root 5u netlink 0t0 261428 AUDIT 2023-12-06T18:42:58.670 INFO:teuthology.orchestra.run.smithi152.stdout:sudo 116215 root 6u unix 0xffff8a57473b1980 0t0 261429 type=DGRAM (CONNECTED) 2023-12-06T18:42:58.670 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-12-06T18:42:58.670 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.670 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root txt REG 8,1 179232 12343 /usr/bin/lsof 2023-12-06T18:42:58.670 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.670 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:58.670 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.671 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.671 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.671 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.671 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.671 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.671 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.671 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.671 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.671 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.671 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root mem REG 8,1 196112 11431 /usr/lib64/libtirpc.so.3.0.0 2023-12-06T18:42:58.671 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.671 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:58.671 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.671 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root 0r FIFO 0,13 0t0 253922 pipe 2023-12-06T18:42:58.671 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root 1w FIFO 0,13 0t0 253923 pipe 2023-12-06T18:42:58.671 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root 2w FIFO 0,13 0t0 253924 pipe 2023-12-06T18:42:58.671 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root 3r DIR 0,20 0 1 /proc 2023-12-06T18:42:58.672 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root 4r DIR 0,20 0 255634 /proc/116217/fd 2023-12-06T18:42:58.673 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root 5w FIFO 0,13 0t0 255639 pipe 2023-12-06T18:42:58.673 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116217 root 6r FIFO 0,13 0t0 255640 pipe 2023-12-06T18:42:58.673 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root cwd DIR 8,1 4096 1026 /home/ubuntu 2023-12-06T18:42:58.673 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root rtd DIR 8,1 4096 2 / 2023-12-06T18:42:58.673 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root txt REG 8,1 179232 12343 /usr/bin/lsof 2023-12-06T18:42:58.673 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root mem REG 8,1 4483280 10007 /usr/lib64/libcrypto.so.3.0.7 2023-12-06T18:42:58.673 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root mem REG 8,1 346132 30861 /usr/lib/locale/C.utf8/LC_CTYPE 2023-12-06T18:42:58.673 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root mem REG 8,1 102552 9686 /usr/lib64/libz.so.1.2.11 2023-12-06T18:42:58.674 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root mem REG 8,1 70512 9633 /usr/lib64/libresolv.so.2 2023-12-06T18:42:58.674 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root mem REG 8,1 906256 11403 /usr/lib64/libkrb5.so.3.3 2023-12-06T18:42:58.674 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root mem REG 8,1 358448 11393 /usr/lib64/libgssapi_krb5.so.2.2 2023-12-06T18:42:58.674 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root mem REG 8,1 2387016 9624 /usr/lib64/libc.so.6 2023-12-06T18:42:58.674 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root mem REG 8,1 23992 9760 /usr/lib64/libkeyutils.so.1.10 2023-12-06T18:42:58.674 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root mem REG 8,1 66192 11405 /usr/lib64/libkrb5support.so.0.1 2023-12-06T18:42:58.674 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root mem REG 8,1 24400 9711 /usr/lib64/libcom_err.so.2.1 2023-12-06T18:42:58.674 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root mem REG 8,1 98992 11397 /usr/lib64/libk5crypto.so.3.1 2023-12-06T18:42:58.674 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root mem REG 8,1 637872 9740 /usr/lib64/libpcre2-8.so.0.11.0 2023-12-06T18:42:58.674 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root mem REG 8,1 196112 11431 /usr/lib64/libtirpc.so.3.0.0 2023-12-06T18:42:58.674 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root mem REG 8,1 175552 9824 /usr/lib64/libselinux.so.1 2023-12-06T18:42:58.674 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root mem REG 8,1 26988 29960 /usr/lib64/gconv/gconv-modules.cache 2023-12-06T18:42:58.674 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root mem REG 8,1 844808 9620 /usr/lib64/ld-linux-x86-64.so.2 2023-12-06T18:42:58.674 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root 4r FIFO 0,13 0t0 255639 pipe 2023-12-06T18:42:58.675 INFO:teuthology.orchestra.run.smithi152.stdout:lsof 116218 root 7w FIFO 0,13 0t0 255640 pipe 2023-12-06T18:42:58.675 INFO:teuthology.orchestra.run.smithi152.stdout:USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND 2023-12-06T18:42:58.675 INFO:teuthology.orchestra.run.smithi152.stdout:root 2 0.0 0.0 0 0 ? S 18:05 0:00 [kthreadd] 2023-12-06T18:42:58.675 INFO:teuthology.orchestra.run.smithi152.stdout:root 3 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [rcu_gp] 2023-12-06T18:42:58.675 INFO:teuthology.orchestra.run.smithi152.stdout:root 4 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [rcu_par_gp] 2023-12-06T18:42:58.675 INFO:teuthology.orchestra.run.smithi152.stdout:root 5 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [slub_flushwq] 2023-12-06T18:42:58.675 INFO:teuthology.orchestra.run.smithi152.stdout:root 6 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [netns] 2023-12-06T18:42:58.675 INFO:teuthology.orchestra.run.smithi152.stdout:root 8 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kworker/0:0H-events_highpri] 2023-12-06T18:42:58.675 INFO:teuthology.orchestra.run.smithi152.stdout:root 10 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [mm_percpu_wq] 2023-12-06T18:42:58.675 INFO:teuthology.orchestra.run.smithi152.stdout:root 12 0.0 0.0 0 0 ? I 18:05 0:00 \_ [rcu_tasks_kthre] 2023-12-06T18:42:58.675 INFO:teuthology.orchestra.run.smithi152.stdout:root 13 0.0 0.0 0 0 ? I 18:05 0:00 \_ [rcu_tasks_rude_] 2023-12-06T18:42:58.675 INFO:teuthology.orchestra.run.smithi152.stdout:root 14 0.0 0.0 0 0 ? I 18:05 0:00 \_ [rcu_tasks_trace] 2023-12-06T18:42:58.675 INFO:teuthology.orchestra.run.smithi152.stdout:root 15 0.0 0.0 0 0 ? S 18:05 0:00 \_ [ksoftirqd/0] 2023-12-06T18:42:58.675 INFO:teuthology.orchestra.run.smithi152.stdout:root 16 0.2 0.0 0 0 ? S 18:05 0:04 \_ [pr/ttyS1] 2023-12-06T18:42:58.675 INFO:teuthology.orchestra.run.smithi152.stdout:root 17 0.0 0.0 0 0 ? I 18:05 0:00 \_ [rcu_preempt] 2023-12-06T18:42:58.676 INFO:teuthology.orchestra.run.smithi152.stdout:root 18 0.0 0.0 0 0 ? S 18:05 0:00 \_ [migration/0] 2023-12-06T18:42:58.676 INFO:teuthology.orchestra.run.smithi152.stdout:root 19 0.0 0.0 0 0 ? S 18:05 0:00 \_ [idle_inject/0] 2023-12-06T18:42:58.676 INFO:teuthology.orchestra.run.smithi152.stdout:root 21 0.0 0.0 0 0 ? S 18:05 0:00 \_ [cpuhp/0] 2023-12-06T18:42:58.676 INFO:teuthology.orchestra.run.smithi152.stdout:root 22 0.0 0.0 0 0 ? S 18:05 0:00 \_ [cpuhp/1] 2023-12-06T18:42:58.676 INFO:teuthology.orchestra.run.smithi152.stdout:root 23 0.0 0.0 0 0 ? S 18:05 0:00 \_ [idle_inject/1] 2023-12-06T18:42:58.676 INFO:teuthology.orchestra.run.smithi152.stdout:root 24 0.0 0.0 0 0 ? S 18:05 0:00 \_ [migration/1] 2023-12-06T18:42:58.676 INFO:teuthology.orchestra.run.smithi152.stdout:root 25 0.0 0.0 0 0 ? S 18:05 0:00 \_ [ksoftirqd/1] 2023-12-06T18:42:58.676 INFO:teuthology.orchestra.run.smithi152.stdout:root 27 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kworker/1:0H-events_highpri] 2023-12-06T18:42:58.676 INFO:teuthology.orchestra.run.smithi152.stdout:root 28 0.0 0.0 0 0 ? S 18:05 0:00 \_ [cpuhp/2] 2023-12-06T18:42:58.676 INFO:teuthology.orchestra.run.smithi152.stdout:root 29 0.0 0.0 0 0 ? S 18:05 0:00 \_ [idle_inject/2] 2023-12-06T18:42:58.676 INFO:teuthology.orchestra.run.smithi152.stdout:root 30 0.0 0.0 0 0 ? S 18:05 0:00 \_ [migration/2] 2023-12-06T18:42:58.676 INFO:teuthology.orchestra.run.smithi152.stdout:root 31 0.0 0.0 0 0 ? S 18:05 0:00 \_ [ksoftirqd/2] 2023-12-06T18:42:58.676 INFO:teuthology.orchestra.run.smithi152.stdout:root 33 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kworker/2:0H-events_highpri] 2023-12-06T18:42:58.676 INFO:teuthology.orchestra.run.smithi152.stdout:root 34 0.0 0.0 0 0 ? S 18:05 0:00 \_ [cpuhp/3] 2023-12-06T18:42:58.676 INFO:teuthology.orchestra.run.smithi152.stdout:root 35 0.0 0.0 0 0 ? S 18:05 0:00 \_ [idle_inject/3] 2023-12-06T18:42:58.677 INFO:teuthology.orchestra.run.smithi152.stdout:root 36 0.0 0.0 0 0 ? S 18:05 0:00 \_ [migration/3] 2023-12-06T18:42:58.677 INFO:teuthology.orchestra.run.smithi152.stdout:root 37 0.0 0.0 0 0 ? S 18:05 0:00 \_ [ksoftirqd/3] 2023-12-06T18:42:58.677 INFO:teuthology.orchestra.run.smithi152.stdout:root 39 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kworker/3:0H-events_highpri] 2023-12-06T18:42:58.677 INFO:teuthology.orchestra.run.smithi152.stdout:root 40 0.0 0.0 0 0 ? S 18:05 0:00 \_ [cpuhp/4] 2023-12-06T18:42:58.677 INFO:teuthology.orchestra.run.smithi152.stdout:root 41 0.0 0.0 0 0 ? S 18:05 0:00 \_ [idle_inject/4] 2023-12-06T18:42:58.677 INFO:teuthology.orchestra.run.smithi152.stdout:root 42 0.0 0.0 0 0 ? S 18:05 0:00 \_ [migration/4] 2023-12-06T18:42:58.677 INFO:teuthology.orchestra.run.smithi152.stdout:root 43 0.0 0.0 0 0 ? S 18:05 0:00 \_ [ksoftirqd/4] 2023-12-06T18:42:58.677 INFO:teuthology.orchestra.run.smithi152.stdout:root 45 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kworker/4:0H-events_highpri] 2023-12-06T18:42:58.677 INFO:teuthology.orchestra.run.smithi152.stdout:root 46 0.0 0.0 0 0 ? S 18:05 0:00 \_ [cpuhp/5] 2023-12-06T18:42:58.677 INFO:teuthology.orchestra.run.smithi152.stdout:root 47 0.0 0.0 0 0 ? S 18:05 0:00 \_ [idle_inject/5] 2023-12-06T18:42:58.677 INFO:teuthology.orchestra.run.smithi152.stdout:root 48 0.0 0.0 0 0 ? S 18:05 0:00 \_ [migration/5] 2023-12-06T18:42:58.677 INFO:teuthology.orchestra.run.smithi152.stdout:root 49 0.0 0.0 0 0 ? S 18:05 0:00 \_ [ksoftirqd/5] 2023-12-06T18:42:58.677 INFO:teuthology.orchestra.run.smithi152.stdout:root 50 0.0 0.0 0 0 ? I 18:05 0:00 \_ [kworker/5:0-mm_percpu_wq] 2023-12-06T18:42:58.677 INFO:teuthology.orchestra.run.smithi152.stdout:root 51 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kworker/5:0H-events_highpri] 2023-12-06T18:42:58.677 INFO:teuthology.orchestra.run.smithi152.stdout:root 52 0.0 0.0 0 0 ? S 18:05 0:00 \_ [cpuhp/6] 2023-12-06T18:42:58.677 INFO:teuthology.orchestra.run.smithi152.stdout:root 53 0.0 0.0 0 0 ? S 18:05 0:00 \_ [idle_inject/6] 2023-12-06T18:42:58.678 INFO:teuthology.orchestra.run.smithi152.stdout:root 54 0.0 0.0 0 0 ? S 18:05 0:00 \_ [migration/6] 2023-12-06T18:42:58.678 INFO:teuthology.orchestra.run.smithi152.stdout:root 55 0.0 0.0 0 0 ? S 18:05 0:00 \_ [ksoftirqd/6] 2023-12-06T18:42:58.678 INFO:teuthology.orchestra.run.smithi152.stdout:root 57 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kworker/6:0H-events_highpri] 2023-12-06T18:42:58.678 INFO:teuthology.orchestra.run.smithi152.stdout:root 58 0.0 0.0 0 0 ? S 18:05 0:00 \_ [cpuhp/7] 2023-12-06T18:42:58.678 INFO:teuthology.orchestra.run.smithi152.stdout:root 59 0.0 0.0 0 0 ? S 18:05 0:00 \_ [idle_inject/7] 2023-12-06T18:42:58.678 INFO:teuthology.orchestra.run.smithi152.stdout:root 60 0.0 0.0 0 0 ? S 18:05 0:00 \_ [migration/7] 2023-12-06T18:42:58.678 INFO:teuthology.orchestra.run.smithi152.stdout:root 61 0.0 0.0 0 0 ? S 18:05 0:00 \_ [ksoftirqd/7] 2023-12-06T18:42:58.678 INFO:teuthology.orchestra.run.smithi152.stdout:root 63 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kworker/7:0H-events_highpri] 2023-12-06T18:42:58.678 INFO:teuthology.orchestra.run.smithi152.stdout:root 70 0.0 0.0 0 0 ? I 18:05 0:00 \_ [kworker/u16:7-flush-8:0] 2023-12-06T18:42:58.678 INFO:teuthology.orchestra.run.smithi152.stdout:root 71 0.0 0.0 0 0 ? S 18:05 0:00 \_ [kdevtmpfs] 2023-12-06T18:42:58.678 INFO:teuthology.orchestra.run.smithi152.stdout:root 72 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [inet_frag_wq] 2023-12-06T18:42:58.678 INFO:teuthology.orchestra.run.smithi152.stdout:root 73 0.0 0.0 0 0 ? S 18:05 0:00 \_ [kauditd] 2023-12-06T18:42:58.678 INFO:teuthology.orchestra.run.smithi152.stdout:root 75 0.0 0.0 0 0 ? S 18:05 0:00 \_ [khungtaskd] 2023-12-06T18:42:58.678 INFO:teuthology.orchestra.run.smithi152.stdout:root 76 0.0 0.0 0 0 ? S 18:05 0:00 \_ [oom_reaper] 2023-12-06T18:42:58.678 INFO:teuthology.orchestra.run.smithi152.stdout:root 77 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [writeback] 2023-12-06T18:42:58.679 INFO:teuthology.orchestra.run.smithi152.stdout:root 78 0.0 0.0 0 0 ? S 18:05 0:00 \_ [kcompactd0] 2023-12-06T18:42:58.679 INFO:teuthology.orchestra.run.smithi152.stdout:root 79 0.0 0.0 0 0 ? SN 18:05 0:00 \_ [ksmd] 2023-12-06T18:42:58.679 INFO:teuthology.orchestra.run.smithi152.stdout:root 81 0.0 0.0 0 0 ? SN 18:05 0:00 \_ [khugepaged] 2023-12-06T18:42:58.679 INFO:teuthology.orchestra.run.smithi152.stdout:root 82 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [cryptd] 2023-12-06T18:42:58.679 INFO:teuthology.orchestra.run.smithi152.stdout:root 83 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kintegrityd] 2023-12-06T18:42:58.679 INFO:teuthology.orchestra.run.smithi152.stdout:root 84 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kblockd] 2023-12-06T18:42:58.679 INFO:teuthology.orchestra.run.smithi152.stdout:root 85 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [blkcg_punt_bio] 2023-12-06T18:42:58.679 INFO:teuthology.orchestra.run.smithi152.stdout:root 86 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [tpm_dev_wq] 2023-12-06T18:42:58.679 INFO:teuthology.orchestra.run.smithi152.stdout:root 87 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [md] 2023-12-06T18:42:58.679 INFO:teuthology.orchestra.run.smithi152.stdout:root 88 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [md_bitmap] 2023-12-06T18:42:58.679 INFO:teuthology.orchestra.run.smithi152.stdout:root 89 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [edac-poller] 2023-12-06T18:42:58.679 INFO:teuthology.orchestra.run.smithi152.stdout:root 90 0.0 0.0 0 0 ? S 18:05 0:00 \_ [watchdogd] 2023-12-06T18:42:58.679 INFO:teuthology.orchestra.run.smithi152.stdout:root 93 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kworker/5:1H-kblockd] 2023-12-06T18:42:58.679 INFO:teuthology.orchestra.run.smithi152.stdout:root 95 0.0 0.0 0 0 ? S 18:05 0:00 \_ [kswapd0] 2023-12-06T18:42:58.679 INFO:teuthology.orchestra.run.smithi152.stdout:root 101 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kthrotld] 2023-12-06T18:42:58.679 INFO:teuthology.orchestra.run.smithi152.stdout:root 109 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [acpi_thermal_pm] 2023-12-06T18:42:58.680 INFO:teuthology.orchestra.run.smithi152.stdout:root 110 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kmpath_rdacd] 2023-12-06T18:42:58.680 INFO:teuthology.orchestra.run.smithi152.stdout:root 111 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kaluad] 2023-12-06T18:42:58.680 INFO:teuthology.orchestra.run.smithi152.stdout:root 112 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [mld] 2023-12-06T18:42:58.680 INFO:teuthology.orchestra.run.smithi152.stdout:root 113 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kworker/4:1H-kblockd] 2023-12-06T18:42:58.680 INFO:teuthology.orchestra.run.smithi152.stdout:root 114 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [ipv6_addrconf] 2023-12-06T18:42:58.680 INFO:teuthology.orchestra.run.smithi152.stdout:root 123 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kstrp] 2023-12-06T18:42:58.680 INFO:teuthology.orchestra.run.smithi152.stdout:root 129 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [zswap-shrink] 2023-12-06T18:42:58.680 INFO:teuthology.orchestra.run.smithi152.stdout:root 238 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kworker/u17:0] 2023-12-06T18:42:58.680 INFO:teuthology.orchestra.run.smithi152.stdout:root 288 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kworker/0:1H-xfs-log/dm-1] 2023-12-06T18:42:58.680 INFO:teuthology.orchestra.run.smithi152.stdout:root 289 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kworker/2:1H-kblockd] 2023-12-06T18:42:58.680 INFO:teuthology.orchestra.run.smithi152.stdout:root 330 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kworker/7:1H-kblockd] 2023-12-06T18:42:58.680 INFO:teuthology.orchestra.run.smithi152.stdout:root 357 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kworker/6:1H-kblockd] 2023-12-06T18:42:58.680 INFO:teuthology.orchestra.run.smithi152.stdout:root 377 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kworker/1:1H-kblockd] 2023-12-06T18:42:58.680 INFO:teuthology.orchestra.run.smithi152.stdout:root 411 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [kworker/3:1H-kblockd] 2023-12-06T18:42:58.680 INFO:teuthology.orchestra.run.smithi152.stdout:root 427 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [nvme-wq] 2023-12-06T18:42:58.681 INFO:teuthology.orchestra.run.smithi152.stdout:root 428 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [nvme-reset-wq] 2023-12-06T18:42:58.681 INFO:teuthology.orchestra.run.smithi152.stdout:root 429 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [nvme-delete-wq] 2023-12-06T18:42:58.681 INFO:teuthology.orchestra.run.smithi152.stdout:root 430 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [nvme-auth-wq] 2023-12-06T18:42:58.681 INFO:teuthology.orchestra.run.smithi152.stdout:root 431 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [ata_sff] 2023-12-06T18:42:58.681 INFO:teuthology.orchestra.run.smithi152.stdout:root 434 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [ixgbe] 2023-12-06T18:42:58.681 INFO:teuthology.orchestra.run.smithi152.stdout:root 443 0.0 0.0 0 0 ? S 18:05 0:00 \_ [scsi_eh_0] 2023-12-06T18:42:58.681 INFO:teuthology.orchestra.run.smithi152.stdout:root 444 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [scsi_tmf_0] 2023-12-06T18:42:58.681 INFO:teuthology.orchestra.run.smithi152.stdout:root 445 0.0 0.0 0 0 ? S 18:05 0:00 \_ [scsi_eh_1] 2023-12-06T18:42:58.681 INFO:teuthology.orchestra.run.smithi152.stdout:root 446 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [scsi_tmf_1] 2023-12-06T18:42:58.681 INFO:teuthology.orchestra.run.smithi152.stdout:root 447 0.0 0.0 0 0 ? S 18:05 0:00 \_ [scsi_eh_2] 2023-12-06T18:42:58.681 INFO:teuthology.orchestra.run.smithi152.stdout:root 448 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [scsi_tmf_2] 2023-12-06T18:42:58.681 INFO:teuthology.orchestra.run.smithi152.stdout:root 449 0.0 0.0 0 0 ? S 18:05 0:00 \_ [scsi_eh_3] 2023-12-06T18:42:58.681 INFO:teuthology.orchestra.run.smithi152.stdout:root 450 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [scsi_tmf_3] 2023-12-06T18:42:58.681 INFO:teuthology.orchestra.run.smithi152.stdout:root 456 0.0 0.0 0 0 ? S 18:05 0:00 \_ [scsi_eh_4] 2023-12-06T18:42:58.681 INFO:teuthology.orchestra.run.smithi152.stdout:root 457 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [scsi_tmf_4] 2023-12-06T18:42:58.681 INFO:teuthology.orchestra.run.smithi152.stdout:root 458 0.0 0.0 0 0 ? S 18:05 0:00 \_ [scsi_eh_5] 2023-12-06T18:42:58.681 INFO:teuthology.orchestra.run.smithi152.stdout:root 459 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [scsi_tmf_5] 2023-12-06T18:42:58.682 INFO:teuthology.orchestra.run.smithi152.stdout:root 460 0.0 0.0 0 0 ? S 18:05 0:00 \_ [scsi_eh_6] 2023-12-06T18:42:58.682 INFO:teuthology.orchestra.run.smithi152.stdout:root 461 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [scsi_tmf_6] 2023-12-06T18:42:58.682 INFO:teuthology.orchestra.run.smithi152.stdout:root 462 0.0 0.0 0 0 ? S 18:05 0:00 \_ [scsi_eh_7] 2023-12-06T18:42:58.682 INFO:teuthology.orchestra.run.smithi152.stdout:root 463 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [scsi_tmf_7] 2023-12-06T18:42:58.682 INFO:teuthology.orchestra.run.smithi152.stdout:root 464 0.0 0.0 0 0 ? S 18:05 0:00 \_ [scsi_eh_8] 2023-12-06T18:42:58.682 INFO:teuthology.orchestra.run.smithi152.stdout:root 465 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [scsi_tmf_8] 2023-12-06T18:42:58.682 INFO:teuthology.orchestra.run.smithi152.stdout:root 466 0.0 0.0 0 0 ? S 18:05 0:00 \_ [scsi_eh_9] 2023-12-06T18:42:58.682 INFO:teuthology.orchestra.run.smithi152.stdout:root 467 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [scsi_tmf_9] 2023-12-06T18:42:58.682 INFO:teuthology.orchestra.run.smithi152.stdout:root 468 0.0 0.0 0 0 ? I 18:05 0:00 \_ [kworker/u16:8-flush-8:0] 2023-12-06T18:42:58.682 INFO:teuthology.orchestra.run.smithi152.stdout:root 490 0.0 0.0 0 0 ? S 18:05 0:00 \_ [jbd2/sda1-8] 2023-12-06T18:42:58.682 INFO:teuthology.orchestra.run.smithi152.stdout:root 491 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [ext4-rsv-conver] 2023-12-06T18:42:58.682 INFO:teuthology.orchestra.run.smithi152.stdout:root 572 0.0 0.0 0 0 ? S 18:05 0:00 \_ [ext4lazyinit] 2023-12-06T18:42:58.682 INFO:teuthology.orchestra.run.smithi152.stdout:root 623 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [ipmi-msghandler] 2023-12-06T18:42:58.682 INFO:teuthology.orchestra.run.smithi152.stdout:root 633 0.0 0.0 0 0 ? SN 18:05 0:00 \_ [kipmi0] 2023-12-06T18:42:58.682 INFO:teuthology.orchestra.run.smithi152.stdout:root 665 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [rpciod] 2023-12-06T18:42:58.683 INFO:teuthology.orchestra.run.smithi152.stdout:root 666 0.0 0.0 0 0 ? I< 18:05 0:00 \_ [xprtiod] 2023-12-06T18:42:58.683 INFO:teuthology.orchestra.run.smithi152.stdout:root 1309 0.0 0.0 0 0 ? I 18:05 0:00 \_ [kworker/5:3-cgroup_destroy] 2023-12-06T18:42:58.683 INFO:teuthology.orchestra.run.smithi152.stdout:root 60485 0.0 0.0 0 0 ? I< 18:09 0:00 \_ [tls-strp] 2023-12-06T18:42:58.683 INFO:teuthology.orchestra.run.smithi152.stdout:root 63002 0.0 0.0 0 0 ? I 18:10 0:00 \_ [kworker/6:3-mm_percpu_wq] 2023-12-06T18:42:58.683 INFO:teuthology.orchestra.run.smithi152.stdout:root 66159 0.0 0.0 0 0 ? I 18:10 0:00 \_ [kworker/0:3-kacpi_notify] 2023-12-06T18:42:58.683 INFO:teuthology.orchestra.run.smithi152.stdout:root 69757 0.0 0.0 0 0 ? I< 18:10 0:00 \_ [kdmflush/253:0] 2023-12-06T18:42:58.683 INFO:teuthology.orchestra.run.smithi152.stdout:root 69876 0.0 0.0 0 0 ? I< 18:10 0:00 \_ [kdmflush/253:1] 2023-12-06T18:42:58.683 INFO:teuthology.orchestra.run.smithi152.stdout:root 69995 0.0 0.0 0 0 ? I< 18:10 0:00 \_ [kdmflush/253:2] 2023-12-06T18:42:58.683 INFO:teuthology.orchestra.run.smithi152.stdout:root 70114 0.0 0.0 0 0 ? I< 18:10 0:00 \_ [kdmflush/253:3] 2023-12-06T18:42:58.683 INFO:teuthology.orchestra.run.smithi152.stdout:root 70233 0.0 0.0 0 0 ? I< 18:10 0:00 \_ [kdmflush/253:4] 2023-12-06T18:42:58.683 INFO:teuthology.orchestra.run.smithi152.stdout:root 71118 0.0 0.0 0 0 ? I< 18:10 0:00 \_ [xfsalloc] 2023-12-06T18:42:58.683 INFO:teuthology.orchestra.run.smithi152.stdout:root 71119 0.0 0.0 0 0 ? I< 18:10 0:00 \_ [xfs_mru_cache] 2023-12-06T18:42:58.683 INFO:teuthology.orchestra.run.smithi152.stdout:root 71120 0.0 0.0 0 0 ? I< 18:10 0:00 \_ [xfs-buf/dm-4] 2023-12-06T18:42:58.683 INFO:teuthology.orchestra.run.smithi152.stdout:root 71121 0.0 0.0 0 0 ? I< 18:10 0:00 \_ [xfs-conv/dm-4] 2023-12-06T18:42:58.683 INFO:teuthology.orchestra.run.smithi152.stdout:root 71122 0.0 0.0 0 0 ? I< 18:10 0:00 \_ [xfs-reclaim/dm-] 2023-12-06T18:42:58.683 INFO:teuthology.orchestra.run.smithi152.stdout:root 71123 0.0 0.0 0 0 ? I< 18:10 0:00 \_ [xfs-blockgc/dm-] 2023-12-06T18:42:58.683 INFO:teuthology.orchestra.run.smithi152.stdout:root 71124 0.0 0.0 0 0 ? I< 18:10 0:00 \_ [xfs-inodegc/dm-] 2023-12-06T18:42:58.684 INFO:teuthology.orchestra.run.smithi152.stdout:root 71125 0.0 0.0 0 0 ? I< 18:10 0:00 \_ [xfs-log/dm-4] 2023-12-06T18:42:58.684 INFO:teuthology.orchestra.run.smithi152.stdout:root 71126 0.0 0.0 0 0 ? I< 18:10 0:00 \_ [xfs-cil/dm-4] 2023-12-06T18:42:58.684 INFO:teuthology.orchestra.run.smithi152.stdout:root 71127 0.0 0.0 0 0 ? S 18:10 0:00 \_ [xfsaild/dm-4] 2023-12-06T18:42:58.684 INFO:teuthology.orchestra.run.smithi152.stdout:root 75519 0.0 0.0 0 0 ? I 18:12 0:00 \_ [kworker/4:9-events] 2023-12-06T18:42:58.684 INFO:teuthology.orchestra.run.smithi152.stdout:root 76912 0.0 0.0 0 0 ? I 18:12 0:00 \_ [kworker/7:3-cgroup_destroy] 2023-12-06T18:42:58.684 INFO:teuthology.orchestra.run.smithi152.stdout:root 77847 0.0 0.0 0 0 ? I 18:12 0:00 \_ [kworker/3:254-events] 2023-12-06T18:42:58.684 INFO:teuthology.orchestra.run.smithi152.stdout:root 77848 0.1 0.0 0 0 ? I 18:12 0:02 \_ [kworker/3:255-events] 2023-12-06T18:42:58.684 INFO:teuthology.orchestra.run.smithi152.stdout:root 78519 0.0 0.0 0 0 ? I 18:12 0:00 \_ [kworker/1:117-mm_percpu_wq] 2023-12-06T18:42:58.684 INFO:teuthology.orchestra.run.smithi152.stdout:root 78768 0.0 0.0 0 0 ? I 18:12 0:00 \_ [kworker/2:255-cgwb_release] 2023-12-06T18:42:58.684 INFO:teuthology.orchestra.run.smithi152.stdout:root 80308 0.0 0.0 0 0 ? I 18:20 0:00 \_ [kworker/0:2-cgwb_release] 2023-12-06T18:42:58.684 INFO:teuthology.orchestra.run.smithi152.stdout:root 80431 0.0 0.0 0 0 ? I 18:20 0:00 \_ [kworker/u16:1-events_unbound] 2023-12-06T18:42:58.684 INFO:teuthology.orchestra.run.smithi152.stdout:root 81272 0.0 0.0 0 0 ? I< 18:25 0:00 \_ [xfs-buf/dm-2] 2023-12-06T18:42:58.684 INFO:teuthology.orchestra.run.smithi152.stdout:root 81273 0.0 0.0 0 0 ? I< 18:25 0:00 \_ [xfs-conv/dm-2] 2023-12-06T18:42:58.684 INFO:teuthology.orchestra.run.smithi152.stdout:root 81274 0.0 0.0 0 0 ? I< 18:25 0:00 \_ [xfs-reclaim/dm-] 2023-12-06T18:42:58.684 INFO:teuthology.orchestra.run.smithi152.stdout:root 81275 0.0 0.0 0 0 ? I< 18:25 0:00 \_ [xfs-blockgc/dm-] 2023-12-06T18:42:58.684 INFO:teuthology.orchestra.run.smithi152.stdout:root 81276 0.0 0.0 0 0 ? I< 18:25 0:00 \_ [xfs-inodegc/dm-] 2023-12-06T18:42:58.684 INFO:teuthology.orchestra.run.smithi152.stdout:root 81277 0.0 0.0 0 0 ? I< 18:25 0:00 \_ [xfs-log/dm-2] 2023-12-06T18:42:58.685 INFO:teuthology.orchestra.run.smithi152.stdout:root 81278 0.0 0.0 0 0 ? I< 18:25 0:00 \_ [xfs-cil/dm-2] 2023-12-06T18:42:58.685 INFO:teuthology.orchestra.run.smithi152.stdout:root 81279 0.0 0.0 0 0 ? S 18:25 0:00 \_ [xfsaild/dm-2] 2023-12-06T18:42:58.685 INFO:teuthology.orchestra.run.smithi152.stdout:root 81356 0.0 0.0 0 0 ? I< 18:25 0:00 \_ [xfs-buf/dm-3] 2023-12-06T18:42:58.685 INFO:teuthology.orchestra.run.smithi152.stdout:root 81357 0.0 0.0 0 0 ? I< 18:25 0:00 \_ [xfs-conv/dm-3] 2023-12-06T18:42:58.685 INFO:teuthology.orchestra.run.smithi152.stdout:root 81358 0.0 0.0 0 0 ? I< 18:25 0:00 \_ [xfs-reclaim/dm-] 2023-12-06T18:42:58.685 INFO:teuthology.orchestra.run.smithi152.stdout:root 81359 0.0 0.0 0 0 ? I< 18:25 0:00 \_ [xfs-blockgc/dm-] 2023-12-06T18:42:58.685 INFO:teuthology.orchestra.run.smithi152.stdout:root 81360 0.0 0.0 0 0 ? I< 18:25 0:00 \_ [xfs-inodegc/dm-] 2023-12-06T18:42:58.685 INFO:teuthology.orchestra.run.smithi152.stdout:root 81361 0.0 0.0 0 0 ? I< 18:25 0:00 \_ [xfs-log/dm-3] 2023-12-06T18:42:58.685 INFO:teuthology.orchestra.run.smithi152.stdout:root 81362 0.0 0.0 0 0 ? I< 18:25 0:00 \_ [xfs-cil/dm-3] 2023-12-06T18:42:58.685 INFO:teuthology.orchestra.run.smithi152.stdout:root 81363 0.0 0.0 0 0 ? S 18:25 0:00 \_ [xfsaild/dm-3] 2023-12-06T18:42:58.685 INFO:teuthology.orchestra.run.smithi152.stdout:root 83757 0.0 0.0 0 0 ? I< 18:25 0:00 \_ [dio/dm-2] 2023-12-06T18:42:58.685 INFO:teuthology.orchestra.run.smithi152.stdout:root 84930 0.0 0.0 0 0 ? I< 18:25 0:00 \_ [dio/dm-3] 2023-12-06T18:42:58.685 INFO:teuthology.orchestra.run.smithi152.stdout:root 87264 0.0 0.0 0 0 ? I 18:26 0:00 \_ [kworker/1:0-cgroup_destroy] 2023-12-06T18:42:58.685 INFO:teuthology.orchestra.run.smithi152.stdout:root 87291 0.0 0.0 0 0 ? I 18:26 0:00 \_ [kworker/2:0-events] 2023-12-06T18:42:58.685 INFO:teuthology.orchestra.run.smithi152.stdout:root 90476 0.0 0.0 0 0 ? I 18:27 0:00 \_ [kworker/u16:2-writeback] 2023-12-06T18:42:58.685 INFO:teuthology.orchestra.run.smithi152.stdout:root 98985 0.0 0.0 0 0 ? I 18:29 0:00 \_ [kworker/7:1-mm_percpu_wq] 2023-12-06T18:42:58.686 INFO:teuthology.orchestra.run.smithi152.stdout:root 103036 0.0 0.0 0 0 ? I 18:30 0:00 \_ [kworker/u16:3-flush-8:0] 2023-12-06T18:42:58.686 INFO:teuthology.orchestra.run.smithi152.stdout:root 109816 0.0 0.0 0 0 ? I 18:32 0:00 \_ [kworker/6:1-xfs-sync/dm-1] 2023-12-06T18:42:58.686 INFO:teuthology.orchestra.run.smithi152.stdout:root 116136 0.0 0.0 0 0 ? I 18:35 0:00 \_ [kworker/4:1-mm_percpu_wq] 2023-12-06T18:42:58.686 INFO:teuthology.orchestra.run.smithi152.stdout:root 116137 0.0 0.0 0 0 ? I 18:38 0:00 \_ [kworker/6:0-mm_percpu_wq] 2023-12-06T18:42:58.686 INFO:teuthology.orchestra.run.smithi152.stdout:root 116139 0.0 0.0 0 0 ? I 18:40 0:00 \_ [kworker/2:1-events] 2023-12-06T18:42:58.686 INFO:teuthology.orchestra.run.smithi152.stdout:root 116140 0.0 0.0 0 0 ? I 18:41 0:00 \_ [kworker/7:0-mm_percpu_wq] 2023-12-06T18:42:58.686 INFO:teuthology.orchestra.run.smithi152.stdout:root 116177 0.0 0.0 0 0 ? I 18:42 0:00 \_ [kworker/u16:0-flush-8:0] 2023-12-06T18:42:58.686 INFO:teuthology.orchestra.run.smithi152.stdout:root 1 0.1 0.0 175176 19200 ? Ss 18:05 0:03 /usr/lib/systemd/systemd --switched-root --system --deserialize 31 2023-12-06T18:42:58.686 INFO:teuthology.orchestra.run.smithi152.stdout:root 566 0.0 0.0 26668 16512 ? Ss 18:05 0:01 /usr/lib/systemd/systemd-journald 2023-12-06T18:42:58.686 INFO:teuthology.orchestra.run.smithi152.stdout:root 581 0.0 0.0 37676 15520 ? Ss 18:05 0:00 /usr/lib/systemd/systemd-udevd 2023-12-06T18:42:58.686 INFO:teuthology.orchestra.run.smithi152.stdout:rpc 649 0.0 0.0 13244 5376 ? Ss 18:05 0:00 /usr/bin/rpcbind -w -f 2023-12-06T18:42:58.686 INFO:teuthology.orchestra.run.smithi152.stdout:root 659 0.0 0.0 91916 4352 ? S, line 50) 2023-12-06T18:42:59.227 ERROR:teuthology.util.scanner:Premature end of data in tag valgrindoutput line 3, line 50, column 1 (, line 50) 2023-12-06T18:42:59.273 ERROR:teuthology.util.scanner:Premature end of data in tag valgrindoutput line 3, line 50, column 1 (, line 50) 2023-12-06T18:42:59.276 INFO:tasks.ceph:Archiving crash dumps... 2023-12-06T18:42:59.278 DEBUG:teuthology.misc:Transferring archived files from smithi061:/var/lib/ceph/crash to /home/teuthworker/archive/sseshasa-2023-12-06_17:50:03-rados-wip-sseshasa-testing-2023-12-04-1848-distro-default-smithi/7480579/remote/smithi061/crash 2023-12-06T18:42:59.279 DEBUG:teuthology.orchestra.run.smithi061:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-06T18:42:59.328 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/crash to /home/teuthworker/archive/sseshasa-2023-12-06_17:50:03-rados-wip-sseshasa-testing-2023-12-04-1848-distro-default-smithi/7480579/remote/smithi152/crash 2023-12-06T18:42:59.329 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-06T18:42:59.447 INFO:tasks.ceph:Compressing logs... 2023-12-06T18:42:59.448 DEBUG:teuthology.orchestra.run.smithi061:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-06T18:42:59.451 DEBUG:teuthology.orchestra.run.smithi152:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-06T18:42:59.481 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-12-06T18:42:59.481 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86718.log 2023-12-06T18:42:59.481 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.7.log 2023-12-06T18:42:59.481 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.86718.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.5.log 2023-12-06T18:42:59.481 INFO:teuthology.orchestra.run.smithi152.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86718.log.gz 2023-12-06T18:42:59.481 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/valgrind/osd.7.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.6.log 2023-12-06T18:42:59.482 INFO:teuthology.orchestra.run.smithi152.stderr: 55.0% -- replaced with /var/log/ceph/valgrind/osd.7.log.gz 2023-12-06T18:42:59.482 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/valgrind/osd.5.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.b.log 2023-12-06T18:42:59.482 INFO:teuthology.orchestra.run.smithi152.stderr: 55.1% -- replaced with /var/log/ceph/valgrind/osd.5.log.gz 2023-12-06T18:42:59.482 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/valgrind/osd.6.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.4.log 2023-12-06T18:42:59.482 INFO:teuthology.orchestra.run.smithi152.stderr: 55.0% -- replaced with /var/log/ceph/valgrind/osd.6.log.gz 2023-12-06T18:42:59.482 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/valgrind/mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-12-06T18:42:59.482 INFO:teuthology.orchestra.run.smithi152.stderr: 74.7% -- replaced with /var/log/ceph/valgrind/mon.b.log.gz 2023-12-06T18:42:59.482 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/valgrind/osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-12-06T18:42:59.482 INFO:teuthology.orchestra.run.smithi152.stderr: 63.9% -- replaced with /var/log/ceph/valgrind/osd.4.log.gz 2023-12-06T18:42:59.482 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86854.log 2023-12-06T18:42:59.482 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86894.log 2023-12-06T18:42:59.482 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.86854.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-12-06T18:42:59.482 INFO:teuthology.orchestra.run.smithi152.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86854.log.gz 2023-12-06T18:42:59.483 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.86894.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-12-06T18:42:59.483 INFO:teuthology.orchestra.run.smithi152.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86894.log.gz 2023-12-06T18:42:59.483 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86934.log 2023-12-06T18:42:59.483 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.93707.log 2023-12-06T18:42:59.484 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.86934.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.95413.log 2023-12-06T18:42:59.484 INFO:teuthology.orchestra.run.smithi152.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86934.log.gz 2023-12-06T18:42:59.484 INFO:teuthology.orchestra.run.smithi152.stderr: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-12-06T18:42:59.509 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/objectstore_tool.93707.log: 94.7% -- replaced with /var/log/ceph/objectstore_tool.93707.log.gz 2023-12-06T18:42:59.509 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86814.log 2023-12-06T18:42:59.510 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/objectstore_tool.95413.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.94490.log 2023-12-06T18:42:59.510 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.86814.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-12-06T18:42:59.510 INFO:teuthology.orchestra.run.smithi152.stderr: -- replaced with /var/log/ceph/ceph-client.admin.86814.log.gz 2023-12-06T18:42:59.510 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-12-06T18:42:59.510 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/objectstore_tool.94490.log: /var/log/ceph/ceph-mgr.x.log: 94.7% -- replaced with /var/log/ceph/objectstore_tool.95413.log.gz 2023-12-06T18:42:59.510 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86774.log 2023-12-06T18:42:59.510 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph.log: 94.7% -- replaced with /var/log/ceph/objectstore_tool.94490.log.gz 2023-12-06T18:42:59.510 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-12-06T18:42:59.510 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.86774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86774.log.gz 2023-12-06T18:42:59.510 INFO:teuthology.orchestra.run.smithi152.stderr: 89.3% -- replaced with /var/log/ceph/ceph.log.gz 2023-12-06T18:42:59.510 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.96176.log 2023-12-06T18:42:59.510 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86974.log 2023-12-06T18:42:59.510 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/objectstore_tool.96176.log: /var/log/ceph/ceph-mon.b.log: 94.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-12-06T18:42:59.510 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87014.log 2023-12-06T18:42:59.511 INFO:teuthology.orchestra.run.smithi152.stderr: 94.6% -- replaced with /var/log/ceph/objectstore_tool.96176.log.gz/var/log/ceph/ceph-client.admin.86974.log: 2023-12-06T18:42:59.511 INFO:teuthology.orchestra.run.smithi152.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86974.log.gz 2023-12-06T18:42:59.511 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.87014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87014.log.gz 2023-12-06T18:42:59.614 INFO:teuthology.orchestra.run.smithi152.stderr:gzip: /var/log/ceph/ceph-osd.7.log: file size changed while zipping 2023-12-06T18:42:59.614 INFO:teuthology.orchestra.run.smithi152.stderr: 90.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-12-06T18:42:59.661 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131589.log 2023-12-06T18:42:59.661 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132892.log 2023-12-06T18:42:59.661 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119401.log 2023-12-06T18:42:59.662 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131589.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131589.log.gz 2023-12-06T18:42:59.662 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.132892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91398.log 2023-12-06T18:42:59.662 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132892.log.gz 2023-12-06T18:42:59.662 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.119401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125123.log 2023-12-06T18:42:59.662 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119401.log.gz 2023-12-06T18:42:59.662 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133648.log 2023-12-06T18:42:59.662 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91398.log.gz 2023-12-06T18:42:59.662 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.125123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131379.log 2023-12-06T18:42:59.662 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125123.log.gz 2023-12-06T18:42:59.662 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132297.log 2023-12-06T18:42:59.662 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133648.log.gz 2023-12-06T18:42:59.662 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107996.log 2023-12-06T18:42:59.662 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131379.log.gz 2023-12-06T18:42:59.663 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.132297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108994.log 2023-12-06T18:42:59.663 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132297.log.gz 2023-12-06T18:42:59.663 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.107996.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.94048.log -- replaced with /var/log/ceph/ceph-client.admin.107996.log.gz 2023-12-06T18:42:59.663 INFO:teuthology.orchestra.run.smithi061.stderr: 2023-12-06T18:42:59.663 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.108994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101561.log 2023-12-06T18:42:59.663 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108994.log.gz 2023-12-06T18:42:59.663 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.94048.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-12-06T18:42:59.663 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94048.log.gz 2023-12-06T18:42:59.663 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.101561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121155.log 2023-12-06T18:42:59.663 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101561.log.gz 2023-12-06T18:42:59.663 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130808.log 2023-12-06T18:42:59.663 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.121155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109121.log 2023-12-06T18:42:59.663 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121155.log.gz 2023-12-06T18:42:59.663 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.130808.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130808.log.gz 2023-12-06T18:42:59.663 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104421.log 2023-12-06T18:42:59.664 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.109121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129986.log 2023-12-06T18:42:59.664 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109121.log.gz 2023-12-06T18:42:59.664 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.104421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91083.log 2023-12-06T18:42:59.664 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104421.log.gz 2023-12-06T18:42:59.664 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.129986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92129.log 2023-12-06T18:42:59.664 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129986.log.gz 2023-12-06T18:42:59.664 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131253.log 2023-12-06T18:42:59.664 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91083.log.gz 2023-12-06T18:42:59.664 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92129.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.92129.log.gz 2023-12-06T18:42:59.664 INFO:teuthology.orchestra.run.smithi061.stderr: /var/log/ceph/ceph-client.admin.123769.log 2023-12-06T18:42:59.664 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130307.log 2023-12-06T18:42:59.664 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131253.log.gz 2023-12-06T18:42:59.664 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.123769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120872.log 2023-12-06T18:42:59.664 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123769.log.gz 2023-12-06T18:42:59.664 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.130307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115419.log 2023-12-06T18:42:59.665 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130307.log.gz 2023-12-06T18:42:59.665 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.120872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120872.log.gz 2023-12-06T18:42:59.665 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91387.log 2023-12-06T18:42:59.665 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.115419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102263.log 2023-12-06T18:42:59.665 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115419.log.gz 2023-12-06T18:42:59.665 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114805.log 2023-12-06T18:42:59.665 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91387.log.gz 2023-12-06T18:42:59.665 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.102263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133018.log 2023-12-06T18:42:59.665 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102263.log.gz 2023-12-06T18:42:59.665 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.114805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93891.log 2023-12-06T18:42:59.665 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114805.log.gz 2023-12-06T18:42:59.665 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133018.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133018.log.gz 2023-12-06T18:42:59.665 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95434.log 2023-12-06T18:42:59.665 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.93891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103151.log 2023-12-06T18:42:59.666 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93891.log.gz 2023-12-06T18:42:59.666 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.95434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123197.log 2023-12-06T18:42:59.666 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95434.log.gz 2023-12-06T18:42:59.666 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.103151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90979.log 2023-12-06T18:42:59.666 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103151.log.gz 2023-12-06T18:42:59.666 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.123197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130048.log 2023-12-06T18:42:59.666 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123197.log.gz 2023-12-06T18:42:59.666 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.90979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124224.log 2023-12-06T18:42:59.666 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90979.log.gz 2023-12-06T18:42:59.666 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.130048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133522.log 2023-12-06T18:42:59.667 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130048.log.gz 2023-12-06T18:42:59.667 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.124224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129102.log 2023-12-06T18:42:59.667 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124224.log.gz 2023-12-06T18:42:59.667 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133522.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133522.log.gz 2023-12-06T18:42:59.667 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87315.log 2023-12-06T18:42:59.667 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.129102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121755.log 2023-12-06T18:42:59.667 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129102.log.gz 2023-12-06T18:42:59.667 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.87315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114363.log 2023-12-06T18:42:59.667 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87315.log.gz 2023-12-06T18:42:59.667 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.121755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114000.log 2023-12-06T18:42:59.667 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121755.log.gz 2023-12-06T18:42:59.667 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.114363.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114363.log.gz 2023-12-06T18:42:59.667 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93531.log 2023-12-06T18:42:59.667 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.114000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133270.log 2023-12-06T18:42:59.667 INFO:teuthology.orchestra.run.smithi061.stderr: 29.5% -- replaced with /var/log/ceph/ceph-client.admin.114000.log.gz 2023-12-06T18:42:59.668 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.93531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121881.log 2023-12-06T18:42:59.668 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93531.log.gz 2023-12-06T18:42:59.668 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120079.log 2023-12-06T18:42:59.668 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133270.log.gz 2023-12-06T18:42:59.668 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.121881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127608.log 2023-12-06T18:42:59.668 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121881.log.gz 2023-12-06T18:42:59.668 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.120079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95126.log 2023-12-06T18:42:59.668 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120079.log.gz 2023-12-06T18:42:59.668 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.127608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91813.log 2023-12-06T18:42:59.668 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127608.log.gz 2023-12-06T18:42:59.668 INFO:teuthology.orchestra.run.smithi061.stderr:gzip/var/log/ceph/ceph-client.admin.95126.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.126940.log 2023-12-06T18:42:59.668 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95126.log.gz 2023-12-06T18:42:59.668 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115480.log 2023-12-06T18:42:59.668 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91813.log.gz 2023-12-06T18:42:59.669 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.126940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119101.log 2023-12-06T18:42:59.669 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126940.log.gz 2023-12-06T18:42:59.669 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.115480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131932.log 2023-12-06T18:42:59.669 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115480.log.gz 2023-12-06T18:42:59.669 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.119101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131934.log 2023-12-06T18:42:59.669 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119101.log.gz 2023-12-06T18:42:59.669 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122742.log 2023-12-06T18:42:59.669 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131932.log.gz 2023-12-06T18:42:59.669 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91399.log 2023-12-06T18:42:59.669 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131934.log.gz 2023-12-06T18:42:59.669 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.122742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108396.log 2023-12-06T18:42:59.669 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122742.log.gz 2023-12-06T18:42:59.669 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131935.log 2023-12-06T18:42:59.669 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91399.log.gz 2023-12-06T18:42:59.670 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.108396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101448.log 2023-12-06T18:42:59.670 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108396.log.gz 2023-12-06T18:42:59.670 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131935.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.93596.log -- replaced with /var/log/ceph/ceph-client.admin.131935.log.gz 2023-12-06T18:42:59.670 INFO:teuthology.orchestra.run.smithi061.stderr: 2023-12-06T18:42:59.670 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.101448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119694.log 2023-12-06T18:42:59.670 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101448.log.gz 2023-12-06T18:42:59.670 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.93596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128157.log 2023-12-06T18:42:59.670 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93596.log.gz 2023-12-06T18:42:59.670 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.119694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91674.log 2023-12-06T18:42:59.670 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119694.log.gz 2023-12-06T18:42:59.670 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.128157.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.c.log 2023-12-06T18:42:59.670 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128157.log.gz 2023-12-06T18:42:59.670 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91674.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.2.log 2023-12-06T18:42:59.670 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91674.log.gz 2023-12-06T18:42:59.671 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/valgrind/mon.c.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.0.log 2023-12-06T18:42:59.671 INFO:teuthology.orchestra.run.smithi061.stderr: 74.7% -- replaced with /var/log/ceph/valgrind/mon.c.log.gz 2023-12-06T18:42:59.671 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/valgrind/osd.2.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.1.log 2023-12-06T18:42:59.671 INFO:teuthology.orchestra.run.smithi061.stderr: 64.1% -- replaced with /var/log/ceph/valgrind/osd.2.log.gz 2023-12-06T18:42:59.671 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/valgrind/osd.0.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.a.log 2023-12-06T18:42:59.671 INFO:teuthology.orchestra.run.smithi061.stderr: 64.0% -- replaced with /var/log/ceph/valgrind/osd.0.log.gz 2023-12-06T18:42:59.671 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/valgrind/osd.1.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.3.log 2023-12-06T18:42:59.671 INFO:teuthology.orchestra.run.smithi061.stderr: 64.0% -- replaced with /var/log/ceph/valgrind/osd.1.log.gz 2023-12-06T18:42:59.671 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/valgrind/mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92287.log 2023-12-06T18:42:59.671 INFO:teuthology.orchestra.run.smithi061.stderr: 74.7% -- replaced with /var/log/ceph/valgrind/mon.a.log.gz 2023-12-06T18:42:59.671 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/valgrind/osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92437.log 2023-12-06T18:42:59.671 INFO:teuthology.orchestra.run.smithi061.stderr: 64.0% -- replaced with /var/log/ceph/valgrind/osd.3.log.gz 2023-12-06T18:42:59.671 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122319.log 2023-12-06T18:42:59.671 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92287.log.gz 2023-12-06T18:42:59.671 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114458.log 2023-12-06T18:42:59.672 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92437.log.gz 2023-12-06T18:42:59.672 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.122319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109856.log 2023-12-06T18:42:59.672 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122319.log.gz 2023-12-06T18:42:59.672 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.114458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99583.log 2023-12-06T18:42:59.672 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114458.log.gz 2023-12-06T18:42:59.672 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.109856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132806.log 2023-12-06T18:42:59.672 INFO:teuthology.orchestra.run.smithi061.stderr: 28.5% -- replaced with /var/log/ceph/ceph-client.admin.109856.log.gz 2023-12-06T18:42:59.672 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.99583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91602.log 2023-12-06T18:42:59.672 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99583.log.gz 2023-12-06T18:42:59.672 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.132806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104151.log 2023-12-06T18:42:59.672 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132806.log.gz 2023-12-06T18:42:59.672 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90917.log 2023-12-06T18:42:59.672 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91602.log.gz 2023-12-06T18:42:59.672 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.104151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104956.log 2023-12-06T18:42:59.672 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104151.log.gz 2023-12-06T18:42:59.672 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.90917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124040.log 2023-12-06T18:42:59.673 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90917.log.gz 2023-12-06T18:42:59.673 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.104956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126186.log 2023-12-06T18:42:59.673 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104956.log.gz 2023-12-06T18:42:59.673 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.124040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97832.log 2023-12-06T18:42:59.673 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124040.log.gz 2023-12-06T18:42:59.673 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.126186.log: /var/log/ceph/ceph-client.admin.116368.log 2023-12-06T18:42:59.673 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126186.log.gz 2023-12-06T18:42:59.673 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.97832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98878.log 2023-12-06T18:42:59.673 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97832.log.gz 2023-12-06T18:42:59.673 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.116368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127200.log 2023-12-06T18:42:59.673 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116368.log.gz 2023-12-06T18:42:59.673 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.98878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124162.log 2023-12-06T18:42:59.673 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98878.log.gz 2023-12-06T18:42:59.673 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.127200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133774.log 2023-12-06T18:42:59.674 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127200.log.gz 2023-12-06T18:42:59.674 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.124162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133354.log 2023-12-06T18:42:59.674 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124162.log.gz 2023-12-06T18:42:59.674 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133774.log.gz 2023-12-06T18:42:59.674 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100786.log 2023-12-06T18:42:59.674 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116876.log 2023-12-06T18:42:59.674 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133354.log.gz 2023-12-06T18:42:59.674 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.100786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99157.log 2023-12-06T18:42:59.674 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100786.log.gz 2023-12-06T18:42:59.674 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.116876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121024.log 2023-12-06T18:42:59.674 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116876.log.gz 2023-12-06T18:42:59.674 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.99157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115297.log 2023-12-06T18:42:59.674 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99157.log.gz 2023-12-06T18:42:59.674 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.121024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116920.log 2023-12-06T18:42:59.675 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121024.log.gz 2023-12-06T18:42:59.675 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.115297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102782.log 2023-12-06T18:42:59.675 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115297.log.gz 2023-12-06T18:42:59.675 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.116920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97212.log 2023-12-06T18:42:59.675 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116920.log.gz 2023-12-06T18:42:59.675 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.102782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120455.log 2023-12-06T18:42:59.675 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102782.log.gz 2023-12-06T18:42:59.675 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.97212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132227.log 2023-12-06T18:42:59.675 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97212.log.gz 2023-12-06T18:42:59.675 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.120455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121839.log 2023-12-06T18:42:59.675 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120455.log.gz 2023-12-06T18:42:59.675 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.132227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106651.log 2023-12-06T18:42:59.675 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132227.log.gz 2023-12-06T18:42:59.675 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.121839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104197.log 2023-12-06T18:42:59.675 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121839.log.gz 2023-12-06T18:42:59.676 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.106651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102066.log 2023-12-06T18:42:59.676 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.106651.log.gz 2023-12-06T18:42:59.676 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.104197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129044.log 2023-12-06T18:42:59.676 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104197.log.gz 2023-12-06T18:42:59.676 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.102066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124547.log 2023-12-06T18:42:59.676 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102066.log.gz 2023-12-06T18:42:59.676 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.129044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130868.log 2023-12-06T18:42:59.676 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129044.log.gz 2023-12-06T18:42:59.676 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.124547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131922.log 2023-12-06T18:42:59.676 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124547.log.gz 2023-12-06T18:42:59.676 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.130868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92069.log 2023-12-06T18:42:59.676 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130868.log.gz 2023-12-06T18:42:59.676 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118312.log 2023-12-06T18:42:59.676 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131922.log.gz 2023-12-06T18:42:59.676 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92689.log 2023-12-06T18:42:59.676 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92069.log.gz 2023-12-06T18:42:59.677 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.118312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113745.log 2023-12-06T18:42:59.677 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118312.log.gz 2023-12-06T18:42:59.677 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98552.log 2023-12-06T18:42:59.677 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92689.log.gz 2023-12-06T18:42:59.677 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.113745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124952.log 2023-12-06T18:42:59.677 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113745.log.gz 2023-12-06T18:42:59.677 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.98552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114881.log 2023-12-06T18:42:59.677 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98552.log.gz 2023-12-06T18:42:59.677 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.124952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113617.log 2023-12-06T18:42:59.677 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124952.log.gz 2023-12-06T18:42:59.677 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.114881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133228.log 2023-12-06T18:42:59.677 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114881.log.gz 2023-12-06T18:42:59.677 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.113617.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113617.log.gz 2023-12-06T18:42:59.677 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100484.log 2023-12-06T18:42:59.677 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99770.log 2023-12-06T18:42:59.678 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133228.log.gz 2023-12-06T18:42:59.678 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.100484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120932.log 2023-12-06T18:42:59.678 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100484.log.gz 2023-12-06T18:42:59.678 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.99770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110038.log 2023-12-06T18:42:59.678 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99770.log.gz 2023-12-06T18:42:59.678 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.120932.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.120932.log.gz -5 2023-12-06T18:42:59.678 INFO:teuthology.orchestra.run.smithi061.stderr: --verbose -- /var/log/ceph/ceph-client.admin.132934.log 2023-12-06T18:42:59.678 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.110038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109407.log 2023-12-06T18:42:59.678 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.110038.log.gz 2023-12-06T18:42:59.678 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.132934.log: /var/log/ceph/ceph-client.admin.119725.log 2023-12-06T18:42:59.678 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132934.log.gz 2023-12-06T18:42:59.678 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.109407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133940.log 2023-12-06T18:42:59.678 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109407.log.gz 2023-12-06T18:42:59.678 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.119725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121363.log 2023-12-06T18:42:59.679 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119725.log.gz 2023-12-06T18:42:59.679 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128354.log 2023-12-06T18:42:59.679 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133940.log.gz 2023-12-06T18:42:59.679 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.121363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132471.log 2023-12-06T18:42:59.679 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121363.log.gz 2023-12-06T18:42:59.679 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.128354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129653.log 2023-12-06T18:42:59.679 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128354.log.gz 2023-12-06T18:42:59.679 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.132471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117950.log 2023-12-06T18:42:59.679 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132471.log.gz 2023-12-06T18:42:59.679 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.129653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113658.log 2023-12-06T18:42:59.679 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129653.log.gz 2023-12-06T18:42:59.679 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.117950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132731.log 2023-12-06T18:42:59.679 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117950.log.gz 2023-12-06T18:42:59.679 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.113658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109543.log 2023-12-06T18:42:59.679 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.113658.log.gz 2023-12-06T18:42:59.680 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.132731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133480.log 2023-12-06T18:42:59.680 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132731.log.gz 2023-12-06T18:42:59.680 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.109543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108633.log 0.0% 2023-12-06T18:42:59.680 INFO:teuthology.orchestra.run.smithi061.stderr: -- replaced with /var/log/ceph/ceph-client.admin.109543.log.gz 2023-12-06T18:42:59.680 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96449.log 2023-12-06T18:42:59.680 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133480.log.gz 2023-12-06T18:42:59.680 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.108633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104720.log 2023-12-06T18:42:59.680 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108633.log.gz 2023-12-06T18:42:59.680 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.96449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89214.log 2023-12-06T18:42:59.680 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96449.log.gz 2023-12-06T18:42:59.680 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.104720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121415.log 2023-12-06T18:42:59.680 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104720.log.gz 2023-12-06T18:42:59.680 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.89214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133606.log 2023-12-06T18:42:59.680 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.89214.log.gz 2023-12-06T18:42:59.680 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.121415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92377.log 2023-12-06T18:42:59.681 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121415.log.gz 2023-12-06T18:42:59.681 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117300.log 2023-12-06T18:42:59.681 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133606.log.gz 2023-12-06T18:42:59.681 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118415.log 2023-12-06T18:42:59.681 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92377.log.gz 2023-12-06T18:42:59.681 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.117300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99973.log 2023-12-06T18:42:59.681 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117300.log.gz 2023-12-06T18:42:59.681 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.118415.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118415.log.gz 2023-12-06T18:42:59.681 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131757.log 2023-12-06T18:42:59.681 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.99973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133732.log 2023-12-06T18:42:59.681 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99973.log.gz 2023-12-06T18:42:59.681 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108934.log 2023-12-06T18:42:59.681 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131757.log.gz 2023-12-06T18:42:59.681 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95524.log 2023-12-06T18:42:59.681 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133732.log.gz 2023-12-06T18:42:59.682 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.108934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91616.log 2023-12-06T18:42:59.682 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108934.log.gz 2023-12-06T18:42:59.682 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.95524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126919.log 2023-12-06T18:42:59.682 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95524.log.gz 2023-12-06T18:42:59.682 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121795.log 2023-12-06T18:42:59.682 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91616.log.gz 2023-12-06T18:42:59.682 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.126919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91397.log 2023-12-06T18:42:59.682 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126919.log.gz 2023-12-06T18:42:59.682 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.121795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119145.log 2023-12-06T18:42:59.682 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121795.log.gz 2023-12-06T18:42:59.682 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101727.log 2023-12-06T18:42:59.682 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91397.log.gz 2023-12-06T18:42:59.682 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.119145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103340.log 2023-12-06T18:42:59.682 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119145.log.gz 2023-12-06T18:42:59.683 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.101727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124609.log 2023-12-06T18:42:59.683 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101727.log.gz 2023-12-06T18:42:59.683 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.103340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127588.log 2023-12-06T18:42:59.683 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103340.log.gz 2023-12-06T18:42:59.683 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.124609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119917.log 2023-12-06T18:42:59.683 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124609.log.gz 2023-12-06T18:42:59.683 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.127588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128031.log 2023-12-06T18:42:59.683 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127588.log.gz 2023-12-06T18:42:59.683 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.119917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120149.log 2023-12-06T18:42:59.683 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119917.log.gz 2023-12-06T18:42:59.683 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.128031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127405.log 2023-12-06T18:42:59.683 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128031.log.gz 2023-12-06T18:42:59.683 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.120149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92645.log 2023-12-06T18:42:59.683 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120149.log.gz 2023-12-06T18:42:59.683 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.127405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126399.log 2023-12-06T18:42:59.683 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127405.log.gz 2023-12-06T18:42:59.684 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133312.log 2023-12-06T18:42:59.684 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92645.log.gz 2023-12-06T18:42:59.684 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.126399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101881.log 2023-12-06T18:42:59.684 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126399.log.gz 2023-12-06T18:42:59.684 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133312.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133312.log.gz 2023-12-06T18:42:59.684 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121598.log 2023-12-06T18:42:59.684 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.101881.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-12-06T18:42:59.684 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101881.log.gz 2023-12-06T18:42:59.684 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.121598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124082.log 2023-12-06T18:42:59.684 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121598.log.gz 2023-12-06T18:42:59.684 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128737.log 2023-12-06T18:42:59.684 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.124082.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.124082.log.gz /var/log/ceph/ceph-client.admin.93173.log 2023-12-06T18:42:59.684 INFO:teuthology.orchestra.run.smithi061.stderr: 2023-12-06T18:42:59.684 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.128737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116075.log 2023-12-06T18:42:59.685 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128737.log.gz 2023-12-06T18:42:59.685 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.93173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104102.log 2023-12-06T18:42:59.685 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93173.log.gz 2023-12-06T18:42:59.685 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.116075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96388.log 2023-12-06T18:42:59.685 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116075.log.gz 2023-12-06T18:42:59.685 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.104102.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.104102.log.gz /var/log/ceph/ceph-client.admin.103763.log 2023-12-06T18:42:59.685 INFO:teuthology.orchestra.run.smithi061.stderr: 2023-12-06T18:42:59.685 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.96388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123080.log 2023-12-06T18:42:59.685 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96388.log.gz 2023-12-06T18:42:59.685 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.103763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132149.log 0.0% 2023-12-06T18:42:59.685 INFO:teuthology.orchestra.run.smithi061.stderr: -- replaced with /var/log/ceph/ceph-client.admin.103763.log.gz 2023-12-06T18:42:59.685 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.123080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99016.log 2023-12-06T18:42:59.685 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123080.log.gz 2023-12-06T18:42:59.685 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.132149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93447.log 2023-12-06T18:42:59.685 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132149.log.gz 2023-12-06T18:42:59.686 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.99016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129490.log 2023-12-06T18:42:59.686 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99016.log.gz 2023-12-06T18:42:59.686 INFO:teuthology.orchestra.run.smithi061.stderr: 93.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-12-06T18:42:59.686 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.93447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124608.log 2023-12-06T18:42:59.686 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93447.log.gz 2023-12-06T18:42:59.686 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.129490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125195.log 2023-12-06T18:42:59.686 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129490.log.gz 2023-12-06T18:42:59.686 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.124608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133564.log 2023-12-06T18:42:59.686 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124608.log.gz 2023-12-06T18:42:59.686 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.125195.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125195.log.gz -5 2023-12-06T18:42:59.686 INFO:teuthology.orchestra.run.smithi061.stderr: --verbose -- /var/log/ceph/ceph-client.admin.127792.log 2023-12-06T18:42:59.686 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118514.log 2023-12-06T18:42:59.686 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133564.log.gz 2023-12-06T18:42:59.686 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.127792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127445.log 2023-12-06T18:42:59.687 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127792.log.gz 2023-12-06T18:42:59.687 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.118514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97097.log 2023-12-06T18:42:59.687 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118514.log.gz 2023-12-06T18:42:59.687 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.127445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95761.log 2023-12-06T18:42:59.687 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127445.log.gz 2023-12-06T18:42:59.687 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.97097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115012.log 2023-12-06T18:42:59.687 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97097.log.gz 2023-12-06T18:42:59.687 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.95761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117077.log 2023-12-06T18:42:59.687 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95761.log.gz 2023-12-06T18:42:59.687 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.115012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117990.log 2023-12-06T18:42:59.687 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115012.log.gz 2023-12-06T18:42:59.687 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.117077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114087.log 2023-12-06T18:42:59.687 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117077.log.gz 2023-12-06T18:42:59.687 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.117990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131927.log 2023-12-06T18:42:59.688 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117990.log.gz 2023-12-06T18:42:59.688 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.114087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127521.log 2023-12-06T18:42:59.688 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114087.log.gz 2023-12-06T18:42:59.688 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108478.log 2023-12-06T18:42:59.688 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131927.log.gz 2023-12-06T18:42:59.688 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.127521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131211.log 2023-12-06T18:42:59.688 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127521.log.gz 2023-12-06T18:42:59.688 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.108478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109188.log 2023-12-06T18:42:59.688 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108478.log.gz 2023-12-06T18:42:59.688 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123399.log 2023-12-06T18:42:59.688 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131211.log.gz 2023-12-06T18:42:59.688 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.109188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115099.log 2023-12-06T18:42:59.688 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109188.log.gz 2023-12-06T18:42:59.688 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.123399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92031.log 2023-12-06T18:42:59.688 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123399.log.gz 2023-12-06T18:42:59.689 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.115099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91754.log 2023-12-06T18:42:59.689 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115099.log.gz 2023-12-06T18:42:59.689 INFO:teuthology.orchestra.run.smithi061.stderr:gzip/var/log/ceph/ceph-client.admin.92031.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.92731.log 2023-12-06T18:42:59.689 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92031.log.gz 2023-12-06T18:42:59.689 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119040.log 2023-12-06T18:42:59.689 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91754.log.gz 2023-12-06T18:42:59.689 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98817.log 2023-12-06T18:42:59.689 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92731.log.gz 2023-12-06T18:42:59.689 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.119040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129535.log 2023-12-06T18:42:59.689 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119040.log.gz 2023-12-06T18:42:59.689 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.98817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104576.log 2023-12-06T18:42:59.689 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98817.log.gz 2023-12-06T18:42:59.689 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.129535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92149.log 2023-12-06T18:42:59.689 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129535.log.gz 2023-12-06T18:42:59.689 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.104576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119167.log 2023-12-06T18:42:59.689 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104576.log.gz 2023-12-06T18:42:59.690 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132592.log 2023-12-06T18:42:59.690 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92149.log.gz 2023-12-06T18:42:59.690 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.119167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131933.log 2023-12-06T18:42:59.690 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119167.log.gz 2023-12-06T18:42:59.690 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.132592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91951.log 2023-12-06T18:42:59.690 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132592.log.gz 2023-12-06T18:42:59.690 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121317.log 2023-12-06T18:42:59.690 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131933.log.gz 2023-12-06T18:42:59.690 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127077.log 2023-12-06T18:42:59.690 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91951.log.gz 2023-12-06T18:42:59.690 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.121317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120323.log 2023-12-06T18:42:59.690 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.121317.log.gz 2023-12-06T18:42:59.690 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.127077.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.127077.log.gz --verbose 2023-12-06T18:42:59.690 INFO:teuthology.orchestra.run.smithi061.stderr: -- /var/log/ceph/ceph-client.admin.119612.log 2023-12-06T18:42:59.691 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.120323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124506.log 2023-12-06T18:42:59.691 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120323.log.gz 2023-12-06T18:42:59.691 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.119612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87260.log 2023-12-06T18:42:59.691 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119612.log.gz 2023-12-06T18:42:59.691 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.124506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131076.log 0.0% 2023-12-06T18:42:59.691 INFO:teuthology.orchestra.run.smithi061.stderr: -- replaced with /var/log/ceph/ceph-client.admin.124506.log.gz 2023-12-06T18:42:59.691 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.87260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128941.log 2023-12-06T18:42:59.691 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87260.log.gz 2023-12-06T18:42:59.691 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131076.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-12-06T18:42:59.691 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131076.log.gz 2023-12-06T18:42:59.691 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.128941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131673.log 2023-12-06T18:42:59.691 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128941.log.gz 2023-12-06T18:42:59.691 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131936.log 2023-12-06T18:42:59.691 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91655.log 2023-12-06T18:42:59.691 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131673.log.gz 2023-12-06T18:42:59.692 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120303.log 2023-12-06T18:42:59.692 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131936.log.gz 2023-12-06T18:42:59.692 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115058.log 2023-12-06T18:42:59.692 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91655.log.gz 2023-12-06T18:42:59.692 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.120303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92975.log 2023-12-06T18:42:59.692 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120303.log.gz 2023-12-06T18:42:59.692 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.115058.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115058.log.gz 2023-12-06T18:42:59.692 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132976.log 2023-12-06T18:42:59.692 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103484.log 2023-12-06T18:42:59.692 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92975.log.gz 2023-12-06T18:42:59.692 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.132976.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132976.log.gz -5 2023-12-06T18:42:59.692 INFO:teuthology.orchestra.run.smithi061.stderr: --verbose -- /var/log/ceph/ceph-client.admin.127384.log 2023-12-06T18:42:59.692 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.103484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117606.log 2023-12-06T18:42:59.692 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103484.log.gz 2023-12-06T18:42:59.692 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.127384.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127384.log.gz 2023-12-06T18:42:59.693 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94568.log 2023-12-06T18:42:59.693 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.117606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95523.log 2023-12-06T18:42:59.693 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117606.log.gz 2023-12-06T18:42:59.693 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.94568.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.94568.log.gz 2023-12-06T18:42:59.693 INFO:teuthology.orchestra.run.smithi061.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.131421.log 2023-12-06T18:42:59.693 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.95523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133690.log 2023-12-06T18:42:59.693 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95523.log.gz 2023-12-06T18:42:59.693 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131421.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131421.log.gz -5 2023-12-06T18:42:59.693 INFO:teuthology.orchestra.run.smithi061.stderr: --verbose -- /var/log/ceph/ceph-client.admin.125942.log 2023-12-06T18:42:59.693 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91392.log 2023-12-06T18:42:59.693 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133690.log.gz 2023-12-06T18:42:59.693 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.125942.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.125942.log.gz 2023-12-06T18:42:59.693 INFO:teuthology.orchestra.run.smithi061.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.128498.log 2023-12-06T18:42:59.693 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130957.log 2023-12-06T18:42:59.693 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91392.log.gz 2023-12-06T18:42:59.694 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.128498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128498.log.gz 2023-12-06T18:42:59.694 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110839.log 2023-12-06T18:42:59.694 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.130957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101904.log 2023-12-06T18:42:59.694 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130957.log.gz 2023-12-06T18:42:59.694 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.110839.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.110839.log.gz 2023-12-06T18:42:59.694 INFO:teuthology.orchestra.run.smithi061.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.125053.log 2023-12-06T18:42:59.694 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.101904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120436.log 2023-12-06T18:42:59.694 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101904.log.gz 2023-12-06T18:42:59.694 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.125053.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125053.log.gz 2023-12-06T18:42:59.694 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103260.log 2023-12-06T18:42:59.694 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.120436.log: -- /var/log/ceph/ceph-client.admin.126239.log 2023-12-06T18:42:59.694 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120436.log.gz 2023-12-06T18:42:59.694 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.103260.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.103260.log.gz 2023-12-06T18:42:59.694 INFO:teuthology.orchestra.run.smithi061.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.105334.log 2023-12-06T18:42:59.694 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.126239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133144.log 2023-12-06T18:42:59.695 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126239.log.gz 2023-12-06T18:42:59.695 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.105334.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105334.log.gzgzip 2023-12-06T18:42:59.695 INFO:teuthology.orchestra.run.smithi061.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.117817.log 2023-12-06T18:42:59.695 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131295.log 2023-12-06T18:42:59.695 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133144.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133144.log.gz 2023-12-06T18:42:59.695 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.117817.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117817.log.gz 2023-12-06T18:42:59.695 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118445.log 2023-12-06T18:42:59.695 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93301.log 2023-12-06T18:42:59.695 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131295.log.gz 2023-12-06T18:42:59.695 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.118445.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.118167.log 2023-12-06T18:42:59.695 INFO:teuthology.orchestra.run.smithi061.stderr: -- replaced with /var/log/ceph/ceph-client.admin.118445.log.gz 2023-12-06T18:42:59.695 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91669.log/var/log/ceph/ceph-client.admin.93301.log: 2023-12-06T18:42:59.695 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93301.log.gz 2023-12-06T18:42:59.695 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.118167.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118167.log.gz 2023-12-06T18:42:59.695 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95595.log 2023-12-06T18:42:59.695 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94390.log 2023-12-06T18:42:59.696 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91669.log.gz 2023-12-06T18:42:59.696 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.95595.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95595.log.gz 2023-12-06T18:42:59.696 INFO:teuthology.orchestra.run.smithi061.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.97510.log 2023-12-06T18:42:59.696 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.94390.log: -- /var/log/ceph/ceph-client.admin.93405.log 2023-12-06T18:42:59.696 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.94390.log.gz 2023-12-06T18:42:59.696 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.97510.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97510.log.gz 2023-12-06T18:42:59.696 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125721.log 2023-12-06T18:42:59.696 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.93405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116836.log 2023-12-06T18:42:59.696 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93405.log.gz 2023-12-06T18:42:59.696 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.125721.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125721.log.gz -5 2023-12-06T18:42:59.696 INFO:teuthology.orchestra.run.smithi061.stderr: --verbose -- /var/log/ceph/ceph-client.admin.92327.log 2023-12-06T18:42:59.696 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92267.log 2023-12-06T18:42:59.696 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.116836.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116836.log.gz 2023-12-06T18:42:59.696 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92327.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92327.log.gz 2023-12-06T18:42:59.696 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132631.log 2023-12-06T18:42:59.697 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91394.log 2023-12-06T18:42:59.697 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92267.log.gz 2023-12-06T18:42:59.697 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.132631.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132631.log.gz 2023-12-06T18:42:59.697 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91043.log 2023-12-06T18:42:59.697 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120537.log 2023-12-06T18:42:59.697 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91394.log.gz 2023-12-06T18:42:59.697 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91043.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91043.log.gz 2023-12-06T18:42:59.697 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126793.log 2023-12-06T18:42:59.697 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.120537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133816.log 2023-12-06T18:42:59.697 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120537.log.gz 2023-12-06T18:42:59.697 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.126793.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.126793.log.gz 2023-12-06T18:42:59.697 INFO:teuthology.orchestra.run.smithi061.stderr: --verbose -- /var/log/ceph/ceph-client.admin.96278.log 2023-12-06T18:42:59.697 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.133816.log: /var/log/ceph/ceph-client.admin.117483.log 2023-12-06T18:42:59.697 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133816.log.gz 2023-12-06T18:42:59.698 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.96278.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96278.log.gzgzip 2023-12-06T18:42:59.698 INFO:teuthology.orchestra.run.smithi061.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.91125.log 2023-12-06T18:42:59.698 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.117483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91175.log 2023-12-06T18:42:59.698 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117483.log.gz 2023-12-06T18:42:59.698 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91125.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.91125.log.gz -- 2023-12-06T18:42:59.698 INFO:teuthology.orchestra.run.smithi061.stderr: /var/log/ceph/ceph-client.admin.133438.log 2023-12-06T18:42:59.698 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.91175.log: /var/log/ceph/ceph-client.admin.130912.log 2023-12-06T18:42:59.698 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91175.log.gz 2023-12-06T18:42:59.698 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133438.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.133438.log.gz /var/log/ceph/ceph-client.admin.97570.log 2023-12-06T18:42:59.698 INFO:teuthology.orchestra.run.smithi061.stderr: 2023-12-06T18:42:59.698 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.130912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122843.log 2023-12-06T18:42:59.698 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130912.log.gz 2023-12-06T18:42:59.698 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.97570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125278.log 2023-12-06T18:42:59.698 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97570.log.gz 2023-12-06T18:42:59.698 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132232.log 2023-12-06T18:42:59.699 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.122843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122843.log.gz 2023-12-06T18:42:59.699 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.125278.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125278.log.gz 2023-12-06T18:42:59.699 INFO:teuthology.orchestra.run.smithi061.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.120254.log 2023-12-06T18:42:59.699 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.132232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132224.log 2023-12-06T18:42:59.699 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132232.log.gz 2023-12-06T18:42:59.699 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.120254.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120254.log.gz 2023-12-06T18:42:59.699 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96060.log 2023-12-06T18:42:59.699 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log/var/log/ceph/ceph-client.admin.132224.log: 2023-12-06T18:42:59.699 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132224.log.gz 2023-12-06T18:42:59.699 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.96060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132655.log 0.0% 2023-12-06T18:42:59.699 INFO:teuthology.orchestra.run.smithi061.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96060.log.gz 2023-12-06T18:42:59.699 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94168.log 2023-12-06T18:42:59.699 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.132655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128559.log 2023-12-06T18:42:59.699 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132655.log.gz 2023-12-06T18:42:59.699 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.94168.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.94168.log.gz -- 2023-12-06T18:42:59.700 INFO:teuthology.orchestra.run.smithi061.stderr: /var/log/ceph/ceph-client.admin.128073.log 2023-12-06T18:42:59.700 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122581.log 2023-12-06T18:42:59.700 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.128559.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128559.log.gz 2023-12-06T18:42:59.700 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.128073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95631.log 2023-12-06T18:42:59.700 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128073.log.gz 2023-12-06T18:42:59.700 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.122581.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122581.log.gz 2023-12-06T18:42:59.700 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128156.log 2023-12-06T18:42:59.700 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101604.log 2023-12-06T18:42:59.700 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.95631.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95631.log.gz 2023-12-06T18:42:59.700 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.128156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116650.log 2023-12-06T18:42:59.700 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128156.log.gz 2023-12-06T18:42:59.700 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.101604.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.101604.log.gz -- 2023-12-06T18:42:59.700 INFO:teuthology.orchestra.run.smithi061.stderr: /var/log/ceph/ceph-client.admin.117441.log 2023-12-06T18:42:59.700 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108901.log 2023-12-06T18:42:59.700 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.116650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116650.log.gz 2023-12-06T18:42:59.700 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.117441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118970.log 2023-12-06T18:42:59.701 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117441.log.gz 2023-12-06T18:42:59.701 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125487.log 2023-12-06T18:42:59.701 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.108901.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108901.log.gz 2023-12-06T18:42:59.701 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.118970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125378.log 2023-12-06T18:42:59.701 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118970.log.gz 2023-12-06T18:42:59.701 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.125487.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.125487.log.gz -- 2023-12-06T18:42:59.701 INFO:teuthology.orchestra.run.smithi061.stderr: /var/log/ceph/ceph-client.admin.126329.log 2023-12-06T18:42:59.701 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131127.log 2023-12-06T18:42:59.701 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.125378.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125378.log.gz 2023-12-06T18:42:59.701 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.126329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96891.log 2023-12-06T18:42:59.701 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126329.log.gz 2023-12-06T18:42:59.701 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118677.log 2023-12-06T18:42:59.701 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131127.log.gz 2023-12-06T18:42:59.701 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91764.log 2023-12-06T18:42:59.701 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.96891.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96891.log.gz 2023-12-06T18:42:59.702 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.118677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132531.log 2023-12-06T18:42:59.702 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118677.log.gz 2023-12-06T18:42:59.702 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91764.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.91764.log.gz 2023-12-06T18:42:59.702 INFO:teuthology.orchestra.run.smithi061.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.131547.log 2023-12-06T18:42:59.702 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125771.log 2023-12-06T18:42:59.702 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.132531.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132531.log.gz 2023-12-06T18:42:59.702 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131547.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131547.log.gz 2023-12-06T18:42:59.702 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129601.log 2023-12-06T18:42:59.702 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114969.log 2023-12-06T18:42:59.702 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.125771.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125771.log.gz 2023-12-06T18:42:59.702 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.129601.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129601.log.gz 2023-12-06T18:42:59.702 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126855.log 2023-12-06T18:42:59.702 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86958.log 2023-12-06T18:42:59.702 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.114969.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114969.log.gz 2023-12-06T18:42:59.702 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.126855.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126855.log.gz 2023-12-06T18:42:59.703 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116711.log 2023-12-06T18:42:59.703 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116522.log 2023-12-06T18:42:59.703 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.86958.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.86958.log.gz 2023-12-06T18:42:59.703 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.116711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112388.log 2023-12-06T18:42:59.703 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116711.log.gz 2023-12-06T18:42:59.703 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.116522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102404.log 2023-12-06T18:42:59.703 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116522.log.gz 2023-12-06T18:42:59.703 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.112388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117667.log 2023-12-06T18:42:59.703 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.112388.log.gz 2023-12-06T18:42:59.703 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.102404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114040.log 2023-12-06T18:42:59.703 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102404.log.gz 2023-12-06T18:42:59.703 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.117667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126002.log 2023-12-06T18:42:59.703 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117667.log.gz 2023-12-06T18:42:59.703 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.114040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97274.log 2023-12-06T18:42:59.703 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114040.log.gz 2023-12-06T18:42:59.704 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.126002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124404.log 2023-12-06T18:42:59.704 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126002.log.gz 2023-12-06T18:42:59.704 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.97274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118850.log 2023-12-06T18:42:59.704 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97274.log.gz 2023-12-06T18:42:59.704 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.124404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92773.log 2023-12-06T18:42:59.704 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124404.log.gz 2023-12-06T18:42:59.704 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.118850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131169.log 2023-12-06T18:42:59.704 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118850.log.gz 2023-12-06T18:42:59.704 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131463.log 2023-12-06T18:42:59.704 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92773.log.gz 2023-12-06T18:42:59.704 INFO:teuthology.orchestra.run.smithi061.stderr:gzip/var/log/ceph/ceph-client.admin.131169.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.125577.log 2023-12-06T18:42:59.704 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131169.log.gz 2023-12-06T18:42:59.704 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130485.log 2023-12-06T18:42:59.704 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131463.log.gz 2023-12-06T18:42:59.705 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.125577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117349.log 2023-12-06T18:42:59.705 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125577.log.gz 2023-12-06T18:42:59.705 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.130485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109470.log 2023-12-06T18:42:59.705 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130485.log.gz 2023-12-06T18:42:59.705 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.117349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87487.log 2023-12-06T18:42:59.705 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117349.log.gz 2023-12-06T18:42:59.705 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.109470.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109470.log.gz 2023-12-06T18:42:59.705 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108736.log 2023-12-06T18:42:59.705 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132235.log 2023-12-06T18:42:59.705 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.87487.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.87487.log.gz 2023-12-06T18:42:59.705 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.108736.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108736.log.gz 2023-12-06T18:42:59.705 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102452.log 2023-12-06T18:42:59.706 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119236.log 2023-12-06T18:42:59.706 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.132235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132235.log.gz 2023-12-06T18:42:59.706 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.102452.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102452.log.gz 2023-12-06T18:42:59.706 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128657.log 2023-12-06T18:42:59.706 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.119236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92603.log 2023-12-06T18:42:59.706 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119236.log.gz 2023-12-06T18:42:59.706 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.128657.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.128657.log.gz -5 2023-12-06T18:42:59.706 INFO:teuthology.orchestra.run.smithi061.stderr: --verbose -- /var/log/ceph/ceph-client.admin.90478.log 2023-12-06T18:42:59.706 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114648.log 2023-12-06T18:42:59.706 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92603.log.gz 2023-12-06T18:42:59.706 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.90478.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.90478.log.gz 2023-12-06T18:42:59.706 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102662.log 2023-12-06T18:42:59.706 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95955.log 2023-12-06T18:42:59.707 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.114648.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114648.log.gz 2023-12-06T18:42:59.707 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.102662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117199.log 2023-12-06T18:42:59.707 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102662.log.gz 2023-12-06T18:42:59.707 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.95955.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95955.log.gz 2023-12-06T18:42:59.707 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96138.log 2023-12-06T18:42:59.707 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.117199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108376.log 2023-12-06T18:42:59.707 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117199.log.gz 2023-12-06T18:42:59.707 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.96138.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96138.log.gz 2023-12-06T18:42:59.707 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97012.log 2023-12-06T18:42:59.707 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91381.log 2023-12-06T18:42:59.707 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.108376.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108376.log.gz 2023-12-06T18:42:59.707 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.97012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.97012.log.gz 2023-12-06T18:42:59.707 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132847.log 2023-12-06T18:42:59.707 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132511.log 2023-12-06T18:42:59.708 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91381.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91381.log.gz 2023-12-06T18:42:59.708 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.132847.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132847.log.gz 2023-12-06T18:42:59.708 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104998.log 2023-12-06T18:42:59.708 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.132511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91379.log 2023-12-06T18:42:59.708 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132511.log.gz 2023-12-06T18:42:59.708 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.104998.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.104998.log.gz /var/log/ceph/ceph-mgr.y.log 2023-12-06T18:42:59.708 INFO:teuthology.orchestra.run.smithi061.stderr: 2023-12-06T18:42:59.708 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115825.log 2023-12-06T18:42:59.708 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91379.log.gz 2023-12-06T18:42:59.708 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93575.log 2023-12-06T18:42:59.708 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.115825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122695.log 2023-12-06T18:42:59.708 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115825.log.gz 2023-12-06T18:42:59.708 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.93575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118100.log 2023-12-06T18:42:59.708 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93575.log.gz 2023-12-06T18:42:59.708 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.122695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100483.log 2023-12-06T18:42:59.709 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122695.log.gz 2023-12-06T18:42:59.709 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.118100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130570.log 2023-12-06T18:42:59.709 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118100.log.gz 2023-12-06T18:42:59.709 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.100483.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100483.log.gz 2023-12-06T18:42:59.709 INFO:teuthology.orchestra.run.smithi061.stderr: --verbose -- /var/log/ceph/ceph-client.admin.92977.log 2023-12-06T18:42:59.709 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.130570.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130570.log.gzgzip 2023-12-06T18:42:59.709 INFO:teuthology.orchestra.run.smithi061.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.102026.log 2023-12-06T18:42:59.709 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92977.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92977.log.gz 2023-12-06T18:42:59.709 INFO:teuthology.orchestra.run.smithi061.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.118534.log 2023-12-06T18:42:59.709 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.102026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109230.log 2023-12-06T18:42:59.709 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102026.log.gz 2023-12-06T18:42:59.709 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.118534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119041.log 2023-12-06T18:42:59.709 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118534.log.gz 2023-12-06T18:42:59.709 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.109230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119766.log 2023-12-06T18:42:59.709 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109230.log.gz 2023-12-06T18:42:59.709 INFO:teuthology.orchestra.run.smithi061.stderr:gzip/var/log/ceph/ceph-client.admin.119041.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119041.log.gz 2023-12-06T18:42:59.710 INFO:teuthology.orchestra.run.smithi061.stderr: /var/log/ceph/ceph-client.admin.126369.log 2023-12-06T18:42:59.710 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.119766.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119766.log.gz 2023-12-06T18:42:59.710 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105375.log 2023-12-06T18:42:59.710 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.126369.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126369.log.gz 2023-12-06T18:42:59.710 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116795.log 2023-12-06T18:42:59.710 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109835.log 2023-12-06T18:42:59.710 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.105375.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105375.log.gz 2023-12-06T18:42:59.710 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.116795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127852.log 2023-12-06T18:42:59.710 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.116795.log.gz 2023-12-06T18:42:59.710 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.109835.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109835.log.gz 2023-12-06T18:42:59.710 INFO:teuthology.orchestra.run.smithi061.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.115542.log 2023-12-06T18:42:59.710 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.127852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109511.log 2023-12-06T18:42:59.710 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127852.log.gz 2023-12-06T18:42:59.710 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.115542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99553.log 2023-12-06T18:42:59.711 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.115542.log.gz 2023-12-06T18:42:59.711 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.109511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130267.log 2023-12-06T18:42:59.711 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109511.log.gz 2023-12-06T18:42:59.711 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.99553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101223.log 2023-12-06T18:42:59.711 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99553.log.gz 2023-12-06T18:42:59.711 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.130267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119510.log 2023-12-06T18:42:59.711 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130267.log.gz 2023-12-06T18:42:59.711 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.101223.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.101223.log.gz 2023-12-06T18:42:59.711 INFO:teuthology.orchestra.run.smithi061.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.126146.log 2023-12-06T18:42:59.711 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92561.log 2023-12-06T18:42:59.711 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.119510.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119510.log.gz 2023-12-06T18:42:59.711 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.126146.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126146.log.gz 2023-12-06T18:42:59.711 INFO:teuthology.orchestra.run.smithi061.stderr: -- /var/log/ceph/ceph-client.admin.104536.log 2023-12-06T18:42:59.711 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92561.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92561.log.gz 2023-12-06T18:42:59.711 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129966.log 2023-12-06T18:42:59.711 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.104536.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104536.log.gz 2023-12-06T18:42:59.712 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117370.log 2023-12-06T18:42:59.712 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.129966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91931.log 2023-12-06T18:42:59.712 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129966.log.gz 2023-12-06T18:42:59.712 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.117370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99430.log 2023-12-06T18:42:59.712 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117370.log.gz 2023-12-06T18:42:59.712 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100249.log 2023-12-06T18:42:59.712 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91931.log.gz 2023-12-06T18:42:59.712 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.99430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95821.log 2023-12-06T18:42:59.712 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99430.log.gz 2023-12-06T18:42:59.712 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.100249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92407.log 0.0% 2023-12-06T18:42:59.712 INFO:teuthology.orchestra.run.smithi061.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100249.log.gz 2023-12-06T18:42:59.712 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92972.log 2023-12-06T18:42:59.712 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.95821.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95821.log.gz 2023-12-06T18:42:59.712 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133900.log 2023-12-06T18:42:59.713 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92407.log.gz 2023-12-06T18:42:59.713 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92972.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-12-06T18:42:59.713 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92972.log.gz 2023-12-06T18:42:59.713 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88075.log 2023-12-06T18:42:59.713 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133900.log.gz 2023-12-06T18:42:59.713 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102941.log 2023-12-06T18:42:59.713 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.88075.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.88075.log.gz 2023-12-06T18:42:59.713 INFO:teuthology.orchestra.run.smithi061.stderr: --verbose -- /var/log/ceph/ceph-client.admin.92200.log 2023-12-06T18:42:59.713 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.102941.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102941.log.gz 2023-12-06T18:42:59.713 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103061.log 2023-12-06T18:42:59.713 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92200.log.gz 2023-12-06T18:42:59.713 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120810.log 2023-12-06T18:42:59.713 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.103061.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103061.log.gz 2023-12-06T18:42:59.713 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132145.log 2023-12-06T18:42:59.713 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.120810.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120810.log.gz 2023-12-06T18:42:59.713 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132633.log 2023-12-06T18:42:59.714 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.132145.log: /var/log/ceph/ceph-client.admin.128861.log 2023-12-06T18:42:59.714 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132145.log.gz 2023-12-06T18:42:59.714 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.132633.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132633.log.gz 2023-12-06T18:42:59.714 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92835.log 2023-12-06T18:42:59.714 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126529.log 2023-12-06T18:42:59.714 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.128861.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.128861.log.gz 2023-12-06T18:42:59.714 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109712.log 2023-12-06T18:42:59.714 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92835.log: /var/log/ceph/ceph-client.admin.126529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105254.log 2023-12-06T18:42:59.714 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92835.log.gz 2023-12-06T18:42:59.714 INFO:teuthology.orchestra.run.smithi061.stderr: -- replaced with /var/log/ceph/ceph-client.admin.126529.log.gz 2023-12-06T18:42:59.714 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.109712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93467.log 2023-12-06T18:42:59.714 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109712.log.gz 2023-12-06T18:42:59.714 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.105254.log: -- /var/log/ceph/ceph-client.admin.99830.log 2023-12-06T18:42:59.714 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105254.log.gz 2023-12-06T18:42:59.715 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.93467.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.93467.log.gz 2023-12-06T18:42:59.715 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131939.log 2023-12-06T18:42:59.715 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-12-06T18:42:59.715 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.99830.log: /var/log/ceph/ceph-client.admin.131939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133858.log 2023-12-06T18:42:59.715 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99830.log.gz 2023-12-06T18:42:59.715 INFO:teuthology.orchestra.run.smithi061.stderr: -- replaced with /var/log/ceph/ceph-client.admin.131939.log.gz 2023-12-06T18:42:59.715 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103723.log 2023-12-06T18:42:59.715 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120099.log 2023-12-06T18:42:59.715 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133858.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133858.log.gz 2023-12-06T18:42:59.715 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.103723.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103723.log.gzgzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.81107.log 2023-12-06T18:42:59.715 INFO:teuthology.orchestra.run.smithi061.stderr: 2023-12-06T18:42:59.716 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.120099.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120099.log.gz 2023-12-06T18:42:59.716 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131631.log 2023-12-06T18:42:59.716 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-12-06T18:42:59.716 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph.tmp-client.admin.81107.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.81107.log.gz 2023-12-06T18:42:59.716 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131631.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131631.log.gz 2023-12-06T18:42:59.716 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133102.log 2023-12-06T18:42:59.716 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-osd.2.log: 93.8% -- replaced with /var/log/ceph/ceph.log.gz 2023-12-06T18:42:59.716 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108448.log 2023-12-06T18:42:59.716 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96190.log 2023-12-06T18:42:59.716 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133102.log.gz 2023-12-06T18:42:59.716 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.108448.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108448.log.gz 2023-12-06T18:42:59.716 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109651.log 2023-12-06T18:42:59.716 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.96190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122058.log 2023-12-06T18:42:59.716 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96190.log.gz 2023-12-06T18:42:59.717 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.109651.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.109651.log.gz 2023-12-06T18:42:59.717 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108152.log 2023-12-06T18:42:59.717 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.122058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117796.log 2023-12-06T18:42:59.717 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122058.log.gz 2023-12-06T18:42:59.717 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.108152.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108152.log.gz 2023-12-06T18:42:59.717 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103804.log 2023-12-06T18:42:59.717 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.117796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119400.log 2023-12-06T18:42:59.717 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.117796.log.gz 2023-12-06T18:42:59.717 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.103804.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103804.log.gz 2023-12-06T18:42:59.717 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96106.log 2023-12-06T18:42:59.717 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.119400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118799.log 2023-12-06T18:42:59.717 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.119400.log.gz 2023-12-06T18:42:59.717 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.96106.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96106.log.gz 2023-12-06T18:42:59.717 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133186.log 2023-12-06T18:42:59.717 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.118799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130999.log 2023-12-06T18:42:59.718 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.118799.log.gz 2023-12-06T18:42:59.718 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133186.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133186.log.gz 2023-12-06T18:42:59.718 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98293.log 2023-12-06T18:42:59.718 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.130999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129580.log 2023-12-06T18:42:59.718 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130999.log.gz 2023-12-06T18:42:59.718 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.98293.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98293.log.gz 2023-12-06T18:42:59.718 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129424.log 2023-12-06T18:42:59.718 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.129580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96555.log 2023-12-06T18:42:59.718 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129580.log.gz 2023-12-06T18:42:59.718 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.129424.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129424.log.gz 2023-12-06T18:42:59.718 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100016.log 2023-12-06T18:42:59.718 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.96555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91217.log 2023-12-06T18:42:59.719 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96555.log.gz 2023-12-06T18:42:59.719 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.100016.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.100016.log.gz 2023-12-06T18:42:59.719 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104895.log 2023-12-06T18:42:59.719 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.91217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124706.log 2023-12-06T18:42:59.719 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.91217.log.gz 2023-12-06T18:42:59.719 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.104895.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.104895.log.gz 2023-12-06T18:42:59.719 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120852.log 2023-12-06T18:42:59.719 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.124706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98652.log 2023-12-06T18:42:59.719 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124706.log.gz 2023-12-06T18:42:59.719 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.120852.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120852.log.gz 2023-12-06T18:42:59.719 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95354.log 2023-12-06T18:42:59.719 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.98652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103300.log 2023-12-06T18:42:59.719 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.98652.log.gz 2023-12-06T18:42:59.719 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.95354.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.95354.log.gzgzip 2023-12-06T18:42:59.719 INFO:teuthology.orchestra.run.smithi061.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.102839.log 2023-12-06T18:42:59.720 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.103300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126635.log 2023-12-06T18:42:59.720 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103300.log.gz 2023-12-06T18:42:59.720 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.102839.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.102839.log.gz 2023-12-06T18:42:59.720 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92519.log 2023-12-06T18:42:59.720 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.126635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133060.log 2023-12-06T18:42:59.720 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126635.log.gz 2023-12-06T18:42:59.720 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92519.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92519.log.gz 2023-12-06T18:42:59.720 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131505.log 2023-12-06T18:42:59.720 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122239.log 2023-12-06T18:42:59.720 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133060.log.gz 2023-12-06T18:42:59.720 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131505.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131505.log.gz 2023-12-06T18:42:59.720 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122439.log 2023-12-06T18:42:59.720 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.122239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126282.log 2023-12-06T18:42:59.720 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122239.log.gz 2023-12-06T18:42:59.720 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.122439.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122439.log.gz 2023-12-06T18:42:59.721 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131337.log 2023-12-06T18:42:59.721 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.126282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114505.log 2023-12-06T18:42:59.721 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126282.log.gz 2023-12-06T18:42:59.721 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131337.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131337.log.gz 2023-12-06T18:42:59.721 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131042.log 2023-12-06T18:42:59.721 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.114505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107952.log 2023-12-06T18:42:59.721 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.114505.log.gz 2023-12-06T18:42:59.721 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131042.log.gz 2023-12-06T18:42:59.721 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92969.log 2023-12-06T18:42:59.721 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.107952.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.107952.log.gz 2023-12-06T18:42:59.721 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96729.log 2023-12-06T18:42:59.721 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130443.log 2023-12-06T18:42:59.721 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.92969.log.gz 2023-12-06T18:42:59.721 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.96729.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96729.log.gz 2023-12-06T18:42:59.721 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133396.log 2023-12-06T18:42:59.722 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.130443.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.130443.log.gz 2023-12-06T18:42:59.722 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122240.log 2023-12-06T18:42:59.722 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.133396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108314.log 2023-12-06T18:42:59.722 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.133396.log.gz 2023-12-06T18:42:59.722 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.122240.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.122240.log.gz 2023-12-06T18:42:59.722 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125680.log 2023-12-06T18:42:59.722 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.108314.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.108314.log.gz 2023-12-06T18:42:59.722 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124808.log 2023-12-06T18:42:59.722 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.125680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132295.log 2023-12-06T18:42:59.722 INFO:teuthology.orchestra.run.smithi061.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125680.log.gz 2023-12-06T18:42:59.722 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.124808.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.124808.log.gz 2023-12-06T18:42:59.722 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123499.log 2023-12-06T18:42:59.722 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125101.log 2023-12-06T18:42:59.723 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.132295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.132295.log.gz 2023-12-06T18:42:59.723 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.123499.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.123499.log.gz 2023-12-06T18:42:59.723 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96684.log 2023-12-06T18:42:59.723 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.125101.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.125101.log.gz 2023-12-06T18:42:59.723 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127990.log 2023-12-06T18:42:59.723 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103382.log 2023-12-06T18:42:59.723 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.96684.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.96684.log.gz 2023-12-06T18:42:59.723 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.127990.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127990.log.gz 2023-12-06T18:42:59.723 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99199.log 2023-12-06T18:42:59.723 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.103382.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.103382.log.gz 2023-12-06T18:42:59.723 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92011.log 2023-12-06T18:42:59.723 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126856.log 2023-12-06T18:42:59.723 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.99199.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.99199.log.gz 2023-12-06T18:42:59.723 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.92011.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120661.log 2023-12-06T18:42:59.723 INFO:teuthology.orchestra.run.smithi061.stderr: -- replaced with /var/log/ceph/ceph-client.admin.92011.log.gz 2023-12-06T18:42:59.724 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.126856.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.126856.log.gz 2023-12-06T18:42:59.724 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131715.log 2023-12-06T18:42:59.724 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-12-06T18:42:59.724 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.120661.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.120661.log.gz 2023-12-06T18:42:59.724 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.131715.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.131715.log.gz 2023-12-06T18:42:59.724 INFO:teuthology.orchestra.run.smithi061.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129211.log 2023-12-06T18:42:59.724 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127258.log 2023-12-06T18:42:59.724 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.129211.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.129211.log.gz 2023-12-06T18:42:59.724 INFO:teuthology.orchestra.run.smithi061.stderr:/var/log/ceph/ceph-client.admin.127258.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.127258.log.gz 2023-12-06T18:42:59.725 INFO:teuthology.orchestra.run.smithi152.stderr: 90.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-12-06T18:42:59.725 INFO:teuthology.orchestra.run.smithi152.stderr: 90.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-12-06T18:42:59.725 INFO:teuthology.orchestra.run.smithi152.stderr: 90.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-12-06T18:42:59.744 INFO:teuthology.orchestra.run.smithi061.stderr: 90.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-12-06T18:42:59.775 INFO:teuthology.orchestra.run.smithi061.stderr: 90.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-12-06T18:42:59.777 INFO:teuthology.orchestra.run.smithi061.stderr: 90.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-12-06T18:42:59.792 INFO:teuthology.orchestra.run.smithi061.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-12-06T18:42:59.793 INFO:teuthology.orchestra.run.smithi061.stderr: 90.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-12-06T18:42:59.830 INFO:teuthology.orchestra.run.smithi152.stderr: 90.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-12-06T18:42:59.831 INFO:teuthology.orchestra.run.smithi152.stderr: 2023-12-06T18:42:59.832 INFO:teuthology.orchestra.run.smithi152.stderr:real 0m0.366s 2023-12-06T18:42:59.832 INFO:teuthology.orchestra.run.smithi152.stderr:user 0m1.098s 2023-12-06T18:42:59.832 INFO:teuthology.orchestra.run.smithi152.stderr:sys 0m0.060s 2023-12-06T18:42:59.981 INFO:teuthology.orchestra.run.smithi061.stderr: 89.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-12-06T18:42:59.983 INFO:teuthology.orchestra.run.smithi061.stderr: 90.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-12-06T18:42:59.985 INFO:teuthology.orchestra.run.smithi061.stderr: 2023-12-06T18:42:59.986 INFO:teuthology.orchestra.run.smithi061.stderr:real 0m0.520s 2023-12-06T18:42:59.986 INFO:teuthology.orchestra.run.smithi061.stderr:user 0m1.920s 2023-12-06T18:42:59.986 INFO:teuthology.orchestra.run.smithi061.stderr:sys 0m0.263s 2023-12-06T18:42:59.986 INFO:tasks.ceph:Archiving logs... 2023-12-06T18:42:59.986 DEBUG:teuthology.misc:Transferring archived files from smithi061:/var/log/ceph to /home/teuthworker/archive/sseshasa-2023-12-06_17:50:03-rados-wip-sseshasa-testing-2023-12-04-1848-distro-default-smithi/7480579/remote/smithi061/log 2023-12-06T18:42:59.987 DEBUG:teuthology.orchestra.run.smithi061:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-06T18:43:00.665 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/log/ceph to /home/teuthworker/archive/sseshasa-2023-12-06_17:50:03-rados-wip-sseshasa-testing-2023-12-04-1848-distro-default-smithi/7480579/remote/smithi152/log 2023-12-06T18:43:00.666 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-06T18:43:00.863 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1112, in cluster yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1449, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/misc.py", line 1173, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 329, in valgrind_post yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1181, in cluster raise e File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1163, in cluster remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi152 with status 32: 'sync && sudo umount -f /var/lib/ceph/osd/ceph-6' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1935, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 252, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: Leak_StillReachable operator new[](unsigned long) allocate allocate 2023-12-06T18:43:00.864 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-06T18:43:00.876 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1112, in cluster yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1449, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/misc.py", line 1173, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 329, in valgrind_post yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1181, in cluster raise e File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1163, in cluster remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi152 with status 32: 'sync && sudo umount -f /var/lib/ceph/osd/ceph-6' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1935, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 252, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: Leak_StillReachable operator new[](unsigned long) allocate allocate 2023-12-06T18:43:00.876 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-06T18:43:00.877 DEBUG:teuthology.orchestra.run.smithi061:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-06T18:43:00.879 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-06T18:43:00.958 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-06T18:43:00.968 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-06T18:43:00.968 DEBUG:teuthology.orchestra.run.smithi061:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-06T18:43:00.971 DEBUG:teuthology.orchestra.run.smithi152:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-06T18:43:00.984 INFO:teuthology.orchestra.run.smithi061.stderr:bash: line 1: ntpq: command not found 2023-12-06T18:43:00.985 INFO:teuthology.orchestra.run.smithi152.stderr:bash: line 1: ntpq: command not found 2023-12-06T18:43:00.989 INFO:teuthology.orchestra.run.smithi061.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-06T18:43:00.989 INFO:teuthology.orchestra.run.smithi061.stdout:=============================================================================== 2023-12-06T18:43:00.989 INFO:teuthology.orchestra.run.smithi061.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 14 +618us[ +633us] +/- 85ms 2023-12-06T18:43:00.989 INFO:teuthology.orchestra.run.smithi061.stdout:^* hv02.front.sepia.ceph.com 3 6 377 11 -2240us[-2226us] +/- 65ms 2023-12-06T18:43:00.989 INFO:teuthology.orchestra.run.smithi061.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +2582us[+2597us] +/- 93ms 2023-12-06T18:43:00.989 INFO:teuthology.orchestra.run.smithi061.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T18:43:00.990 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-06T18:43:00.991 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2023-12-06T18:43:00.991 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 14 +552us[ +562us] +/- 85ms 2023-12-06T18:43:00.991 INFO:teuthology.orchestra.run.smithi152.stdout:^* hv02.front.sepia.ceph.com 3 6 377 11 -2295us[-2286us] +/- 65ms 2023-12-06T18:43:00.991 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +2528us[+2537us] +/- 93ms 2023-12-06T18:43:00.991 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-06T18:43:00.991 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-06T18:43:01.003 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-06T18:43:01.003 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-06T18:43:01.016 DEBUG:teuthology.orchestra.run.smithi061:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-06T18:43:01.096 DEBUG:teuthology.orchestra.run.smithi152:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-06T18:43:01.138 DEBUG:teuthology.orchestra.run.smithi061:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-06T18:43:01.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-06T18:43:01.173 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-06T18:43:01.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-06T18:43:01.205 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-06T18:43:01.215 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-06T18:43:01.230 INFO:teuthology.task.internal:Duration was 2239.458822 seconds 2023-12-06T18:43:01.231 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-06T18:43:01.243 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-06T18:43:01.243 DEBUG:teuthology.orchestra.run.smithi061:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-06T18:43:01.245 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-06T18:43:01.289 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-06T18:43:01.294 INFO:teuthology.orchestra.run.smithi061.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-06T18:43:01.676 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-06T18:43:01.676 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi061.front.sepia.ceph.com 2023-12-06T18:43:01.676 DEBUG:teuthology.orchestra.run.smithi061:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-06T18:43:01.697 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi152.front.sepia.ceph.com 2023-12-06T18:43:01.697 DEBUG:teuthology.orchestra.run.smithi152:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-06T18:43:01.721 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-06T18:43:01.721 DEBUG:teuthology.orchestra.run.smithi061:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-06T18:43:01.740 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-06T18:43:01.789 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-06T18:43:01.789 DEBUG:teuthology.orchestra.run.smithi061:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-06T18:43:01.813 DEBUG:teuthology.orchestra.run.smithi152:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-06T18:43:01.966 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-06T18:43:01.976 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-06T18:43:01.976 DEBUG:teuthology.orchestra.run.smithi061:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-06T18:43:02.006 DEBUG:teuthology.orchestra.run.smithi152:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-06T18:43:02.034 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-06T18:43:02.044 DEBUG:teuthology.orchestra.run.smithi061:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-06T18:43:02.050 DEBUG:teuthology.orchestra.run.smithi152:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-06T18:43:02.076 INFO:teuthology.orchestra.run.smithi061.stdout:kernel.core_pattern = core 2023-12-06T18:43:02.100 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = core 2023-12-06T18:43:02.161 DEBUG:teuthology.orchestra.run.smithi061:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-06T18:43:02.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-06T18:43:02.175 DEBUG:teuthology.orchestra.run.smithi152:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-06T18:43:02.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-06T18:43:02.215 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-06T18:43:02.235 INFO:teuthology.task.internal:Transferring archived files... 2023-12-06T18:43:02.236 DEBUG:teuthology.misc:Transferring archived files from smithi061:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/sseshasa-2023-12-06_17:50:03-rados-wip-sseshasa-testing-2023-12-04-1848-distro-default-smithi/7480579/remote/smithi061 2023-12-06T18:43:02.236 DEBUG:teuthology.orchestra.run.smithi061:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-06T18:43:02.273 DEBUG:teuthology.misc:Transferring archived files from smithi152:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/sseshasa-2023-12-06_17:50:03-rados-wip-sseshasa-testing-2023-12-04-1848-distro-default-smithi/7480579/remote/smithi152 2023-12-06T18:43:02.273 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-06T18:43:02.306 INFO:teuthology.task.internal:Removing archive directory... 2023-12-06T18:43:02.307 DEBUG:teuthology.orchestra.run.smithi061:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-06T18:43:02.310 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-06T18:43:02.356 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-06T18:43:02.369 INFO:teuthology.task.internal:Not uploading archives. 2023-12-06T18:43:02.369 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-06T18:43:02.385 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-06T18:43:02.385 DEBUG:teuthology.orchestra.run.smithi061:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-06T18:43:02.387 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-06T18:43:02.399 INFO:teuthology.orchestra.run.smithi061.stdout: 262166 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Dec 6 18:43 /home/ubuntu/cephtest 2023-12-06T18:43:02.400 INFO:teuthology.orchestra.run.smithi061.stdout: 655676 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 6 18:25 /home/ubuntu/cephtest/ceph.data 2023-12-06T18:43:02.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-06T18:43:02.401 INFO:teuthology.orchestra.run.smithi061.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-12-06T18:43:02.401 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1112, in cluster yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1449, in run_daemon teuthology.stop_daemons_of_type(ctx, type_, cluster_name) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/misc.py", line 1173, in stop_daemons_of_type daemon.stop() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 329, in valgrind_post yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1181, in cluster raise e File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1163, in cluster remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi152 with status 32: 'sync && sudo umount -f /var/lib/ceph/osd/ceph-6' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 1935, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 252, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_96fa2578c622ca9c169af6f426ca8b1616130e58/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: Leak_StillReachable operator new[](unsigned long) allocate allocate During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_f2de108beb0a6dde7f3e6c96039071828a1f6269/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi061 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-12-06T18:43:02.402 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-06T18:43:02.412 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-06T18:43:02.424 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: Exception: valgrind error: Leak_StillReachable operator new[](unsigned long) allocate allocate 2023-12-06T18:43:02.424 INFO:teuthology.nuke:Checking targets against current locks 2023-12-06T18:43:02.428 INFO:teuthology.orchestra.run.smithi152.stdout: 265244 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Dec 6 18:43 /home/ubuntu/cephtest 2023-12-06T18:43:02.428 INFO:teuthology.orchestra.run.smithi152.stdout: 655390 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 6 18:25 /home/ubuntu/cephtest/ceph.data 2023-12-06T18:43:02.430 INFO:teuthology.orchestra.run.smithi152.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-12-06T18:43:02.454 DEBUG:teuthology.nuke:shortname: smithi061 2023-12-06T18:43:02.454 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-06T18:43:02.477 DEBUG:teuthology.nuke:shortname: smithi152 2023-12-06T18:43:02.477 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-06T18:43:02.481 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi061.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sseshasa-2023-12-06_17:50:03-rados-wip-sseshasa-testing-2023-12-04-1848-distro-default-smithi/7480579', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-06 17:52:29.601430', 'locked_by': 'scheduled_sseshasa@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOOmUHjTe45/NHixBF/csb/zjzk1zdv1ekMBaylRiQCGZ9jpEg1Wr/laHEkskFVFyHo59Ut5qZjChz0i6UlJ+Gw='} 2023-12-06T18:43:02.505 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sseshasa-2023-12-06_17:50:03-rados-wip-sseshasa-testing-2023-12-04-1848-distro-default-smithi/7480579', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-06 17:52:29.604306', 'locked_by': 'scheduled_sseshasa@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMB/1Zaef7NTky5wA6AE8OcK28GIDiVUGrOtfy53WeFfxKrMrUh4n/UUWyealTa1cbnzMBnXCmaa32+rVaKVxfM='} 2023-12-06T18:43:02.537 INFO:teuthology.orchestra.console.smithi061:Power off 2023-12-06T18:43:02.537 DEBUG:teuthology.orchestra.console.smithi061:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-06T18:43:02.554 INFO:teuthology.orchestra.console.smithi152:Power off 2023-12-06T18:43:02.554 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-06T18:43:02.563 DEBUG:teuthology.orchestra.console.smithi061:power off output: Chassis Power Control: Down/Off 2023-12-06T18:43:02.563 DEBUG:teuthology.orchestra.console.smithi061:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T18:43:02.579 DEBUG:teuthology.orchestra.console.smithi152:power off output: Chassis Power Control: Down/Off 2023-12-06T18:43:02.579 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T18:43:02.590 DEBUG:teuthology.orchestra.console.smithi061:check power output: Chassis Power is on 2023-12-06T18:43:02.602 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2023-12-06T18:43:06.592 DEBUG:teuthology.orchestra.console.smithi061:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T18:43:06.603 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T18:43:06.702 DEBUG:teuthology.orchestra.console.smithi061:check power output: Chassis Power is on 2023-12-06T18:43:06.712 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2023-12-06T18:43:09.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-12-06T18:43:09.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-12-06T18:43:09.198 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-12-06T18:43:10.703 DEBUG:teuthology.orchestra.console.smithi061:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T18:43:10.713 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T18:43:10.813 DEBUG:teuthology.orchestra.console.smithi061:check power output: Chassis Power is on 2023-12-06T18:43:10.822 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2023-12-06T18:43:14.202 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~5s 2023-12-06T18:43:14.202 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~5s 2023-12-06T18:43:14.203 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~5s 2023-12-06T18:43:14.815 DEBUG:teuthology.orchestra.console.smithi061:pexpect command: ipmitool -H smithi061.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T18:43:14.824 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-06T18:43:14.924 DEBUG:teuthology.orchestra.console.smithi061:check power output: Chassis Power is off 2023-12-06T18:43:14.932 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is off 2023-12-06T18:43:15.026 INFO:teuthology.orchestra.console.smithi061:Power off completed 2023-12-06T18:43:15.033 INFO:teuthology.orchestra.console.smithi152:Power off completed 2023-12-06T18:43:15.138 INFO:teuthology.run:Summary data: description: rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/mon_recovery validater/valgrind} duration: 2239.458822250366 failure_reason: 'valgrind error: Leak_StillReachable operator new[](unsigned long) allocate allocate' flavor: default owner: scheduled_sseshasa@teuthology status: fail success: false 2023-12-06T18:43:15.138 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-06T18:43:15.231 INFO:teuthology.run:FAIL